Web&: An Architecture for Non-Interactive Web - CiteSeerX

Web&: An Architecture for Non-Interactive Web S. H. Phatak, V. Esakki, B. R. Badrinath and L. Iftode Department of Computer Science Busch Campus, Rutgers University Frelinghuysen Road Piscataway, NJ 08855 e-mail:{phatak,esakki,badri,iftode}@cs.rutgers.edu Abstract As the use of the world wide web grows, the profile of web use has changed significantly. Web use has gone from purely information access to e-commerce. The interactive (synchronous) model which the world wide web currently supports is time consuming. Web browsing can easily become a frustrating experience if the network or servers incur delays, are not accessible, or the same interaction must be repeated many times. This model is even more painful for clients who are disconnected most of the time or are accessing the web via low bandwidth connections and resource constrained devices such as PDAs. Therefore, there is a legitimate need to support non-interactive, (asynchronous) transactions between client and servers on the web. A non-interactive model is more efficient because (i) there is reduced sensitivity to network latencies/unavailability, (ii) web activities can be performed concurrently and (iii) client mobility can be easily supported. In this paper, we propose a novel non-interactive service architecture for the web called Web&. The architecture incorporates server and service discovery, support for disconnected and heterogeneous clients, web transactions via a uniform server interface, and persistent client state. We also present a prototype that we have implemented using JAVA, XML and JDBC. The prototype consists of client proxies that can store client state and perform web queries on a client’s behalf, and server proxies that provides a uniform XML based interface to servers on the web. A directory service provides a schema server and service discovery to the system. Our experiments indicate that even on a single processor Sun workstation with 512M we can support 25000 tasks with a peak throughput of 150 tasks per second. Two applications IsItLate for flight tracking and IsItThere for package tracking are currently running on the prototype.

1

1

Motivation

The number of users using the world wide web has seen exponential growth in the last few years. As this number grows, the profile of such users is changing from home users surfing the web at leisure to mobile users who are always under time pressure when accessing the web. For these people, interactive web is often painful because it consumes time and needs attention. What they would like to have instead is a non-interactive architecture that accepts queries, finds the appropriate servers and performs web tasks without user intervention. In addition, more and more potential users are using devices that are not always connected to the web, such as laptops and PDAs. Such users often need access to the web, but are forced to wait until they can connect to the web either through dial up lines or through fixed terminals. This is unfortunate since many of the tasks that such a user may perform could often be performed without the users intervention. As the profile of users is changing, the use of web data is also changing. In the past much of web use was related to recreational or soft data. Today much of web use has migrated to hard or critical data. This is exemplified by the growing popularity of e-commerce sites. Often interactive web browsing involves long waits (hence the acronym world wide wait) and needlessly repetitive tasks. As data becomes more and more critical users are forced to spend more and more fruitless time in such non-productive tasks. Consider, for example, a user who wants to buy tickets for a baseball game from an e-commerce site on the net. A variety of circumstances can lead to a frustrating web experience: the site might be inaccessible or slow; the user might be using a PDA or other device that is incapable of rendering the site’s home page; the tickets may not be available until a later date/time, which may be unknown at present; the user may not want to be online until a window of opportunity to actually complete the purchase manifests itself. The question is how to automate such a web transaction to the point where it can be performed when the user is offline. In this paper we propose a new architecture that allows users to perform tasks on the world wide web in a non-interactive fashion. This architecture addresses the problems created by interactive web browsing. Our goal is to provide non-interactive users with an online experience and to relieve the frustration of interactive users performing repetitive tasks. We achieve this by interposing an intermediate layer of proxies between the client and server that allows us to replace the synchronous client server interaction with an asynchronous interaction between the client and the proxies and moving the synchronous interaction between the proxies and the server. In doing so we have created a new paradigm for web interaction. In the above example, this new layer of proxies would take over the task of actually purchasing the tickets. The user would no longer be required to control the interaction. Thus, the user can disconnect and reconnect at leisure to check the status of this task. The proxies are always online and can avail themselves of any window of opportunity to buy the tickets. In addition, the layer reformats/interprets results so that they can be accessed by a variety of clients in a device independent fashion.

2

Client

Client

Server

Web&

reque

requ

st

est

requ

est

ack

e

ons

p res

Server

stat

us?

se

pon

res

se

respon

Synchronous

Asynchronous

Figure 1: Client Server Interaction for Synchronous and Asynchronous Models

2

The Non-Interactive Web Model

Traditionally, the web interaction model is synchronous (see Figure 1) which means that client and server have to be online for any web transaction to be performed. The client manually controls the transaction and keeps the state. In case the server is not accessible or fails, the client will repeat the query again possibly with a different server. Interactive web is easy to support but can be extremely inefficient when user wants to perform many different queries at the same time or repeat, condition or compose queries. To alleviate some of the problems related to fully interactive synchronous web interaction, we propose an non-interactive asynchronous model for web interaction called Web&1 . Web& mediates between the client and the server and relieves the client of having to stay online to control the interaction with the server. However, in the context of the web support for executing tasks asynchronously, every web paradigm must be revisited. The Web& execution model is based on four primitive entities: services, tasks, actions and sessions. Each is defined in the subsections that follow. Each entity 1

& is inspired by the background shell operator in UNIX [10], to which our non-interactive web is an analogue.

3

encapsulates a facet of an always interactive user and together these provide us the building blocks required to model interactions between the user and the web in the absence of the user. Together these building blocks provide us a simulation of user behavior. Obviously, the power of a web& implementation depends on how comprehensively and realistically these blocks simulate a real always connected user.

2.1

Service

A service is an abstraction for a set of analogous operations offered by a group of web servers. A service is characterized by a Uniform Service Descriptor which provides a uniform way to specify and invoke the operations and provide results in a standard form. For example, many different servers might provide the operation of buying a book. However, each server provides a different interface (HTML form) and a different sequence of steps to complete the purchase. A corresponding Book Purchase service would provide a uniform interface that given a book and server would do the purchase. In addition the result (i.e. whether the book was bought or not) would be available in a standard form irrespective of the web server used. A service is the very basic unit of web-interaction in our system. In particular each service models one or more key clicks from the user when he/she is performing the underlying operations. A service may thus be used as a uniform one click alternative by the user for the underlying operations. However, the great power of the service model comes from the fact that services provide a uniform interface to the web servers, which implies services may be executed by a program. The service also provides a convenient abstraction for web servers allowing us to smoothly transition with progress in content formats, such as the introduction of XML. A service is analogous to the UNIX system call. It is invoked with input parameters which cause the web servers to operate and respond in a particular fashion. As a result of these operations it provides a return value as a form.

2.2

Task

A task is an instance of execution of a set of services on behalf of the user. A task models the sequence of operations requested by a user in an interactive web session. Note that while a service is an passive abstraction and is thus stateless, a task is an active entity which represents an invocation of the component services and has state information associated with it. Furthermore, a task may be scheduled for execution at any arbitrary time in the future. A task is analogous to the operating system notion of a process. It is an active entity that has associated execution state. Tasks are usually ephemeral and disappear as soon as the requested services are completed leaving behind the results of the execution of the services.

4

2.3

Action

An action is performed as a consequence of the execution of a task. The action can manipulate the state of a session (to be defined later), create and delete sessions or generate notifications to a user. An action models the reaction of a user to the result of a web interaction. In addition, an action may also trigger notifications to a disconnected user. This additional facet is required since unlike the interactive web model, the user is not available to monitor the results, and thus may need to be informed about these results. Usually, actions are also associated with conditions on the results of a task execution that define when an action must be invoked. This models the fact that users will take various actions if the results from an interactive session satisfy certain conditions. The conditions are constraints on the form, content or structure of the results of the task execution. The actions are analogous to user I/O in the Unix operating environment. When a task completes execution an action inputs “commands” to the system based on the results (output) of the task execution.

2.4

Session

A session is a user defined entity which consists of three components: 1. A set of tasks 2. A set of (conditional) actions 3. A set of scheduling constraints on the tasks The set of tasks defines the operations the user wants performed on his/her behalf. The conditional actions are performed only if the results of a task satisfy the conditions. Finally the scheduling constraints define when the tasks are to be performed. The session models for complete instance of a series of web interactions in an interactive web session. Each session can thus be considered a substitute for an always connected user. A user/application using a web& system interacts with the system by creating/deleting/modifying sessions on the system. The system in turn provides services, schedules and executes tasks, and performs actions based on the session parameters as specified by the user/application. The session is analogous to the UNIX background session that acts in lieu of a foreground session (in this case the user). Unless a task is executing the session is passive and may be stored on disk. Each time a task is to be executed the session must first be fetched into memory, so that the task may be instantiated. The execution schedule is determined by the scheduling constraints.

3

The Web& Architecture

Figure 2 is a block diagram of the Web& architecture. As can be seen in the figure, the architecture consists of 4 top level blocks. Each block manages a given Interaction 5

Session Manager

User

Session Database

I n t e r f a c e

Scheduler

Wrapper Interface task instances

Action Manager

completed tasks/results User notifications

...

Clients

C l i e n t

Service Manager

service providers

World Wide Web

Service Invocation

Task Manager

Notification Manager

Figure 2: The Web& Architecture between the blocks can be performed using any arbitrary protocol that is understood by the interacting entities.

3.1

Service Manager

This block provides the basic functionality in the system to provide services and interface with the web. Since the current web does not implement a service based interface, this module currently has to provide the translation between the web server specific interface and the generic service based interface. 3.1.1

Service wrappers

Service wrappers wrap individual web servers and export a service interface to the rest of the system. To do so, they need to provide a translation between server specific HTML/XML interface and the internal protocol used to invoke a service. In effect each service wrapper implements a Uniform Service Descriptor (USD) interface for a web server. If a web server is capable of providing services using the USD interface, then a service wrapper is not required for such a web server. To achieve grouping of web servers providing analogous services as is demanded by the service definition, web servers are grouped into overlapping domains. Services are defined on these domains rather than individual servers (though certain domains may contain only one server). For example, all web-sites selling books might be grouped into a Books domain, which provides services such as List (given some criteria about books, such as title, author, ISBN number, etc.; return a list of books matching this criteria), Price (given an ISBN number for a book returns its price), Buy (given an ISBN number of a book and customer information, buy the book and ship it to the customer). Similarly other domains may be defined.

6

The service interface block also provides clients with structural information, i.e. how web servers are organized into domains, what services each domain provides, what schemas must a client use to invoke a given service, etc. If a client knows the location (IP address/port number) of the service manager, it can use this structural information to invoke a service on a web server without any prior knowledge of the interfaces to each service and the domain structure.

3.2

Task Manager

The task manager is responsible for executing a task. In particular, it invokes the services in the task and collates the results. It also handles fatal error conditions which occur due to incorrect service specification, network and host failures, etc. Note that these are different from soft error conditions (a book was not found, travel itineraries could not be generated, etc). The task manager needs to be carefully designed so that tasks belonging to one client/user can not affect/manipulate tasks belonging to another client/user. Thus, the task manager needs to incorporate the UNIX notion of process protection by uids.

3.3

Action Manager

After a task completes execution, the task manager returns control to the the action manager which checks to see whether the results of task execution satisfy the conditions required to perform each conditional action in the session. Actions whose conditions are satisfied are immediately executed. Actions which create, modify or delete a session are dealt with internally, while actions generating notifications are sent to the notification manager. The action manager only allows actions access to the session that encapsulates them or to the sessions that they create. This is done for security reasons, so that actions do not manipulate state of other “unsuspecting” sessions. 3.3.1

Notification Manager

Once a task has completed there might be conditions/results that the user needs to be notified about. At this point a typical web& user would either be disconnected or mobile, and hence may not be available for interaction via conventional means (i.e. by providing output in a window on the users terminal). Notifications are special actions in the web& architecture that send specified conditions/results to a user via non-conventional communication channels. To handle such actions, the action manager offloads the action to a notification manager, which understands a variety of heterogeneous communication channels and has the ability to send notifications via these channels. Note, that we do not require that the notification manager “search” for the user, but we do allow this mode of operation.

7

3.4

Session Manager

The session manager maintains state for sessions; schedules tasks from sessions for execution by the task manager; tests action conditions and performs and schedules actions after each tasks completes execution. It also provide the client with an interface that allows it to create, manipulate and delete sessions. 3.4.1

Client Interface

The client interface provides the users/applications interacting with a web& system with the ability to create, modify, delete and monitor sessions. To do so it needs to export to the client a command language and primitives for session creation, manipulation and deletion. The client interface also allows the client to query the system for statistics, status of various sessions, list of sessions, results of tasks from a session, program information for a session, etc. The client interface needs to be carefully designed to avoid security problems. In particular, it must provide client authentication and verification services and restrict access to session information by client/user id. 3.4.2

Session Database

The session database contains all the information for each session. This includes the status of the session and all meta-information required to reconstruct a session in case of failures. In particular the session database maintains the session meta-data, all task results and task execution history, all actions and action execution history. The session database also provides user information to the system, including user-names, profiles, session portfolios for a given user and password information. 3.4.3

Scheduler

The scheduler monitors each session in the database for pending tasks. It is responsible for submitting tasks from each session to the task manager for execution at the time when these tasks are scheduled. A task that is scheduled is extracted from the session and sent to the task manager. In addition, the scheduler also provides the task manager with state information about the session that encapsulates the tasks.

4

The Prototype

A prototype implementation of web& has been implemented in JAVA 2. The implementation consists of three modules: session hosting and task manager module; service manager module and the notification module. We use XML as the standard interface language to invoke services. A block diagram of the implementation can be seen in figure 3. The system broadly follows the architecture except that for performance reasons we have merged the session manager and task manager into one module. This allows us to eliminate the latency of creating tasks. However, in the future, we shall be separating the two for a distributed implementation of the system. 8

Session and Task Hosting Engine

session database

JDBC

DOM

...

XML

Semantic Cache

interface

task execution action handling

Service Wrappers and Semantic Cache

Service Wrappers feedback

HTML

Web servers

Notification requests

DB

Notification Engine

XML + Program

EMail

Web Interface (Servlets) web/

Palm

Phone

USER

palm Vii

Figure 3: WEB& Prototype Implementation

4.1

Session and Task Hosting Engine

In our implementation a fully instantiated task (i.e. with all the data structures such as the DOM2 tree’s populated) has a memory footprint of about 100KB. This is substantially reduced if we store the in-core state in the session database and upload it on demand. The task can then be fully instantiated when it is about to be scheduled. We do this by encapsulating all the transient task state in JAVA soft references. Whenever the JVM (Java Virtual Machine) senses memory pressure, the soft references are garbage collected on an LRU basis and the memory released. Since this only happens when the JVM is running out of memory we automatically ensure the on-demand fetching of task state only takes place when the system is heavily loaded. Further the LRU properties of the garbage collector ensure that the most frequently scheduled task are most likely to remain in memory. In addition, to reduce the latency of executing a task after it is scheduled, the scheduler also prefetches the task state a certain number tasks at the head of the scheduling queue based on a tunable parameter. Thus, most tasks are already instantiated when they are scheduled. This software component also incorporates the action manager from our architecture. Each time a task completes execution, it is dispatched to the action manager for testing conditions on the result and invoking/dispatching actions as needed. The action manager is designed using a core which can dynamically load action classes from disk. This allows us to add actions to the system without having to restart the core engine. Currently, we can perform all the session manipulation allowed by the client interface in an action. We can also control scheduling of future tasks using 2

Document Object Model (DOM) is a parse tree representation of an XML document

9

these actions. The conditions on action execution are encoded as prefix operators on XPath-like expressions on the XML results. For example, to check if a flight is delayed by more than 30 minutes the condition would be “(EXISTS result/status) && (GT 30)”, where the first expression checks that a valid result was returned and the second expression pulls out the contents of the node results/delayedby under the document element and checks if this numeric value is greater than thirty minutes.

4.2

Client Interface

The client interface has been designed to accept XML templates and program information. It allows a client to specify a task template for the session; scheduling and task repetition constraints for the tasks in the session; and a set of triggers which include conditions on task results and actions. Currently, we only allow one task per session which can then be repeatedly re-instantiated to create a series of tasks (repeatable tasks). This is a limitation of the interface and not the session management core of the system. We are currently working on eliminating this constraint. The interface also allows the client to manipulate and query a session including stopping execution of tasks in a session; deleting sessions; rescheduling sessions; attaching and deleting triggers for a session; fetching results and program information for a session; checking on the status of tasks in a session. While the client interface provides a powerful data definition language and data manipulation language; using this interface directly requires a degree of sophistication on the part of a user. To solve this problem, we have implemented JAVA servlets to provide a user friendly web interface to the system. These servlets generate the appropriate commands to the client interface based on the specifications provided by the user. These servlets are being used to provide a standard web browser as well as PALM VII interface to the system. This servlet engine also allows us to create customized applications on top of the web& core. We can easily “drop” servlets into the servlet engine to provide customized interfaces to the user for specific classes of tasks.

4.3

Service Wrappers and Semantic Cache

The service wrappers export an XML based interface to the rest of the system. This component of the system takes as input an XML form which specifies which service to invoke, which server to invoke it on, and the parameters to be passed to the service wrapper. As a return value, it provides an XML form encoding either an error condition or the result of the service execution. The service manager is designed currently designed around a semantic cache for efficiency reasons. The service invocation request is made to the cache which uses a dynamically loaded class to obtain domain specific information to determine if a semantically equivalent unexpired document is present in its cache. If not, it loads up the appropriate wrapper class from disk and passes the input form to an instance of a class. If there is no error the result is cached and sent back to the invoker, else an error document is returned. 10

List, ListServices, ListMetaServices

Root List, ListServices, ListMetaServices, Index

Books

Price, Buy DiscountBuy

Price, Buy

BookSeller

BookRetailer

Figure 4: Directory Example In order to organize services and provide rudimentary service discovery we have organized web servers into a logical directory structure (see figure 4) that is maintained by the service wrapper interface software. Web servers providing the same service are grouped into domains. The domains themselves belong to a logical top level domain called the root domain. The entire directory structure is bootstrapped using a special set of services that are internally handled by the service manager. Both the root and each domain export (i.e. implement) a minimal set of services which consists of the following services: List, ListServices, GetInterface. We call these services primary services or operations in our system. The List service returns a list of entities within the entity exporting the service: i.e. at root level it returns a list of domains, whereas at domain level it returns a list of servers in that domain. (The architecture can be easily extended to include sub-domains, but for simplicity we do not include them in this discussion). ListServices is a list of services that all constituent entities (i.e. those returned by List) must provide. GetInterface is a list of services that this entity provides. (While obtaining the service schemas is logically a part of ListServices as well as GetInterface, in practice we need an additional meta-service/operation GetServiceSchema to actually obtain the schema for using a particular service.) These operations can be used to bootstrap the entire logical directory structure. Thus, a List on the root server returns a list of domains, whereas a List on each domain server returns a list of servers (or sub-domains) within a domain. The ListServices operations can be used to obtain the operations mandated by the root and each domain server. As a backup to the root and domain servers, our prototype also includes a web hosted directory.

4.4

Notification Engine

The notification engine provides web& an interface to post-pc devices over both data and non-data channels. Currently the notification engine is actually implemented as

11

set of servers, one for each type of notification. Each server provides the necessary conversion to the target device; e.g. the Phone server uses TTS software to provide voice notifications and touch tone key-press sensing to provide feedback to the session manager. Currently, we support notification via Email, Pagers, Phones (voice notifications) and Palm VIIs. In order to support voice notifications, we have configured a Linux machine as a phone server. We use the free-ware festival text to speech converter to translate test notifications to voice. The Linux machine machine uses a voice modem card to connect to the phone line and a set of shell scripts to implement a simple phone server. This server understands return codes from DTMF phones and returns them to the client system.

5

Performance Measurements

The prototype is in preliminary stages of testing. As such we have performed a limited number of performance measurements on this system. We report here measurements for saturation levels in our prototype to determine what kind of performance level it is able to sustain at high loads. It should be noted that an implementation in C/C++ will have considerably better performance, since we are limited by the performance of the underlying JDK 1.2 virtual machine. Since our tasks are often network I/O bound; this causes a considerable drop in peak performance. However, JAVA provides many features which we believe are critical for a Web& prototype, especially the dynamic loading functionality that makes the system highly extensible. The results presented here include memory footprint and task execution time measurements. We consider only active fully instantiated tasks since these represent the performance of the system under peak load conditions which is of primary interest to us. In these experiments we schedule a certain number of tasks to execute at exactly the same time and then study the performance characteristics of the system. We ran two classes of experiments. In each class of experiments we measure peak performance by scheduling a certain number of tasks to execute simultaneously and then studying the system response. In the first class we use null tasks that simply terminate without doing anything when executed. This class of experiments measures the scheduling and context switching overheads in the system. In the second class of experiments, the tasks send a null form (a single newline character) to a wrapper on a remote machine. The service wrappers for our experiments are null wrappers: light weight processes that take an XML form and echo it back to the client. Thus we can measure the overheads due to the networking code. In each case the input load is increased until the task throughput peaks and degrades indicating that the system has saturated. In general, we face the problem of performing useful measurements while compensating for the effects of asynchronous garbage collection which can not be turned off in JDK 1.2+. To solve this problem, we first saturate the heap with a large integer array. We then remove the reference to this array and explicitly request garbage collection. This eliminates garbage collection. Nevertheless, all logs need to be carefully inspected to make sure that no garbage collection occurred before a data point can

12

Task Execution Time (ms)

Sustained Load:Tasks/second

4500 1200 tasks/sec 1000 800 600 400 200

execution time

4000 3500 3000 2500 2000 1500 1000 500 0

0

0 0

5 10 15 20 Input Load (Tasks/100)

25

5 10 15 20 Input Load (Tasks/100)

25

Figure 6: null task response time

Figure 5: null task throughput

140

Task Execution Time (ms)

Sustained Load:Tasks/second

14000 160 tasks/sec

120 100 80 60 40 20

12000

execution time

10000 8000 6000 4000 2000 0

0

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Input Load (Tasks/100)

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Input Load (Tasks/100)

Figure 8: network task response time

Figure 7: network task throughput

be trusted. The performance experiments were performed with the session and task hosting engine running in a Sun JRE 1.2.2 JVM on a 300MHz single processor SUN Ultra Sparc II machine with 512 MB ram and 1.2 GB swap running Solaris 2.6. The service wrappers were located on 300MHz dual processor Sun Ultra Sparc II on the same subnet. Figures 6 through 5 provide the results of our first class of experiments. Figures 8 and 7 represent the results of the second class of experiments. In each graph, the X-axis represents the input load, the number of tasks that were scheduled for simultaneous execution, scaled down by a factor of 100. Figures 6 and 8 represent the response times. Figures 5 and 7 represent the task throughput. Without encapsulating task state in JAVA soft references we can support about 4300 tasks in core on a 512MB single processor Sun UltraSparc-2. With the soft reference code, this number can be improved to over 25000 tasks. As can be seen from the graphs the peak throughput for null tasks is 1032 tasks for an input load of 1125 tasks per second whereas network tasks peak at 150 tasks per second for an input load of 150 tasks on our setup. We are working on eliminating the DOM data structures even for active tasks, which we hope will improve the peak load figures for null tasks in our experiments. These numbers capture the peak performance of our system. However, it should be noted that in practice task execution times are staggered so we can support many more tasks in a production environment. Also as expected these figures indicate that network is a serious bottleneck for our system. We are currently working on solutions for this problem by sharing sockets from a pre-

13

allocated socket pool and by sharing connections between all the tasks in our system. We believe that this will significantly improve the performance of the system. Both figures 5 and figure 7 indicate show saturation behavior with performance increasing linearly until saturation and then dropping sharply thereafter. Once saturation occurs, we also have increasing variance in the results, probably due to the response of the JVM under pressure. Figures 6 and 8 both indicate that the response time is bilinear with a lower slope pre-saturation component and a high slope component post-saturation component. These graphs also indicate that in both cases the response times saturation is up to two times higher than the throughput saturation. Thus, if response time is the major goal of a system designer, our prototype can be run at much higher peak loads.

6

Applications

We currently have two applications running on the system: IsItLate for tracking flights and IsItThere for tracking Federal Express and UPS packages. These can both be accessed on the web using the url: http://marti.rutgers.edu:8001/webamp2/login.jsp and logging in by inputting any string as the username and the same string as the password. We also have a variety of internal test applications running on the system; e.g. to do real time comparison shopping. These applications have been designed to test the performance of various components of the prototype. Note the user still has access to the core without going through these applications. The power of web& is obvious from these ease of creating these applications. The applications took a couple of days to implement once the core was completed. The major effort is in building customized interfaces for the applications in form of servlets and in creating the service wrappers for the web sites used to provide the operations underlying the services. We expect that with the XMLization of the web, that latter will be eliminated.

7 7.1

Design Issues Semantic Caches

For performance reasons we have implemented a semantic cache which can match and extract subsets of documents that are semantically equivalent to the cache query. Designing a flexible semantic cache is a considerable challenge. Currently our cache can use semantic functions defined by the wrappers, by the domains or by the root in decreasing order of specificity. The cache timeout for various documents can also be controlled in this fashion. We are working on various new implementations. It also turns out that semantic caches have a profound impact on XML cache synchronization mechanisms for distributed caches. We are currently investigating these aspects of semantic caches.

14

7.2

Fault Tolerance and Recovery

A major concern for us are the fault tolerance and recovery issues. This is because unlike and interactive system, our system does not have a user present to react to system failures. Thus the user will will be unaware of system failures and may thus loose tasks. To achieve this we serialize all state through JDBC to a database backend. This state can be used to provide failover as well as recovery on restart features. We are currently experimenting with a number of ways to provide high levels of fault tolerance by using mirror servers, clustered failovers, etc. We are also looking at loosely coupled architectures where multiple instances of web& run at multiple, geographically-distributed, sites in a co-operative fashion so that network failures which disconnect any single server can still be dealt with. In such system we can run the same task on multiple sites in a staggered fashion. If any one site fails due to any reason another instance of the task will be executed. As soon as a site successfully executes a task it can inform other sites so that they can cancel their task instances.

8

Related Work

Web& is designed to make the web experience of any user more productive. There is a large body of work on other approaches designed to make the web more accessible for a variety of devices and users. Agents (for example, see [3]) have been used to provide adaptive web browsing support to enhance the capabilities of a web user. Agents have the capacity to learn user specific “meta-information” such as browsing habits, preferences, etc., and to tailor the user’s web experience according to the information collected. Agents can also be used to execute complex tasks on behalf of a client. Agent architectures can be used to provide an analogous system. Web& differs from agents in that our focus in on providing programmable interfaces to the web using our service and session model and in that the issues we have tried to address are orthogonal to most agent based systems. Active proxies [4] have been used to adapt web interaction for resource constrained devices. The proxy can filter and distill information to make it renderable on a specific device. In particular, such proxies can be used to remove images or reduce image complexity for PDAs and low bandwidth links and perform text filtering on a per device basis. The client still maintains control of the web interaction, the proxy only operates on the content. Web& is different in that we do not try to match the capability of each device, rather the heterogeneity problem is solved by exporting a simple but standard interface to all clients and it is upto the client to decide how to render this interface. Furthermore, in case of Web& the client relinquishes control over web interaction and offloads this responsibility to the client proxy. Disconnected operation has been supported in a variety of databases (Microsoft’s SQL Server, Sybase’s SQL Anywhere and Oracle’s OracleLite) as well as file systems such CODA [11]. In such systems, the client downloads enough state from the server

15

to be able to operate even while it is disconnected. Web& is significantly different from these architectures. A Web& client is not responsible for any interaction, rather it only hosts the user interface. The responsibility of performing web tasks is with the client proxy which does not run on a disconnected machine. Considerable work has also been done on service architectures, for example, JINI [15] and E-Speak [7]. These architectures provide support for clients to discover and access services provided by servers. Such architectures can be used as an underlying foundation to deploy a Web& implementation; however, Web& attempts to achieve far more than just service discovery. In particular, we are also interested in providing a transactional framework where services are the basic operations and to provide fault tolerance and persistence. T-Spaces [18] provide a middleware architecture with database capabilities. This approach is complementary to our approach in the sense that it addresses a number of orthogonal issues. In particular, TSpaces can be used to create a Web& implementation.

9

Conclusions

The web is growing exponentially, both in size and capability. Exploiting the resources provided by the web is becoming frustrating to do manually. We need an architecture to automate synchronous web tasks using a system which can operate on the web and perform such tasks on behalf of the user even when he/she is disconnected or otherwise unwilling to control the interaction. We achieve this by providing the blueprint for a novel non-interactive web architecture called Web&. The Web& architecture allows the user to execute web related tasks asynchronously and concurrently, while providing a uniform interface to the results. As a result, Web& uniformly supports disconnected, mobile and heterogeneous clients. Furthermore, since the system is asynchronous Web& is able to hide network/server latencies and failures from the user. Web& also provides composite services by supporting the notion of schedulable web transactions. All of this leads to a highly enriched and fruitful web experience for the user. We have implemented a web& prototype in JAVA 2. We have performed preliminary evaluations of this prototype that indicate that we can support upto 25000 tasks on a single SUN Ultra-II with 512MB ram with a peak network throughput of 150 tasks per second. These numbers can be improved with more hardware and additional optimizations to the code. Web& is also our testing ground for various concepts in disconnected operation on the web, web-based transaction processing and novel operating system architectures including advanced caching work. Currently we are working with semantic caching of XML documents that semantically classifies and caches XML documents. Thus, semantically equivalent documents can be served using domain specific knowledge that is provided by the individual service wrappers. Furthermore subset queries where the answer to a query is a subset of a cached document can be efficiently answered.

16

References [1] The JDBC 1.2 specification. http://java.sun.com/products/jdbc/.

Technical

report.

[2] B. R. Badrinath and K. Ramamritham. Semantics-based concurrency control: Beyond commutativity. ACM Transactions on Database Systems, 17(1):163–199, 1992. [3] A. Chavez and P. Maes. A real-life experiment in creating and agent marketplace. Proceedings of the Second International Conference on the Practical Application of Intelligent Agents and Multiagent Technology, April 1997. [4] A. Fox, S. D. Gribble, T. Chawathe, and E. A. Brewer. Adapting to network and client variation using active proxies: Lessons and perspectives. IEEE Communications, August 1998. [5] J. Gosling and H. McGilton. The JAVA Language Environment. Sun Microsystems. http://java.sun.com/docs/white/langenv/. [6] J. Gray and A. Reuter. Transaction Processing: Concepts and Techniques. Morgan Kauffmann, 1993. [7] Hewlett Packard. E-speak Architectural Specification. speak.hp.com/pdf/archspec.pdf. [8] IETF. RFC 1777: Lightweight http://www.ietf.org/rfc/rfc1777.txt.

Directory

http://www.eAccess

Protocol.

[9] IETF. RFC 2068: HTTP 1.1. http://www.ietf.org/rfc/rfc2068.txt. [10] D. M. Ritchie and K. Thompson. The unix time-sharing system. Technical report. http://cm.bell-labs.com/cm/cs/who/dmr/cacm.html. [11] M. Satyanarayanan. Coda: A highly available file system for a distributed workstation environment. Proceedings of the Second IEEE Workshop on Workstation Operating Systems, pages 447–459, Sept. 1989. [12] A. Silberschatz, H. Korth, and S. Sudarshan. McGraw-Hill, 1997.

Database System Concepts.

[13] Sun Microsystems. The JAVA Platform. http://java.sun.com/docs/white/platform/javaplatformTOC.doc.html. [14] Sun Microsystems. Jini Technology Architectural http://www.sun.com/jini/whitepapers/architecture.html. [15] Sun Microsystems. http://www.sun.com/jini/specs/.

Jini

17

Technology

Overview. Specification.

[16] W3C. W3C Recommendation DOM 1.0, http://www.w3.org/TR/1998/REC-DOM-Level-1-19981001/.

Oct

1998.

[17] W3C. W3C Recommendation XML http://www.w3.org/TR/1998/REC-xml-19980210.

Feb

1998.

1.0,

[18] P. Wycroft, S. W. McLaughry, T. J. Lehman, and D. A. Ford. Tspaces. IBM Systems Journal, August 1998. http://www.research.ibm.com/journal/sj/373/wyckoff.html.

18