tions on Modeling and Computer Simulation. The de - Semantic Scholar

NOTE: This is a preliminary release of an article accepted by the ACM Transactions on Modeling and Computer Simulation. The de nitive version is currently in production at ACM and, when released, will supersede this version. c 1998 by the Association for Computing Machinery, Inc. Permission Copyright make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for pro t or direct commercial advantage and that copies show this notice on the rst page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works, requires prior speci c permission and/or a fee. Permissions may be requested from Publications Dept, ACM Inc., 1515 Broadway, New York, NY 10036 USA fax +1 (212) 869-0481, or [email protected].

The Purdue University Network-Computing Hubs: Running Unmodi ed Simulation Tools via the WWW Nirav H. Kapadia Jose A. B. Fortes and Mark S. Lundstrom School of Electrical and Computer Engineering, Purdue University Association for Computing Machinery, Inc., 1515 Broadway, New York, NY 10036, USA Tel: (212) 555-1212; Fax: (212) 555-2000 This paper describes the Web interface management infrastructure of a functional networkcomputingsystem (PUNCH) that allows users to run unmodi ed simulationpackages at geographically dispersed sites. The system currently contains more than forty university and commercial simulation tools, and has been used to carry out more than one hundred thousand simulations via the World Wide Web. Dynamically-constructed virtual URLs allow the Web interface management infrastructure to support the semantics associated with an interface to computing services without requiring any changes to Web browsers or WWW protocols. Virtual URLs also facilitate customizable control of access to networked resources. Simulation tools with text-based interfaces are supported via dynamically-generated, virtual interfaces, whereas tools with graphical interfaces are supported by leveraging available remote display-management technologies. Virtual interface generation and interactivity emulation are handled by a programmable state machine in conjunction with a mechanism to embed variables and objects within standard HTML. General Terms: Internet computing, Web-based simulation, network-computing

1. INTRODUCTION

Systems and technologies that currently provide access to computing services via the Web are typically custom-built for a single or a relatively small set of applications. In many cases, each application must be modi ed or even speci cally redesigned to work within the custom-built Web computing environment. This paper describes the Web interface management infrastructure of a functional network-computing system (PUNCH) that allows users to run unmodi ed software applications at geographically dispersed sites. PUNCH, the Purdue University Network Computing Hubs, is a network-based This work was partially funded by the National Science Foundation under grants EEC-9700762, ECS-9809520, EIA-9872516, and EIA-9975275, and by an academic reinvestment grant from Purdue University. Address: 1285 Electrical Engineering Bldg., Purdue University, West Lafayette, IN 47907-1285

2




N. H. Kapadia, J. A. B. Fortes, and M. S. Lundstrom

computing system that allows users to access and run unmodi ed software applications via standard World Wide Web browsers. Users can de ne simulations, run them, and view the text and graphical output. Tools do not have to be written in any particular language, and access to source or object code is not required. The PUNCH infrastructure is geographically distributed, but this is transparent to users, who can run tools wherever they reside.1 To our knowledge, PUNCH is the rst and, to date, the only Web-based computing system that is designed to support arbitrary tools and is utilized on a regular basis in a live environment. This paper focuses on aspects of PUNCH that allow it to operate within the framework of the World Wide Web without requiring any modi cations to applications, Web browsers, or WWW protocols. The remaining sections are organized as follows. Section 2 presents the advantages of PUNCH over \point solutions". Section 3 outlines a user's view of PUNCH and provides a brief overview of the underlying infrastructure. Section 4 discusses issues that arise in the design of a WWW-compatible interface to computing services and presents solutions for the described problems. Section 5 evaluates the Web interface management infrastructure of PUNCH. Finally, Section 6 presents concluding remarks and outlines quanti able impact of the technology. 2. RELATED WORK

The work on PUNCH was motivated by the fact that many of the systems and technologies that currently allow computing on the Web target a single or a relatively small set of tools and/or work within controlled environments in which many of the issues that arise in production environments can be ignored. Solutions that target individual tools tend to be non-reusable in spite of the fact that they involve a signi cant amount of duplicated eort. (Reusability, in this context, implies an ability to reuse the computing system with other tools without making any modi cations to the system itself.) For example, a large number of systems (e.g., the Exploratorium [Adasiewicz 1995], JSPICE [Souder et al. 1998], and others | see [Kapadia 1999] for details) are based on scripts that need to be modi ed in order to add any new application to the system. Other designs are more exible. The MOL [Reinefeld et al. 1997] prototype, for example, employs static Web interfaces that can be adapted for individual tools. The CSLab prototype [Chatterjee et al. 1997] provides a Java-based environment that allows users to create simulation experiments using available modules. The 1 PUNCH has been operational for four years, and currently serves more than ve hundred users from Purdue, across the US, and in Europe. To date, PUNCH users have logged more than two million hits and have performed over one hundred thousand simulations. PUNCH can be accessed at \http://www.ecn.purdue.edu/labs/punch/"; courtesy accounts are available.

The Purdue University Network-Computing Hubs




3

NetSolve [Casanova and Dongarra 1996], Ninf [Sato et al. 1997], and RCS [Arbenz et al. 1997] systems are based on structured designs that target numerical software libraries. However, static interfaces are not adequate for all tools. Custom environments and structured approaches cannot be easily applied to general-purpose tools. Another problem is that the majority of these designs either utilize specially built tools or assume the availability of source and/or object code for the tools | which eectively precludes the installation of most commercial tools. Solutions that address individual issues are generally reusable, but the tasks of adapting them for a production environment and integrating them into a complete computing infrastructure are non-trivial. For example, VNC [Richardson et al. 1998] and WinFrame [Citrix 1996] provide very exible mechanisms for exporting graphical displays to remote consoles in a platform-independent manner | but, by themselves, the technologies do not help address other issues (e.g., access control and management) that arise in a wide-area distributed computing environment. The goal of the PUNCH project is to design and deploy a production wide-area computing framework in which the computing infrastructure is not tied to the characteristics of individual applications. Functionally, this is equivalent to designing a multi-user operating system for networked resources that provides user-transparent le, process, and resource-management functions; handles security and access control across multiple administrative domains; and manages state information (session management). The need for such a system and the associated bene ts in the context of the World Wide Web have also been recognized elsewhere [Lamine et al. 1997; Reynolds 1996]. 3. THE PURDUE UNIVERSITY NETWORK-COMPUTING HUBS 3.1 PUNCH: A User's View

From a user's perspective, PUNCH is a WWW-accessible collection of simulation tools and related information. It allows geographically dispersed tools to be indexed and cross-referenced, and makes them available to users world-wide. The infrastructure hides all details associated with the remote invocation of tools from its users. Functionally, PUNCH allows users to 1) upload and manipulate input les, 2) run programs, and 3) view and download output | all via standard WWW browsers. Users enter the hubs (\hubs" are logical groupings of discipline-speci c tools; see Figure 1) on PUNCH either as members or as guests. Members have their own, private hub-accounts, and can access the full functionality of PUNCH. Guests, on the other hand, can only view the example input les and the precomputed output available in a \public" PUNCH account. Access to all information and resources on PUNCH can be customized and/or restricted on a user-speci c basis by PUNCH

4




N. H. Kapadia, J. A. B. Fortes, and M. S. Lundstrom

The Purdue University Network-Computing Hubs (PUNCH) The Parallel Programming Hub

The Computer Architecture Hub

The VLSI Design Hub

...

Network -Desktop Interface

The Computational Electronics Hub

The DES CA RT ES

...

Toolkit

Network -Desktop Interface

Front End

(Network Desktop) Internet / Intranet

SCION Tool (Application) Tool (Application)

Tool (Application)

Parallel Machine

..

Physical Location ‘1’

..

Workstation Cluster

...

Compute-Server

SCION Software

Compute-Server Parallel Machine

..

.. Tool (Application)

SCION Software

Tool (Application) Tool (Application)

Workstation Cluster

Physical Location ‘n’

Fig. 1. The main components of the PUNCH infrastructure: the network desktop manages data and interfaces, and SCION manages resources.

The Purdue University Network-Computing Hubs




5

system administrators. 3.2 PUNCH: Infrastructure Overview

The PUNCH infrastructure can be divided into two parts (see Figure 1). The network desktop allows users to interact with the network-computing infrastructure (SCION) via standard Web browsers, and generates customized views of available resources for each class of users. SCION serves as PUNCH's user-transparent middleware. It consists of hierarchically distributed servers that cooperate to provide on-demand network-computing. SCION manages the run-time environment, controls resource access and visibility, and manages available resources. A Web-based computing system can be characterized in terms of four parameters: 1) its interface to the external world, 2) its internal design, 3) the class of software and hardware resources that it supports, and 4) the capabilities of its resource management system. The following discussion outlines the characteristics of PUNCH in terms of these parameters. 3.2.1 External System Interface. PUNCH provides a universally-accessible, Webbased interface to a network-based computing environment. The system supports logical organization of resources (i.e., hierarchical indexing and cross-referencing) across multiple administrative domains, and can present users with a dynamically customized (individualized) view of resources [Kapadia et al. 1997]. PUNCH users have their own logical, network-accessible user-accounts; the system manages its own physical accounts on remote resources and dynamically recycles them among users as necessary. 3.2.2 Internal System Design. PUNCH employs a hierarchically distributed architecture in order to process requests in an ecient and a scalable manner [Kapadia and Fortes 1999b]. The number of levels in the hierarchy can be increased as the system grows, which further contributes to the scalability of the core architecture. The infrastructure supports multiple, concurrent users and processes (jobs), and uses resource replication to address single-point failures. The PUNCH infrastructure can be deployed on heterogeneous platforms across multiple administrative domains. This is accomplished by partitioning the system into cells, each of which is independently-managed [Kapadia and Fortes 1999b]. PUNCH users do not need physical accounts on the remote resources available to the system. 3.2.3 Support for Software Resources. The PUNCH infrastructure is designed to allow administrators to integrate (install) unmodi ed tools with little eort; access to source or object code is not required. Stand-alone tools with batch, text-based interactive, and graphical user interfaces are supported [Kapadia et al. 1998]. Tool

6




N. H. Kapadia, J. A. B. Fortes, and M. S. Lundstrom

installation is accomplished by way of a speci cally-designed high-level language that describes the behavior of the tool to the PUNCH infrastructure [Kapadia and Fortes 1999a] | the description provided by a tool-installer is automatically compiled and integrated into PUNCH's knowledge management system. The Webbased interfaces for tools are dynamically generated on the basis of structural information speci ed in HTML templates (see Section 4.4). Tools with relatively simple interfaces can be installed in as little as thirty minutes. 3.2.4 Resource Management Capabilities. PUNCH employs an application management framework that allows it to focus on achieving the best possible cost and performance tradeo at run-time, based on speci cations of \acceptable" cost and \desired" performance. Cost gures are provided by administrators. Performance is estimated by a prediction system [Kapadia et al. 1998; Kapadia et al. 1999] that utilizes machine learning techniques in order to estimate run-speci c resource requirements before a scheduling decision is made. PUNCH utilizes a non-preemptive, decentralized, adaptive, sender-initiated resource management framework [Kapadia 1999]; the actual scheduling algorithms can be selected dynamically on the basis of predicted run-speci c resource-usage information. 4. WEB-BASED COMPUTING: INTERFACE DESIGN ISSUES

A Web-based computing system is a wide-area distributed-computing system in which standard World Wide Web browsers serve as the user interface. This section outlines the issues that must be addressed in order to make unmodi ed simulation tools usable via the World Wide Web, and describes the corresponding solutions in the context of the network desktop of PUNCH. The World Wide Web was primarily designed to serve documents and information | as an interface to a computing system, it must accommodate radically dierent demands (see Table 1). An added complication is brought about by the need to reorient and enhance the applicability of the World Wide Web framework without requiring any changes to the functionality and operation of existing Web browsers or WWW protocols. 4.1 Computing-Service Semantics via URLs

The semantics associated with an interface to computing services are supported within the context of the World Wide Web by treating URLs as locations in a dynamic, virtual, and active (side-eect based) address space. The client (i.e., user/browser) initiating a given transaction is equated to a distinct user-level process in a multi-user computing system, and the request is treated as an attempt to access a memory location (represented by a URL) with a lexicographical address within the private address space of the associated process (i.e., user, in the case of

The Purdue University Network-Computing Hubs




7

Table 1. A comparison of the demands imposed on the World Wide Web infrastructure when it is employed as a user-interface to a computing system, as opposed to a mechanism for document and information delivery. WWW Framework

WWW-Based Internet/Intranet Computing

Issues

Unique mapping between URL and some entity within the filesystem (physical URL).

URL is a functional request that is mapped in a context-sensitive manner (virtual URL).

WWW URL model does not support truly dynamic URL mappings.

Mostly public space, with some support for access control.

Mostly private space, with some support for sharing user data.

Authentication model for WWW does not scale.

Designed for document serving, with some support for dynamic information.

Primarily provides OS functions (e.g., process management). Document serving is secondary.

Standard WWW resource management model is not appropriate.

Designed for a (mostly) read-only environment.

A read-write environment is critical for correct operation of OS services.

Need synchronization and resource-locking. Support for a multi-user web OS.

Processing is supported as an add-on via CGI scripts.

Processing is an integral part of the service.

Integration of distributed computing and WWW.

8




N. H. Kapadia, J. A. B. Fortes, and M. S. Lundstrom

PUNCH). A virtual URL is made up of multiple components, each of which is a base-oset pair consisting of a \base" and an \oset" (see Figure 2). The components are delimited by a `/'. The URL may also contain meta-information that is utilized by the modules that process the physical URL. The number of components in a URL and the amount and type of meta-information contained in it depend on the type of request and the associated context, and change dynamically. In order to map a virtual URL to a physical one, PUNCH translates each base-oset pair individually to an internal object handle, starting from the left-hand side. Object handles point to resources (e.g., users, tools, etc.) and to PUNCH modules responsible for processing the request. The translation process implicitly depends on the current state associated with the user and the system, and typically generates side-eects that modify the state of the user and/or the system. As an example, consider the following virtual URL | which is associated with the process of initiating a run: \/Member97/CeHub/Programs/4135 Minimos/Exec/". The rst translation step will map \Member97" to the internal identi er (identical to the user's PUNCH login) for the user initiating the request and initialize the internal user-state (e.g., preferences, access restrictions, etc.). The second translation step uses the tag \CeHub" to access and con gure the content for the logical \hub" represented by the tag. The con guration is personalized according to the userstate generated by the previous step. The third step uses the tag \Programs" to set up the internal state so that the request is forwarded to the module that handles program-related requests. The fourth and the fth components in the virtual URL represent the meta-information (name and access code for program, and type of view, respectively) required for this request. Conceptually, the request is forwarded for processing after all of the translation steps have been completed; in practice, the translation is interspersed with processing steps for eciency reasons. 4.2 Context-Sensitive Access Control

PUNCH draws on the basic premise of virtual memory management to control access to its resources (see Figure 2). Requests are assumed to contain virtual addresses (i.e., URLs) that can be dynamically translated to physical ones. Analogous to virtual memory management, each PUNCH request can be veri ed for validity in terms of 1) being within the address-space bounds for the associated process (user) and 2) the type of access (read or write). All accesses to PUNCH undergo this validation process, allowing the system to control access to its resources on a per-user and per-resource basis. Once the validity of a request has been veri ed, it is mapped to a physical resource. This mapping process is implicitly dependent upon the source (user-id) of the request. Thus, dierent users attempting to access

The Purdue University Network-Computing Hubs




9

a) Virtual Memory Management Virtual Address

Address Translation

Base

Offset

Translation process involves a simple table lookup.

Physical Address

Page Address

Offset in Page

b) URL Translation in PUNCH Virtual URL

URL Translation

Partially Translated URL

Base

Offset

...

Base

Offset

Meta-Information

Translation process involves database queries, and generates side-effects (e.g., state and context initialization).

Object Handle

...

Object Handle

...

Base

Offset

Meta-Information

Target Module

Meta-Information

Repeat Process for each Base-Offset Pair

Physical ‘‘URL’’

Fig. 2. In conventional operating systems (part `a' in gure), a virtual address is mapped to a physical one by way of a lookup table, and the mapping does not generate any side-eects. In PUNCH (part `b' in gure), each component of the virtual URL is individually translated to a physical counterpart. The translation process generates side-eects, and is context-sensitive in the sense that the mapping for a given component depends on the side-eects generated by earlier mappings.

10




N. H. Kapadia, J. A. B. Fortes, and M. S. Lundstrom

the same virtual URL could potentially trigger completely dierent actions. PUNCH leverages this fact to dynamically generate a logical (virtual) view of available tools and resources for each class of users. For example, PUNCH can be con gured so that a proprietary tool `T' is only visible to a speci ed subset of users `U'. With this con guration, PUNCH will exclude `T' when generating Web pages for users who do not belong to `U'. Moreover, PUNCH enforces such access-restrictions; a user who does not belong to `U' will not be able to access `T' regardless of the URL that he/she uses (the relevant URL will be deemed \invalid" by PUNCH for unauthorized users). 4.3 Web-Accessible Operating System Services

PUNCH provides Web-accessible, multi-user operating system services for networked resources. The OS services include user-transparent le, process, and resource management, security and access control across multiple administrative domains, and state management (i.e., session management). The PUNCH operating system is unique in that it can be dynamically (re)con gured to meet the needs of dierent types of tools. Resources can be added to PUNCH incrementally via a resource-description language speci cally designed to facilitate the speci cation of tool and machine characteristics. For example, a machine can be incorporated into PUNCH simply by specifying its architecture (make, model, operating system, etc.) and starting a server on it. Similarly, a tool can be added by \telling" PUNCH the tool's location, its input behavior (e.g., command-line arguments), what machines it can run on (e.g., Ultra-Sparc), and how it ts into the logical organization of software resources (e.g., device simulation tool). The Web pages needed for the tool are generated automatically from HTML templates, as described in Section 4.4. Adding a machine or a tool to PUNCH typically takes less than thirty minutes. The PUNCH methodology allows unmodi ed tools to be integrated into the WWW computing infrastructure. Tools do not have to be written in any particular language, and access to source or object code is not required. For tools with textbased user-interfaces, the system 1) presents users with a virtual interface for each tool and 2) provides the network-computing infrastructure with the means to map the virtual interface to the tool's native interface at run-time. This allows users to interact with a virtual environment that is independent of the tool's native interface and/or site-speci c idiosyncrasies. For tools with graphical user-interfaces, PUNCH leverages remote display-management technologies such as Broadway [X Consortium, Inc. 1996] and VNC [Richardson et al. 1998]. The use of virtual and graphical user-interfaces represents complementary approaches to making legacy tools executable via the World Wide Web. Virtual in-

The Purdue University Network-Computing Hubs




11

terfaces provide tool-installers with the exibility needed to modify and customize a tool's native user-interface for dierent types of users (e.g., a novice versus an expert) and environments. They also help shield users from site-speci c idiosyncrasies. On the other hand, the graphical approach allows a tool's native interface to be \exported" into a network-computing environment with very little eort. Together, the two approaches provide an extremely exible and powerful mechanism for making tools usable via the World Wide Web; to date, more than forty programs developed by eight universities and four vendors have been made available via PUNCH. 4.4 Read-Write Capabilities via Standard HTML

PUNCH provides support for read-write capabilities within the World Wide Web framework by allowing variables and objects to be embedded within standard HTML documents. In order to interoperate with standard Web browsers, PUNCH dynamically replaces the tokens representing the embedded variables and objects with their current values when the HTML documents are sent to clients (i.e., browsers). Documents (HTML pages) that contain embedded variables and/or objects are called HTML templates. Variables include standard types provided by high-level languages | integers, oating point numbers, and strings. Objects include HTMLspeci c constructs such as check-boxes, menus, and radio-boxes. HTML templates are made up of two parts: 1) variable and object declarations, and 2) HTML code. The variables and objects serve the same purpose as corresponding constructs in high-level languages | they allow the PUNCH compiler and run-time system to track, interpret, and manipulate information ( ow control is handled by a programmable state machine; see Section 4.5). The functionality provided by HTML templates is best explained by way of an example. Consider a situation in which users need to \walk" through the les and directories in their (PUNCH) accounts via a browser. Figure 3 shows a simple HTML template that can be used to accomplish this task. The left-hand side of the rst declaration statement speci es an object of type menu called myFiles. The right-hand side speci es the size of the menu (one) and initializes its contents via a directory handle (UserFiles) exported by the desktop infrastructure. The second declaration statement binds the contents of the menu object to the current directory of the user for whom the transaction is being processed. Once a variable or an object has been declared, it can be referenced by name in the HTML part of the template, as shown in the gure. The name will be dynamically replaced with appropriate \static" HTML code and data when the template is accessed via an appropriate URL. For example, the reference to myFiles in Figure 3 will be replaced with the HTML code for a menu whose contents are the list of les in

12




N. H. Kapadia, J. A. B. Fortes, and M. S. Lundstrom

Begin PageTemplate Begin Declarations menu myFiles = 1:; bind myFiles = ; End Declarations Begin HTML Select a file (or folder to open):

End HTML End PageTemplate

Fig. 3. An example HTML template. The rst part of the template consists of declaration statements and bindings for variables and objects (the gure only contains one object | myFiles). A \bind" statement attaches speci c semantics to a given variable or object (e.g., contents of a directory, email address, etc.). The second part of the template consists of HTML code that can reference the variables and objects de ned in the rst part of the template (or in other templates referenced via a common metaprogram). Except for the embedded variables and objects, the code is standard in the sense that it can consist of arbitrary HTML, including scripting languages (e.g., JavaScript).

The Purdue University Network-Computing Hubs




13

the current directory of the user who initiated the transaction that references the template. The syntax associated with the template shown in Figure 3 was speci cally designed to supplement standard HTML without requiring any changes to the language. Unfortunately, this syntax does not lend itself to ecient run-time parsing of the template. This problem is addressed by using a compiler to translate the template into a format that is more conducive to ecient run-time substitution of variables and objects. The compiled version of the template is shown in Figure 4. The rst line contains the size of the compiled template le in bytes; this is used to eliminate race conditions that arise when the le is accessed while it is in the process of being (asynchronously) loaded into the disk cache for the network desktop. The second line contains information that is used for version control. The database \record" starts at the third line, which contains the access code (i.e., byte oset that allows O(1) lookup of information; see [Kapadia and Fortes 1999b]) for the record. The next four lines list the names and access codes of the four sub-records that make up this record (byte osets for sub-records begin after the end of the record header). The next two lines contain the (sub)record separator and the access code for the rst sub-record (VarSpec), respectively (the \5" represents the number of lines in the sub-record). The VarSpec record tells the HTML generation module how to locate and interpret the variables and objects in the PageTemplate sub-record. The location information is contained in the rst line within the sub-record; the numbers (2 5 8, in the gure) refer to the line numbers in the PageTemplate record that require substitution. The rest of the record contains information that supplies the semantics for the substitution process. Each variable/object is stored by itself on a separate line in the PageTemplate record. This organization allows for very ecient variable/object substitution. When a template is accessed, the network desktop rst loads the VarSpec record. Then, it jumps to the PageTemplate record and starts sending its contents over the network connection. When it reaches a line whose number matches the ones listed in the VarSpec record, it sends (generates) the contents of the appropriate variable (object) instead of sending the tag on the line. Experiments show that the overhead of variable and object substitution using this procedure is negligible in comparison to document serving times. 4.5 Programmable State Machine

PUNCH provides processing services and emulates interactivity by way of a programmable state machine that works in conjunction with the HTML templates

described earlier. The programmable state machine dynamically customizes the behavior of the network desktop interface for users and resources according to

14




N. H. Kapadia, J. A. B. Fortes, and M. S. Lundstrom

543 V2.0 70 VarSpec:1 PageTemplate:117 BindSpec:290 VariableNames:397 1 5 2 5 8 B B M _formTarget $formTarget _hiddenTags @hiddenTags _myFiles 1 0 0 - /$dirName/Input/$cUrl End VarSpec 117 11 _hiddenTags Select a file (or folder to open): _myFiles

End PageTemplate

. . .

Fig. 4. The compiled version of the HTML template. This is the version that is used at run-time by the network desktop. The format for the compiled version has been designed to allow the contents of the template to be retrieved and parsed in an ecient manner.

The Purdue University Network-Computing Hubs




15

speci ed criteria. The programmable state machine is made up of three functional units: a metaprogram interpreter, a virtual machine, and a persistence engine. A metaprogram is a program whose instructions are themselves programs. In this environment, metaprograms are used to de ne the behavior of the programmable state machine, and are written in a specially designed `C'-like language. In addition to the standard

ow control constructs (conditionals and loops) available in high-level languages, this language provides instructions to 1) customize and manipulate information for dierent types of users, 2) manage les and directories, 3) customize and serve documents and templates in response to user requests, 4) save and retrieve state information, and 5) start and stop child metaprograms and processes on local or remote machines. These instructions are summarized in Table 2. The metaprogram interpreter parses and executes the metaprograms.2 The virtual machine manages the run-time environment for the metaprogram interpreter. It executes the lower-level code that makes up the instructions in the metaprograms, enforces access control at the instruction level, and maintains low-level data structures (e.g., the stack) and state information (e.g., the program counter) associated with the underlying compute engine. Finally, the persistence engine maintains state information that allows metaprograms to be started and stopped as and when necessary. The programmable state machine is used to generate a customized HTML interface for each tool. The input to this state-machine consists of 1) a list of available states and 2) a description of the transitions between these states ( ow control). States are speci ed in terms of HTML templates. Flow-control information is speci ed via metaprograms. The state machine executes metaprograms in response to an attempt to access corresponding URLs. It keeps track of the values of variables and objects for interfaces that span multiple HTML pages by storing them in hidden form elds and/or encoding them within URLs. Moreover, the state machine can react to any run-speci c information (e.g., values embedded within user-supplied strings, data within les, etc.) that can be extracted by the network desktop's programmable parser [Kapadia and Fortes 1999a]. This allows the desktop to support programs that accept input in an interactive manner.3 The roles of the three functional units described above are best explained by using an example. Figure 5 shows a sample metaprogram that could be used in conjunction with the HTML template described in Section 4.4 to allow users to The metaprogram interpreter actually parses and executes a compiled version of the administrator-speci edmetaprograms; this helps improvethe run-timeeciency of the interpreter. 3 Programs with graphical user interfaces are managed via browser-compatible remote-display protocols such as VNC [Richardson et al. 1998]. 2

16




N. H. Kapadia, J. A. B. Fortes, and M. S. Lundstrom

Table 2. List of instructions supported by the metaprogramming language. In addition to the instructions shown in this table, standard ow-control instructions (e.g., if, else, while, etc.) provided by the `C' programming language are also supported. Instruction

Description

cache

on|off

Enables/disables the caching of templates for this metaprogram

chdir



Changes directory; this instruction updates internal state information

cleardir



Deletes the contents of the specified directory

dirof



Gets the name of the directory in which the specified file resides

display



Generates HTML code using specified template; stops metaprogram

end



Terminates metaprogram; execution cannot be resumed

execute

|

Executes metaprogram/tool on possibly remote machine; non-blocking

exists



Finds the absolute path to a file, if it exists within searched directories

isdef



Evaluates to one if variable is defined, zero otherwise

isdir



Evaluates to one if variable contains a valid directory, zero otherwise

length



Determines the length of the string in bytes

logstatus



Logs current status; used for error notification and recovery

maxof



Determines the largest value within a space delimited list

minof



Determines the smallest value within a space delimited list

retrieve

, []

Retrieves saved values of specified variables

retrievestate

Restores the state of the metaprogram from a previously saved image

save

,

Saves specified variables to persistent storage

savestate



Saves an image of the current state of the metaprogram

set

=

Assigns a value to the specified variable Stops the execution of the metaprogram; execution may be resumed

stop sumof



Determines the numerical sum of the values in a space delimited list

writeln



Writes values of the variables in the list to the current output

The Purdue University Network-Computing Hubs




17

menu myFiles = ’/’; retrievestate ’directory’; display ’Page1’; while(isdir()) { chdir(); display ’Page1’; } savestate ’directory’;

. . . Fig. 5. An example metaprogram. This metaprogram directs the network desktop to retrieve user-speci c state (e.g., preferences and defaults) from persistent storage, and to repeatedly (as the user selects directories in his/her PUNCH account) display the HTML template associated with Page1 until the user selects a le (which could then be used as the input le for a run, for example).

18




N. H. Kapadia, J. A. B. Fortes, and M. S. Lundstrom

\walk" through their les and directories. The execution of the metaprogram is triggered when a user accesses an appropriate URL, and proceeds as follows. The rst instruction shown in the gure is a declaration statement that causes the value of the object myFiles to be initialized to the contents of the root directory (for this user). The second instruction shown in the gure causes the persistence engine to set the variables and objects in the metaprogram to the values saved previously (for this particular user) via a `savestate' instruction. This instruction does not aect the values of variables and objects for which no state information exists. The third instruction triggers several steps. The virtual machine 1) instructs the persistence engine to save the internal state of the metaprogram, 2) generates access code and program counter information that can be used to restart the metaprogram at the appropriate instruction (i.e., the one following the current `display' instruction), 3) uses the HTML generation module to send the template associated with `Page1' (previously speci ed via a declaration statement) to the user who initiated the transaction, and 4) shuts the metaprogram down. At this point, the transaction that initiated the execution of the metaprogram is considered to be complete. On the browser (client) side, it simply appears as if a standard HTML document was returned in response to an attempt to access a given URL. Subsequently, when the user selects an entry from the contents of the menu and submits another request, the metaprogram is automatically restarted at the instruction that follows the previously executed `display' instruction. This is accomplished as follows. The access code and program counter information generated by the virtual machine in response to a `display' instruction is cached within hidden elds in the corresponding document and/or is encoded into the URLs that appear within that document. On the subsequent request, this information is extracted and presented to the metaprogram interpreter. The interpreter, in turn, restarts the metaprogram and instructs the persistence engine to restore its internal state. At this point the instruction that follows the `display' instruction can be executed without regard to the fact that the previous instructions were executed by a dierent process, possibly on a dierent machine. In this particular example, the rst instruction in the restarted metaprogram is the test for the `while' statement. If the menu entry selected by the user happens to be a directory, the programmable state machine will change the current directory for the user and display the HTML template again. The contents of the menu will now represent the contents of the new current directory. Eventually, when the menu entry selected by the user is not a directory, the test for the loop will fail and the metaprogram interpreter will execute subsequent instructions. In the interest of run-time eciency, metaprograms are compiled prior to execution. The instructions generated by the compiler for the example metaprogram

The Purdue University Network-Computing Hubs




19

27::1::set::::/ 59::2::retrievestate::directory 121::3::display::Page1::/home/users/punch/templates/page1.meta 167::4::test::167::5::277::8::-dir 193::5::chdir:: 256::6::display::Page1::/home/users/punch/templates/page1.meta 277::7::goto::121::4 306::8::savestate::directory

. . .

Fig. 6. The compiled version of the example metaprogram shown in Figure 5. This version is the one that is used by the network desktop at run-time. The format for the compiled version is designed to allow the metaprogram instructions to be retrieved and parsed in an ecient manner. It also allows metaprograms to be stopped and restarted as necessary (see text for details).

20




N. H. Kapadia, J. A. B. Fortes, and M. S. Lundstrom

in Figure 5 are shown in Figure 6. Each instruction in the compiled metaprogram consists of multiple elds that are separated by two colons, as shown in the gure. The number in the rst eld of any given instruction represents the byte oset of the following instruction. When a metaprogram is stopped after executing an instruction (e.g., display), this number represents the byte oset at which the metaprogram should be restarted. The number in the second eld identi es the relative position of the instruction in the metaprogram | it is used by the virtual machine to maintain a program counter. The information conveyed by the rst and the second elds is redundant | this allows the virtual machine to maintain correct operation in the presence of incorrect or missing access codes. The string in the third eld represents the name of the instruction; the set of valid strings is the instruction set supported by the virtual machine. The semantics of each of the remaining elds depends on the type of instruction. For example, in the case of the test instruction, the fourth and the fth elds represent the byte oset and the relative position of the instruction to be executed if the result of the test is true; the sixth and the seventh elds contain the address of the instruction to be executed if the test evaluates to either zero, false, or unde ned, using the evaluation semantics of Perl [Wall et al. 1996]. The nal eld in the instruction contains the expression that is to be evaluated for the test. This expression is represented in the pre x notation, and is executed by way of a stack machine. The ability to start and stop metaprograms at will has several advantages. For example, it allows the network desktop infrastructure to work within the framework of the stateless World Wide Web protocols. On the other hand, this ability makes it necessary to speci cally address information-consistency issues. Given the timedistributed nature of the execution process, it is possible that a metaprogram will be updated and recompiled by an administrator before it has been completely executed. In order to help detect the resulting information inconsistency, all compiled metaprograms contain a sequence number that represents the time of modi cation of the source metaprogram le. This sequence number is encoded into the program counter used to restart a metaprogram. During the startup procedure, the encoded sequence number is compared to the one contained in the compiled metaprogram and execution is aborted if the numbers do not match.4 5. EVALUATION AND RESULTS

The average response times of the PUNCH network desktop for several common types of transactions are shown in Table 3. Tool interface transactions (see 4 In principle, one could lock and/or copy metaprograms being executed. However, this does not guarantee consistency because a metaprogram may reference other metaprograms that could also have been modi ed.

The Purdue University Network-Computing Hubs




21

Table 3. PUNCH network desktop response times for dierent types of transactions. Static information transactions are equivalent to WWW document serving; consequently, the corresponding response times can be used to benchmark the times for the other transactions. Response Time (milliseconds)

Static Info

Tool Interface

Type of Transaction

No Preforking With Preforking

Entry Page

60

27

Input Management

61

27

Run Management

75

35

Output Management

61

27

Publicly Accessible URL

47

20

Access Control Enforced

53

24

Process Status

64

30

Dynamic Information

58

29

22




N. H. Kapadia, J. A. B. Fortes, and M. S. Lundstrom

table) consist of accesses that de ne run-speci c tool input parameters and initiate tool execution. They also include accesses to input and output data les. Static information transactions are made up of accesses that map to les that reside on a local lesystem. The information is static in the sense that there is no userspeci c view-customization associated with these accesses. Functionally, this is equivalent to traditional document serving on the World Wide Web. Process status transactions are generated when users attempt to check on or modify (e.g., abort) the status of their processes. Dynamic information transactions consist of accesses to information that must be preprocessed or customized in some manner before being presented to users. A typical use of this class of transactions is to personalize information for individual users. In order to lter out the eects of the network, the response times shown in the table were obtained with the client and the server running on the same machine (a 300MHz Ultra-4) and communicating via lehandles. The responses generated by the PUNCH server for the speci c transactions used to obtain the results shown in Table 3 ranged in size from about one to three kilobytes. Experiments show that, for the described setup and the given range of data sizes, the dierences in data transfer times are less than two milliseconds. (Large dierences and/or variations in data transfer times would adversely aect the reliability of the results.) Two main conclusions can be drawn from the data in Table 3. A comparison of the data across the two columns shows that there is a signi cant time penalty associated with forking a process in response to a (real-time) request. This illustrates the bene ts of preforking multiple network desktop processes at server startup. The second conclusion is that the Web interface management infrastructure described in this paper is quite ecient. This is exempli ed by the fact that the most expensive operation performed by the network desktop only takes about fty percent more time than serving a static HTML document. A more detailed analysis of the results is available [Kapadia 1999; Kapadia and Fortes 1999a; Kapadia et al. 1999]. 6. CONCLUSIONS AND QUANTIFIABLE IMPACT

The PUNCH infrastructure integrates the World Wide Web and distributed computing technologies to allow users to run arbitrary simulation tools via standard Web browsers. The semantics associated with an interface to computing services are supported via standard WWW browsers and protocols by treating URLs as addresses in a dynamic, virtual, and side-eect-based address space. Read and write capabilities within the Web-based operating system environment are provided by allowing variables and objects to be embedded within standard HTML. Flow control and interactivity management are supported via a programmable state machine. Results show that the described Web interface management infrastructure

The Purdue University Network-Computing Hubs




23

is ecient and has a exible design. Instructors from several universities (e.g., Arizona State University, Chicago State University, Northwestern University, Purdue University, Technion University in Israel, University of California at Berkeley, University of Illinois at Chicago, and University of Louisville in Kentucky) have already utilized PUNCH to allow their students to access and run specialized simulation tools. The School of Electrical and Computer Engineering at Purdue University routinely uses PUNCH in about nine undergraduate and graduate courses. We have found that PUNCH facilitates the use of simulation in classes by providing an easy, platform-independent means to access and run simulation tools. Many students use PUNCH through a modem from their residences rather than making use of on-campus computer labs where seats are frequently dicult to nd. Results from user surveys indicate that the system performs well under the highly peaked usage patterns (very high usage in the hours before homework and project assignments are due) characteristic of an academic environment. The ideas and solutions presented in this paper are based on (and validated by) our experiences in scaling PUNCH from a research project to a production system that is regularly used by several hundred students each semester. REFERENCES

Adasiewicz, C. 1995.

cess 9, 2, 10{11.

Exploratorium: User friendly science and engineering. NCSA Ac-

Arbenz, P., Gander, W., and Oettli, M. 1997.

allel Computing 23, 1421{1428.

Casanova, H. and Dongarra, J. 1996.

The Remote Computation System. Par-

NetSolve: A network solver for solving computational science problems. In Proceedings of the Supercomputing Conference (1996). Also Technical Report #CS-95-313, University of Tennessee. Chatterjee, S., Paramasivam, M., and Yakowenko, W. J. 1997. Architecture for a web-accessible simulation environment. IEEE Computer 30, 6 (June), 88{91. Citrix. 1996. ICA technical paper. WWW document at www.citrix.com/technology/. Kapadia, N. H. 1999. On the Design of a Demand-Based Network-Computing System: The Purdue University Network-Computing Hubs. Ph. D. thesis, Department of Electrical and Computer Engineering, Purdue University. Kapadia, N. H., Brodley, C. E., Fortes, J. A. B., and Lundstrom, M. S. 1998. Resource-usage prediction for demand-based network-computing. In Proceedings of the Workshop on Advances in Parallel and Distributed Systems (APADS) (West Lafayette, Indiana, October 1998), pp. 372{377. IEEE Computer Society. Kapadia, N. H. and Fortes, J. A. B. 1999a. The network desktop of the Purdue University Network Computing Hubs. Technical Report TR-ECE 99-1, Department of Electrical and Computer Engineering, Purdue University. Kapadia, N. H. and Fortes, J. A. B. 1999b. PUNCH: An architecture for web-enabled wide-area network-computing. Cluster Computing: The Journal of Networks, Software Tools and Applications 2, 2 (September), 153{164. In special issue on High Performance Distributed Computing. Kapadia, N. H., Fortes, J. A. B., and Brodley, C. E. 1999. Predictive applicationperformance modeling in a computational grid environment.In Proceedings of the 8th IEEE

24




N. H. Kapadia, J. A. B. Fortes, and M. S. Lundstrom

International Symposium on High Performance Distributed Computing (HPDC'99) (Redondo Beach, California, August 1999), pp. 47{54. Kapadia, N. H., Fortes, J. A. B., and Lundstrom, M. S. 1997. The Semiconductor Simulation Hub: A network-based microelectronics simulation laboratory. In Proceedings of the 12th Biennial University Government Industry Microelectronics Symposium (July 1997), pp. 72{77. Kapadia, N. H., Fortes, J. A. B., and Lundstrom, M. S. 1999. Statewide enterprise computing with the purdue university network computing hubs. In Proceedings of the 1st International Conference on Enterprise Information Systems (ICEIS'99) (Setubal, Portugal, March 1999). Kapadia, N. H., Robertson, J. P., and Fortes, J. A. B. 1998. Interface issues in running computer architecture tools via the world-wide web. In Proceedings of the Workshop on Computer Architecture Education at the 25th Annual International Symposium on Computer Architecture (ISCA'98) (Barcelona, Spain, June 1998). Lamine, S. B., Plaice, J., and Kropf, P. 1997. Problems of computing on the web. In Proceedings of the 1997 High Performance Computing Symposium (1997), pp. 296{301. Reinefeld, A., Baraglia, R., Decker, T., Gehring, J., Laforenza, D., Ramme, F., Romke, T., and Simon, J. 1997. The MOL project: An open, extensible metacom-

puter. In Proceedings of the 1997 IEEE Heterogeneous Computing Workshop (HCW97) (1997), pp. 17{31. Reynolds, F. D. 1996. Evolving an operating system for the web. Computer 29. Richardson, T., Stafford-Fraser, Q., Wood, K. R., and Hopper, A. 1998. Virtual network computing. IEEE Internet Computing 2, 1 (January-February), 33{38. Sato, M., Nakada, H., Sekiguchi, S., Matsuoka, S., Nagashima, U., and Takagi, H.

1997. Ninf: A network based information library for global world-wide computing infrastructure. In High-Performance Computing and Networking (Lecture Notes in Computer Science, 1225), pp. 491{502. Berlin: Springer-Verlag. Souder, D., Herrington, M., Garg, R. P., and DeRyke, D. 1998. JSPICE: A component-based distributed Java front-end for SPICE. In Proceedings of the 1998 Workshop on Java for High-Performance Network Computing (1998). Wall, L., Christiansen, T., and Schwartz, R. L. 1996. Programming Perl (2nd ed.). O'Reilly & Associates. X Consortium, Inc. 1996. X Window System Version 11 Release 6.3: Release Notes. X Consortium, Inc.