WWWorkflow: World Wide Web based Workflow - CiteSeerX

WWWorkflow: World Wide Web based Workflow Charles K Ames, Scott C Burleigh, Stephen J Mitchell Jet Propulsion Laboratory {chuck,scott,steve}@tel.jpl.nasa.gov Abstract This paper describes WWWorkflow, a system developed at the Jet Propulsion Laboratory for the computer mediation of work through an organization. WWWorkflow exploits an opportunity created by organizational intranets to provide a common user interface across heterogeneous platforms. A distinctive feature of WWWorkflow is the careful separation of process mediation from product data management. WWWorkflow supports many types of workflow and is not focused on any single information structure. Three major components of the system are presented: a relational database approach to process definition and management, process initiation and event response software used to drive the flow of work through the organization, and a World Wide Web (WWW) based user interface and mechanism for integrating workflow with existing data management services. Lessons learned from its deployment at the Johnson Space Center, Houston are discussed.

1. Introduction Proactive information systems that do not allow users to make mistakes or omit steps of a task are becoming common. Two examples of such software are Product Data Management (PDM) and Workflow Management systems [1]. This paper describes WWWorkflow, a system developed at the Jet Propulsion Laboratory (JPL) for computer mediation of work through an organization. A distinctive feature of WWWorkflow is the careful separation of process mediation from product data management. Product Data Management is by definition product focused. PDM tools typically provide capabilities for controlling access, preserving revision histories, synchronizing updates, and establishing semantic relationships among data products. Some PDM allow the definition of arbitrary actions which are triggered by particular transactions, such as “check for the existence of a product of type Y before allowing a product of type X to be checked out”. These triggers implement a limited form

of workflow management with respect to the data products managed by the PDM tool. Workflow Management is process focused— coordinating activities of people working on a common task or project. Where PDMs attempt to prevent people from making mistakes, workflow management tools attempt to prevent people from forgetting to do things. Once a process is defined, a workflow management system makes sure that activities occur in proper sequence, and that users are informed so they will complete tasks on time. For example, rather than relying on Joe to tell Sally and Jeff that his part of the job is done and they should start their parts, Workflow Management systems do the notification automatically and immediately. Most commercially available products marketed as either PDM or Workflow Management systems are really a mix of the two. Most often they are PDM tools with triggered actions. Such systems can implement effective workflows, but only for products managed by the PDM system. They are completely ineffective for flows which involve products that are not or cannot be managed by the PDM. These systems solve specific data management problems very well. However, assumptions built into them about the problems they will be used to solve and the environment in which they will be deployed limit their flexibility as general purpose workflow management systems. WWWorkflow was designed to provide a lowcost, general purpose, add-on process mediation service using widely available tools. This paper is organized as follows: first, an architecture for a World Wide Web (WWW) based Workflow Management system is presented. Next, the supporting database and workflow language is discussed, followed by descriptions of the user interface and typical use scenarios. Finally, results and lessons learned are summarized.

1.1. Workflow for Intranets The purpose of Workflow Management systems is coordinate activities of people. Many existing Workflow Management systems provide very good support for

Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE

1060-3425/97 $10.00 (c) 1997 IEEE

process engineering but limited support for process execution, often requiring additional proprietary desktop applications with limited platform availability. Unfortunately, process engineering [2,3] cannot be effective unless the results are accessible to the people who must carry out the processes. The WWWorkflow system provides a process mediation service using tools that most users already have and know how to use. Users interact with the system using their Web browser to retrieve a list of tasks for which they are currently responsible and read Task Descriptions about assignments. In its “nag daemon” role, WWWorkflow uses email and pagers to notify people when they need to do something. Many productivity applications are emerging on enterprise Intranets, and more are being developed. WWWorkflow is designed to interoperate seamlessly with Intranet applications. The architecture of WWWorkflow allows workflow management to be added to pre-existing data and information management systems without requiring significant modifications to the underlying system. For example, a WWWorkflow procedure called “Process Problem Report” can be defined and implemented to interact with an institutional Problem Report (PR) system, notifying managers, engineers, and testers when a particular PFR needs their attention. This provides a link directly to an automatically generated HTML form for editing that PR. Such procedures are being developed at JPL.

to notify the workflow engine of activities the user has performed. Figure 1 shows the system architecture as deployed at Johnson Space Center (JSC), Houston in their Change Request (CR) system. The system drives a document change request process used to approve changes to Space Shuttle documentation. The document change request workflow requires that proposed changes to existing documents be reviewed by several management offices and committees before being enacted. The CR must be routed to several participants with specific instructions regarding what actions are required of them at a given step in the CR process. Shown in the diagram are both the workflow system and the document change request system. While WWWorkflow is used to embody and execute change request procedures, the system is carefully separated from the product data within the CR system itself. The CR system is distinct from the WWWorkflow system, utilizing its own programs, CR database, and supporting documentation. Also shown is the optional communication with existing enterprise authentication services. The workflow engine is described in the next section. Descriptions of the users’ interactions via their web browser and CGI interface are described in detail in the Web Gateway section.

auth service

browser

HTTP

2. Architecture The WWWorkflow system architecture contains three main parts:

shell

engine TCP Socket

CGI TCP Socket

HTTP

httpd

httpd CGI

CR System TCP Socket I/O

DBMS

rdbms

1.

2. 3.

A Common Gateway Interface (CGI) [4] front end allowing users to interact with the system via a browser. A workflow “engine” which sends and receives messages into and out of the workflow system. A database which contains predefined workflow procedures, status information for procedures underway, and complete histories of previously executed procedures.

The user interface to the system is the user’s web browser software. The web browser uses the Hyper Text Transfer Protocol (HTTP) [5] to send requests and receive documents from an HTTP daemon process. The HTTP daemon helps the user interact with the workflow system by calling the CGI program responsible for WWWorkflow activities. The CGI program communicates both with the workflow engine and with the workflow database to retrieve instructions for the user and

I/O

wfdb

HTML

www_cr

HTML

task descriptions

crdb

Figure 1. System architecture

3. The Workflow Engine The WWWorkflow system relies on a workflow “engine” to drive the flow of work and manage the workflow database. The workflow engine, along with a simple “shell” program for sending commands to the engine, is used to adjust and modify the workflow system in an administrative fashion or to write ad-hoc procedures outright. This and the next two sections describe the system’s engine, database, and Workflow Language (WFL). The workflow engine is a program which uses a relational database to store information about procedures that have been defined or are already in motion. The


1060-3425/97 $10.00 (c) 1997 IEEE

engine is both commanded and programmed via WFL. To load new procedures into the system, WFL commands describing the procedures are sent to the engine. To start instances of procedures or edit procedures already underway, WFL commands are sent to the engine. The engine is an enterprise resource for workflow activities in much the same way a relational database server is an enterprise resource for database activities. The workflow engine is programmed and commanded with WFL in much the same way a relational database server is seeded and commanded with Structured Query Language (SQL). Since workflow is about “making a plan and putting it into motion,” the objective of the workflow engine’s design is to improve an organization’s performance in doing planned work. Specifically it aims to minimize error and delay in the progress of a plan instance, enabling users to know the current status of any instance of any plan at any time; allowing users to analyze plan completion history to aid in improving the plans. For the purpose of this paper, a “plan” is a guideline describing how certain work is to be done in an organization, “processes” are a models of these plans, and a “procedure” is a template of a process—ready to run in our workflow system. The engine is designed to facilitate process participation, process management, and process engineering. Some existing workflow management systems already provide some or all of these capabilities. The workflow engine derives its unique character from constraints that were placed on its design. These included the following: •

• • •

The supported workflow management had to be generic, i.e., fully programmable by process engineers and applicable to all types of processes (rather than only to those that related to some predefined data product or class of data products). Seamless integration of automated and manual work. Protection of data integrity, yet virtually unlimited query access. Minimal development and operational cost.

The workflow engine runs as a UNIX daemon process on a TCP/IP network-accessible server. Its function is to manage the contents of a relational database, updating process data in that database in response to commands received over network connections. The database contains both designs for plans and also every instance of execution of each plan in the database. Commands, expressed in “Workflow Language” (discussed in the WFL Overview section later in this paper), enable WWWorkflow users to manage workflow in an organization by:

•

Defining the terms (“statuses”) used to express progress through a plan that is being executed.

•

Creating new designs for plans (“procedures”) and adding them to the database. Each procedure consists of one or more “steps”. A step is an action carried out by a user of the system or by an automated process. Every step in a procedure can be “iterated.” Each iteration of a step is a “task.” Each procedure may also have an arbitrary number of operands whose values constitute the procedure memory. An operand serves roughly the same purpose within a procedure or initiative as a variable in a computer program. It provides a common, accessible place to store something (in WFL, a character string) that multiple tasks of the procedure may refer to. Finally, associated with each procedure is a “terminal condition” stating what combination of task completion statuses must hold true to declare the procedure itself completed.

•

Creating instances of those procedures (“initiatives”) and adding them to the database. Each initiative is an exact copy of the procedure it instantiates (the same terminal condition, encompassing the same tasks, and endowed with all the same operands) except for any initial modifications specified at the moment of instantiation.

•

Setting initiatives in motion. When an initiative is started, each of its tasks having no precondition is automatically activated: automated tasks are executed immediately, and the assignees for all non-automated tasks are notified that they are authorized to start work on them.

While an initiative is in progress it can be edited: task assignees can be changed, reference text can be modified, preconditions can be changed, step iterations can be added, even entire steps can be added or deleted. For example, in a document CR system, the assignee to a specific task might not be known until the document has undergone some initial review, the current version of a supporting document may have been changed since the plan was put into motion, or the number of authorization signatures required for a document to be released might have been increased since the plan was started. All such editing is performed by the workflow engine in response


1060-3425/97 $10.00 (c) 1997 IEEE

to commands written in WFL. Because the activation scripts of automated tasks are themselves WFL commands, initiatives can automatically modify themselves in response to conditions that arise as the initiatives progress. When the user responsible for a given task (the “assignee”) determines that it has reached some terminal state (for example, the document has been reviewed, or cannot take on more tasks), the user “stops” that task: that is, the assignee notifies the workflow engine of the task’s completion status. This automatically causes the engine to activate every other task whose precondition is now satisfied by the completion of this task. In this way, the workflow engine automatically both guides and tracks the progress of the initiative until the procedure completes. While the initiative is in progress, its state can be examined simply by using SQL to query the database. After termination, cost and effectiveness can be analyzed with reference to the procedure it instantiated and/or other similar initiatives. The system is set up to allow only the workflow engine to write to the workflow database, while other users are allowed only read permission for reporting purposes. None of this database interrogation involves the workflow engine or requires the use of WFL—the engine is needed only for activities that update the workflow database, and WFL is used only to drive the engine.

3.1. Workflow Database The workflow engine operates on these data objects: statuses, procedures, initiatives, operands, steps and step iterations (tasks). Rows in the “Status” table contain status names, labels, and descriptions. The label of a status is used to label user interface buttons on an interface panel that a workflow participant uses to declare the completion status of a task. Description provides a detailed explanation of the meaning and significance of the status. It may be simple text, or it may be a URL pointing to an elaborate Web hypertext page. The “Procedure” table contains both procedures (abstract workflow plans) and initiatives (the plans themselves). Each row in the table contains name, discussion, start script, and stop script. Discussion, like the “description” of a Status, may be either simple text or a URL. The initiative can have optional start and stop scripts. Initiatives additionally contain start and stop times. Each “Operand” consists of a reference to a single procedure or initiative, a name, a value (always an ASCII text string), and an optional explanatory discussion (simple text or a URL).

Each row of the “Step” table simply names a procedure step of which there may be any number of distinct iterations. Steps serve only to group together tasks tat are essentially minor variants of the same activity. Each row of the “Task” table contains a discussion (simple text or a URL), an assignee, priority, an effort estimate (as a guide to the assignee), activation script, start time, stop time, completion status, completion status discussion, and optionally a time-out interval and script. Activation script is an optional WFL command, executed upon activation of the task, that might either notify the assignee that the task is active or else perform some automated function. Time-out interval indicates a length of time following task activation, at the end of which the time-out script (a WFL command) will be executed unless the task is already in some completion status. The prescribed possible completion statuses for tasks are an unbounded repeating group, so they occupy a table of their own named “TaskOption”. Each row of this table merely pairs a reference to some Task with a reference to a Status, indicating one of the Statuses into which it is expected that the Task will be placed on completion. The precondition for activation of a task can be arbitrarily complex. For this reason, the representation of preconditions on the database requires three tables: Term, SubTerm, and TermSql. Each row of the “Term” table expresses a single Boolean term of the precondition expression: either a simple positive or negative assertion (either pairing some Task—within the same initiative or any other—with some Status or else asserting that some database relationship holds) or a conjunction (AND) or disjunction (OR). To do so it contains a logical term type and, if a Task or Status assertion, references to the relevant task and status. If the term is a database relationship assertion, then the SQL query text that must be executed to evaluate this assertion occupies a row in the separate TermSql table. If the term is a conjunction or disjunction, the subsidiary terms that must be tested in order to evaluate the term (which constitute an unbounded repeating group) occupy rows in the SubTerm table; each SubTerm is simply a reference to some other term.

3.2. Workflow Language (WFL) Overview WFL is a dialect of Tcl, the Tool Command Language devised by John K. Ousterhout [6] and augmented by the WFL-specific commands discussed below: NEW, EDIT, DELETE, START, STOP. Subsets of these commands can be applied directly to objects of the fundamental types in the workflow database and may collaterally affect any number of objects of other types.[7] The NEW command can be applied to all object types. It creates a object of the indicated type, inserts it into the


1060-3425/97 $10.00 (c) 1997 IEEE

database, and then sets its attributes to the indicated values. The Tcl return value produced by NEW is the object ID of the newly created object. The EDIT command can be applied to objects of all types. The EDIT command’s first argument is an object ID, and on success the Tcl result it returns is always that same object ID. The EDIT command is used to modify attributes of database objects through the workflow engine. The DELETE command can be applied to objects of all types. The effect of a DELETE command is simply to remove from the database the indicated object and every other object whose qualified name begins with the qualified name of the deleted object. Starting an initiative automatically activates every task included in the initiative whose precondition is true. Tasks are activated automatically in response to the starting of an initiative or the stopping of some other task. The iteration START command is provided to activate a task that is added after its parent initiative has been started. An initiative is normally stopped automatically when the workflow engine determines that its stop condition is met; the command to stop an initiative explicitly is provided as a convenient way to shut down an initiative that has been entirely canceled for some reason. Stopping a task sets its stop time to the current time, sets its completion status, and automatically activates all other inactive tasks whose preconditions are now true. It will stop the entire initiative if the initiative’s stop condition is now met. The stop condition of an initiative (inherited from its parent procedure) and the activation preconditions of a task are string attributes for which WFL defines a separate syntax. A condition expression can be either the word NONE or a CONDITION TERM. The NONE condition expression has different meanings depending on where it is used: when used as a task precondition it means the task is immediately eligible for activation, but when used as a plan’s terminal condition it means that the plan can never be stopped automatically in response to the completion of any task or combination of tasks. An assertion expression simply specifies the exit condition of a procedure, step, and iteration that must be true for the assertion to be true. Condition expressions can be arbitrarily complex, specifying that steps should be activated based on the completion statuses of many different steps and procedures within the workflow system.

uses Common Gateway Interface (CGI) technology to provide an interface to the users. The module is written in Perl. [8] It has been shown that CGI programs can provide effective user interfaces to complex systems.[9] To interact with the system, the user directs a web browser to access the Uniform Resource Locator (URL) of the CGI program that will act as the user’s interface to the organization’s workflow. The CGI program interrogates the user for authentication and identification information. The main menu contains options to interact with the user's current “todo” list, start previously defined procedures, and view the status or historical information of procedures.

4.1. The “Todo” List The user’s “todo” list is the result of a query to the workflow database. The query finds all active instances of steps within active procedures for which the user is currently responsible. The query is performed by the CGI front-end to the system upon the end users’ request. The query might also be performed due to some trigger within a procedure underway and (for example) emailed to an end user. The list presents the user with information regarding each step such as: • • • • • • • •

Priority assigned to the task Time and date the task began Estimated time required to complete the task. Time and date the completion of the task is due. Name of the procedure to which the task belongs. Name of the step itself. Location (a hypertext link) of the HTML page that describes the work. Name of the user who initiated the procedure.

The user’s todo list is at the top of a “todo” screen that includes three HTML frames. The top frame is scrollable and contains a table showing the todo list. When the user first accesses the todo screen, the todo list populates the top frame while the other two frames are blank.

4. Web Gateway The critical component necessary to put WWWorkflow to work is the Web Gateway to the workflow engine and database. The WWWorkflow system


1060-3425/97 $10.00 (c) 1997 IEEE

4.2. Starting Procedures Previously defined procedures are started from a WWWorkflow page. A user can view a scrolling list of procedures previously defined. By selecting a procedure from the list and pressing an “init” button, the user sends a command to the workflow engine to begin the initialization of the procedure. The user supplies any operands to the procedure, then starts or activates the procedure. The workflow engine receives a start command and the procedure is underway.

4.3. Status of procedures underway

Figure 2. An example “Todo” list If a user selects the hypertext link for an individual task description, information specific to that task populates the second and third frames. The second frame or “workspace” frame shows the HTML document used to describe the task. The text or URL for the workspace frame is retrieved from the “discussion” attribute of the task’s database object. The task description information should contain general information regarding the task, including how to complete the task. The task description comes from the Task Description Library, described below. If other WWW based resources are available to help complete the task, links to those resources can be included in the workspace frame. As users select links to other resources, those resources are shown in the workspace frame while the “todo” list frame at the top of the screen and the third frame at the bottom of the screen remain intact. The third frame on the user’s screen shows one button for each of the prescribed completion statuses for the task at hand. The label and value for the buttons are retrieved from the “label” attributes of the task’s TaskOption database objects. The user selects one of these completion statuses when he has completed the task. The selection of a completion status button will cause the CGI to send a completion command to the workflow engine for that particular task. This is one of the ways the underlying workflow engine is notified of user participation, and helps to drive the workflow system.

Since many procedures are likely to be underway at any given time, the system facilitates inquiries to the status of certain procedures as they are being executed. The user may want to know why a particular procedure has not yet been completed, and will be able to identify the task and its associated responsible user that are holding up the procedure. Users inquire about the status of procedures underway by accessing a scrolling list of currently active instantiations of procedures. After selecting a procedure from the list and pressing a button, they are shown a report—the result of several queries to the workflow database—showing the status of every task for the particular instantiation of the procedure they selected.

4.4. Historical information on procedures After each procedure is completed, all information relating to that procedure and all tasks that were carried out to complete that procedure remain on the database, forming histories of the procedures. These histories can be queried using SQL and analyzed to determine how many times a particular procedure has run, how long it took on average to complete, average time each task within the procedure took to complete, which tasks were most time consuming, etc. This historical reporting information provides valuable feedback for process engineering.

4.5. Task Description Library Individual tasks within a procedure are documented with HTML files. These use hypertext links to help users begin working on a certain task. A process engineer authors these documents when defining the procedure. Files in the Task Description Library (TDL) are also part of on-line procedure manuals or other worker documentation.


1060-3425/97 $10.00 (c) 1997 IEEE

5. Integration with other systems The hypertext nature of the WWW is critical to WWWorkflow. A user’s todo list is a jumping off point from the workflow system to other enterprise systems. The TDL not only provides a user with instructions on how to perform a task, but provides links to other resources that can aid or directly facilitate task completion. Since it is often necessary to carry state information out of the workflow system and pass it on to external resources, HTML documents are served up via an intermediate CGI gateway program which scans text for special ‘variables’ and replaces them with the values of certain operands for the currently active initiative. For example, a workflow procedure to add a new employee started with a supplied operand of ‘name’ given a value of ‘John Smith’, the HTML document for a given step might contain the embedded string: Add $$name to the department phone list. The text would then show up on the user's workspace frame as: Add John Smith to the department phone list. Additionally, operand values are arguments to other CGI programs within an organization. Supplied as an argument, an operand carries state information for a current procedure out of the workflow system and into external systems. For example, to carry the current document number for a newly created document Change Request (CR) out of the workflow system and into the documentation section’s WWW based CR system, the following link might be embedded in the TDL page for the ‘create new CR’ step of a procedure: Create New CR Now The above example will show a hypertext link to the user: “Create New CR Now”, and will automatically pass the value of the current CR request number ($$req_number) to the CGI program traditionally used to create new CR documents. It is also frequently necessary to import state information from external resources into the workflow system in order to determine what steps should be activated next. One approach is to define automated steps which “activate” scripts that interrogate some external resource, then edit some part of the current procedure

based on the result. For example, In JSC’s CR system, approval authorities are identified in the workflow procedure by title, while the mapping from title to person is tracked in the CR system itself. Therefore, the workflow system must query the CR system to determine what person to assign “approve” steps to. This is facilitated by the workflow engine’s use of Tcl as the language for “activate” scripts: Tcl extensions for interoperation with various external resources can easily be added to the workflow engine’s Tcl interpreter.

6. Summary This paper has described a workflow management system deployed with a World Wide Web interface. The system consists of three major components. First, a workflow database, contains database objects that embody procedure designs and procedures already underway. The second component, a workflow engine, drives workflow processes and uses a workflow database to store state information and analyze progress. A final element is a CGI based user interface and mechanism for integrating workflow with existing data management services. WWWorkflow is currently being deployed at the Johnston Space Center (JSC), Houston. The project will drive a document change request workflow for approval of changes to Space Shuttle documentation. Our experiences in the project show the following: •

The system is flexible. While the system was initially prototyped on Sun Solaris systems, requirements at JSC dictated that WWWorkflow be deployed on Microsoft NT based servers. Since WWWorkflow is built around widely available and easily recompiled components, deployment on NT based systems was not difficult..

•

Designers of workflow plans must be careful to separate product data management from the flow of work. It is tempting to view the workflow database as a place where product specific information is stored. However, the open nature of WWWorkflow allows product information to be managed separately from the workflow procedures. This approach allows organizations to deploy workflow along side existing information systems, rather than being forced to abandon existing infrastructure to accommodate a closed workflow system.


1060-3425/97 $10.00 (c) 1997 IEEE

[7] S. Burleigh, “Work Flow Language User’s Guide,” [http://tel.jpl.nasa.gov/wwwork/doc/wfl_guide/], February 1996.

7. Future Work Current areas of research include the development of a World Wide Web based workflow authoring tool. This tool will enable users to graphically compose workflows which can be given directly to the WWWorkflow system for storage and activation. Cognitive models to describe the flow of work through an organization are being investigated.

[8] L. Wall and R. Schwartz, Programming Perl, O’Reilly & Associates, Inc., 1990. [9] J. Rice, A. Farquhar, P. Piernot, & T. Gruber, “Lessons Learned Using the Web as an Application Interface,” Knowledge Systems Laboratory, KSL-95-69, September 1995.

8. Acknowledgments The work described in this paper was performed at the Jet Propulsion Laboratory, California Institute of Technology, under contract with the National Aeronautics and Space Administration. The authors acknowledge David Hermsen, manager of the Test Engineering Lab at JPL, the Advanced Technology Information Network at California State University Fresno for support of this work, and Brent Auernheimer and Harry Avant for tirelessly making suggestions about the manuscript.

9. Availability The URL for the documentation of the system is [http://tel.jpl.nasa.gov/wwwork/].

10. References [1] A. Kumar and J. Leon Zhao, “A Framework for Dynamic Routing and Operational Integrity Controls in a Workflow Management System,” Proceedings 29th Hawaii International Conference on System Sciences, IEEE Computer Society, January 1996. [2] H. Lucas, D. Berndt, and G. Truman, “A Reengineering Framework for Evaluating a Financial Imaging System,” Communications of the ACM, May 1996. [3] M. Manheim, “Opportunities for Competitive Advantage in Logistics through Information Technology,” Proceedings 29th Hawaii International Conference on System Sciences, IEEE Computer Society, January 1996. [4] “The Common Gateway Interface” [http://hoohoo.ncsa.uiuc.edu/cgi]. [5] T. Berners-Lee, R. Fielding, and H. Frystyk, “Hypertext Transfer Protocol—HTTP/1.0,” RFC 1945, May 1996. [6] J. Ousterhout, TCL and the Tk Toolkit, Addison Wesley, 1994.


1060-3425/97 $10.00 (c) 1997 IEEE