my committee, Roger King, Ben Zorn, Alex Wolf and Marcia Derr, for their critiques, comments and time. ...... U. Hahn, M. Jarke, et al. CoAUTHOR: A hypermedia ...
A MULTISTATE SERVICE BASED ON DELTAS AND ITS APPLICATION TO SUPPORT COLLABORATIVE WORK by Michael Doherty B.S., University of Florida, 1983 M.S., University of Rhode Island, 1992
A thesis submitted to the Faculty of the Graduate School of the University of Colorado in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Computer Science 1998
This thesis entitled: A Multistate Service Based on Deltas and Its Application to Support Collaborative Work written by Michael Doherty has been approved for the Department of Computer Science
Richard Hull
Roger King
Date
The final copy of this thesis has been examined by the signatories, and we find that both the content and the form meet acceptable presentation standards of scholarly work in the above mentioned discipline.
Doherty, Michael (Ph. D., Computer Science) A Multistate Service Based on Deltas and Its Application to Support Collaborative Work Thesis directed by Visiting Associate Professor Richard Hull In many applications, the representation and management of change is of primary interest, often being as important as the representation and management of state. The concurrency control policies of database management systems, which are focused on the maintenance of single consistent states, present an obstacle to the support of such applications. The research presented here has resulted in the development of theory to support new data storage techniques that overcome such obstacles without losing the other benefits provided by database systems. This theory, called Heraclitus[OO], is centered on the use of delta values, which capture state changes as first class objects available for manipulation by applications and users. The theory has been applied in the development of the H2O/SST data storage manager. In the general area of cooperative work support, authoring systems and related applications are in particular need of support for flexible management of change as well as support for the resolution of resulting conflicts. To this end, this research has also developed an architecture by which H2O/SST can be used to support such applications. This architecture, called Coral, is designed to allow for the encorporation of application specific policies regarding the management of change and conflict. The principles of Coral have been demonstrated through the implementation of a prototype authoring system. Additionally, the development of applications based on Coral and H2O/SST is illustrated through an example system design. This system design includes the design of procedures to support authoring through delta values and the specification of particular delta values to support text documents.
iv ACKNOWLEDGEMENTS I owe a great deal of thanks to the many people who made technical contributions to the work leading to and contained in this dissertation. Foremost, a heap of thanks to Rick Hull for introducing me to Heraclitus and for leading and supporting my efforts. Great thanks to Brian Otte and Mohammed Ruppawalla for their tremendous efforts realizing the code to implement my (occassionally useful) ideas. Thanks to Skip Ellis and Carlos Maltzahn for thoughts on groupware architectures. Thanks to Julie DiBiase for many interesting conversations on the nature of objects and functions that were instrumental in my initial conceptualizations of H2O. Thanks to the members of my committee, Roger King, Ben Zorn, Alex Wolf and Marcia Derr, for their critiques, comments and time. Finally, thanks to Roger and Rick Osborne for their considerable assistance in managing the nontechnical aspects of this research. This work would not have been realized without all the people who contributed encouragement and support, endured my rantings, helped check my sanity and made life a pleasure over the past few years. For all this and more I’m grateful to Julie DiBiase, Joan Peckham, Carlos Maltzahn, Missy Schreiner, Brain Otte, Gina Cherry, Christian Och, John Todd and Steve Doherty. This research was supported in part by NSF grants IRI-931832 and IIS-931-8326 and ARPA grants BAA-92-1092, 33825-RT-AAS and DAAH0494-G0382.
CONTENTS CHAPTER 1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . .
1
1.1
Overview of the Research . . . . . . . . . . . . . . . . . . . . .
1
1.2
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
1.3
An Example Application . . . . . . . . . . . . . . . . . . . . .
11
1.4
Organization of the Dissertation . . . . . . . . . . . . . . . . .
16
2 H2O: A DELTA-BASED MULTISTATE DATA STORE . . . . . .
17
2.1
An Example Delta Form . . . . . . . . . . . . . . . . . . . . .
18
2.2
Delta Forms and Requirements . . . . . . . . . . . . . . . . .
25
2.3
The H2O State Delta Form . . . . . . . . . . . . . . . . . . .
33
2.4
Delta Values and Multistates . . . . . . . . . . . . . . . . . . .
46
2.5
H2O Implementation . . . . . . . . . . . . . . . . . . . . . . .
53
3 SST: AN INTERFACE FOR COLLABORATION CONTROL . . .
65
3.1
SST Definition . . . . . . . . . . . . . . . . . . . . . . . . . .
67
3.2
SST Operation . . . . . . . . . . . . . . . . . . . . . . . . . .
69
3.3
Notes on Concurrency . . . . . . . . . . . . . . . . . . . . . .
75
4 INCORPORATING H2O/SST INTO SYSTEMS TO SUPPORT COLLABORATIVE WORK . . . . . . . . . . . . . . . . . . . . . .
77
4.1
The Coral Architecture for Delta-Based Groupware . . . . . .
78
4.2
Reef: A Prototype Authoring System . . . . . . . . . . . . . .
82
4.3
Eniwetok: A Coral Application . . . . . . . . . . . . . . . . .
85
4.4
A Delta Form for Documents . . . . . . . . . . . . . . . . . .
95
5 CONCLUSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
109
5.1
Summary of Contributions . . . . . . . . . . . . . . . . . . . .
109
5.2
Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . .
112
5.3
Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . .
119
BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
123
vi APPENDIX A A DELTA FORM FOR SEQUENCES . . . . . . . . . . . . . . . .
131
FIGURES FIGURE 1.1
Work Process for Collaborative Authoring . . . . . . . . . . . . .
7
1.2
Example Authoring System Interface. . . . . . . . . . . . . . . . .
11
1.3
Overview of System Architecture. . . . . . . . . . . . . . . . . . .
15
2.1
An Integer Class Specification in C++. . . . . . . . . . . . . . . .
19
2.2
An Integer Class Implementation in C++. . . . . . . . . . . . . .
19
2.3
A Program Fragment Operating on an Integer Object. . . . . . .
19
2.4
Replacement Methods for Integer Class Deltafication. . . . . . . .
21
2.5
Deltafication Operation for Example Integer Delta Form. . . . . .
21
2.6
Smash Operation for Example Integer Delta Form. . . . . . . . .
23
2.7
Merge Operation for Example Integer Delta Form. . . . . . . . .
23
2.8
Apply Operation for Example Integer Delta Form. . . . . . . . .
24
2.9
Delta Form for Object Granularity Deltas. . . . . . . . . . . . . .
28
2.10 A Delta Form for Sets. . . . . . . . . . . . . . . . . . . . . . . . .
30
2.11 Analysis of Smash and Apply Operations on Set Delta Form. . .
32
2.12 Class Specification for an Object-Oriented Database State. . . . .
34
2.13 Delta Values for Database State. . . . . . . . . . . . . . . . . . .
35
2.14 Deltafication Process for Database State. . . . . . . . . . . . . . .
36
2.15 H2O State Smash Function. . . . . . . . . . . . . . . . . . . . . .
38
2.16 H2O State Merge Function. . . . . . . . . . . . . . . . . . . . . .
39
2.17 H2O State Apply Function. . . . . . . . . . . . . . . . . . . . . .
40
2.18 Analysis of Deltafication and Apply on State Delta Form. . . . .
41
2.19 Analysis of Smash and Apply on State Delta Form (Part 1). . . .
42
2.20 Analysis of Smash and Apply on State Delta Form (Part 2). . . .
43
2.21 Analysis of Smash and Merge on State Delta Form. . . . . . . . .
44
2.22 A State Specified by a Base State and Delta Values. . . . . . . .
47
2.23 A State Specified by an Empty State and Delta Values. . . . . . .
48
viii 2.24 StoreDelta Function. . . . . . . . . . . . . . . . . . . . . . . . . .
49
2.25 Eager Implementation of Object Evaluation Function. . . . . . .
51
2.26 Lazy Implementation of Object Evaluation Function. . . . . . . .
52
2.27 H2O System Architecture. . . . . . . . . . . . . . . . . . . . . . .
54
2.28 Classes Required to Support Delta Forms. . . . . . . . . . . . . .
56
2.29 ODL Specification of Paragraph Class. . . . . . . . . . . . . . . .
57
2.30 ODL Generated Paragraph Class. . . . . . . . . . . . . . . . . . .
57
2.31 ODL Generated Paragraph Surrogate (Part 1). . . . . . . . . . .
58
2.32 ODL Generated Paragraph Surrogate (Part 2). . . . . . . . . . .
59
2.33 ODL Generated Paragraph Surrogate (Part 3). . . . . . . . . . .
60
2.34 ODL Generated Paragraph Delta Value. . . . . . . . . . . . . . .
61
2.35 Deltafication of Concurrent Applications. . . . . . . . . . . . . . .
62
3.1
A State Specification Tree . . . . . . . . . . . . . . . . . . . . . .
67
3.2
System Architecture of H2O/SST. . . . . . . . . . . . . . . . . .
69
3.3
State Specification Tree Interface. . . . . . . . . . . . . . . . . . .
70
3.4
SST NewState Method. . . . . . . . . . . . . . . . . . . . . . . .
72
3.5
SST Copy Method. . . . . . . . . . . . . . . . . . . . . . . . . . .
73
3.6
SST Delete Method. . . . . . . . . . . . . . . . . . . . . . . . . .
73
3.7
SST Split Method. . . . . . . . . . . . . . . . . . . . . . . . . . .
74
3.8
SST Smash Method. . . . . . . . . . . . . . . . . . . . . . . . . .
74
3.9
SST Merge Method. . . . . . . . . . . . . . . . . . . . . . . . . .
75
3.10 SST PartialMerge Method. . . . . . . . . . . . . . . . . . . . . .
75
4.1
Coral Architecture for Cooperative Work Applications. . . . . . .
79
4.2
The Reef Prototype Authoring System. . . . . . . . . . . . . . . .
83
4.3
SST Operations to Support Editing With Undo. . . . . . . . . . .
87
4.4
Conflict Resolution: Partial Merge. . . . . . . . . . . . . . . . . .
90
4.5
Conflict Resolution: Manual Resolution. . . . . . . . . . . . . . .
91
4.6
Conflict Resolution: Copy. . . . . . . . . . . . . . . . . . . . . . .
91
4.7
Conflict Resolution: Smash. . . . . . . . . . . . . . . . . . . . . .
92
4.8
Conflict Resolution: Merge. . . . . . . . . . . . . . . . . . . . . .
93
4.9
System Operation to Support Conflict Detection Agents. . . . . .
94
ix 4.10 An Example Document. . . . . . . . . . . . . . . . . . . . . . . .
96
4.11 LDM Schema for Hierarchical Documents. . . . . . . . . . . . . .
97
4.12 Object Schema for Hierarchical Documents. . . . . . . . . . . . .
99
4.13 Updates Violating Paragraph Constraint. . . . . . . . . . . . . . .
104
4.14 Document Structure Constraint Example. . . . . . . . . . . . . .
106
4.15 Document Structure Constraint Example. . . . . . . . . . . . . .
107
A.1 Class Specification for Sequence Data Type. . . . . . . . . . . . .
132
A.2 Delta Values for Sequences. . . . . . . . . . . . . . . . . . . . . .
133
A.3 Deltafication Operation for Sequence Objects. . . . . . . . . . . .
134
A.4 Apply Operation for Sequence Delta Form. . . . . . . . . . . . . .
135
A.5 Smash Operation for Sequence Delta Form. . . . . . . . . . . . .
137
A.6 Positioning Set Modification Algorithms for Sequence Delta Form.
138
CHAPTER 1 INTRODUCTION 1.1
Overview of the Research In many applications, the representation and management of change
is of primary interest, often being as important as the representation and management of state. Traditionally, database management systems (DBMSs) have focused on the management of state as the fundamental representation of some enterprise or artifact. While this focus is appropriate for the realm of applications handled by such an approach, it also limits the range of applications by introducing system enforced policies on concurrent updates to the state. These concurrency control policies are intended to ensure the consistency of the database state, in terms of the semantics of the enterprise or artifact that the state represents, by explicitly preventing the concurrent creation of updates that collectively violate the consistency of the state. Two updates which together introduce some inconsistency in the state are said to conflict. However, in situations where change is of greater interest than state, it is often the case that the temporary existence of conflicting updates is both natural and beneficial. One family of applications in which change is a primary concern and the temporary existence of conflicting updates is useful is systems that support cooperative work. Such systems are also referred to as groupware applications. In such as applications, it is often natural for the activities of collaborators to be divided into distinct asynchronous and synchronous phases. During the asynchronous phase, collaborators work separately on individual tasks and during the synchronous phase they work together to recombine their individual efforts towards the common goal. During the asynchronous phase it is beneficial to allow the existence of potentially conflicting updates, in order to allow each collaborator freedom in carrying out his or her individual task, without
2 interruption or interference from the activities of others. The consequence of allowing the existence of conflicting updates is that the conflicts must then be resolved during the synchronous work phase. From this basic description of the problem domain, the results of the current research can be outlined. The first result is the development of a data storage manager1 (DSM) in which representations of change are primary values, easily accessible to applications and users. In this DSM, which is called Heraclitus[OO], these representations of change are called delta values. Through the use of these delta values, Heraclitus[OO] naturally supports multiple simultaneous database states. In addition to providing delta values and multiple states, Heraclitus[OO] also has the following features: • It is fully integrated into a standard object-oriented database model. This has the benefit of allowing the functionality of Heraclitus[OO] to be easily supported on existing object-oriented database systems (OODBs) without requiring adaptations or modifications to the underlying OODB. It also means that the benefits of Heraclitus[OO] are available to a wide range of applications, due to the robustness of the object-oriented data model. • It allows for customization of the form and semantics of delta values. This is accomplished in a regular manner, through the specification of delta forms associated with class specifications that make up the application schema. These delta forms also allow for application specific refinement of the definition of conflict between the changes that are represented as delta values. • The implementation of delta values in Heraclitus[OO] is such that their creation is essentially orthogonal to the transaction mechanisms of the native database system. This means that delta values can be used to 1
The term ‘data storage manager’ is used here to distinguish this work from traditional database systems, generally called ‘database management systems’. In this work, a DBMS will be assumed to support the traditional requirements (see, for example, [Dat86, Ull82, BDK92]) particularly in terms of concurrency control policies and mechanisms and a focus on the storage of a single consistent database state. A DSM will refer to a more general class of systems supporting persistent data storage, but not necessarily the notions of state and update consistency defined for DBMSs.
3 capture updates in very flexible manners, without being constrained by concurrency control policies. However, because the delta values are themselves “first-class” objects, they are subject to the concurrency control. As a result, updates can be executed free from the consistency constraints imposed on transactions, while maintaining the other benefits of concurrency control, such as durability and atomicity. • As mentioned above, removing the consistency constraints on delta values and the updates they capture requires support for conflict resolution. This is handled in Heraclitus[OO] through various merging operations that identify the sources of conflict. • Heraclitus[OO] supports two interfaces for controlling delta values and the states that are defined from them. The first is the H2O DBPL, in which delta value operations are provided as operators integrated into a standard object-oriented database application language. The second is a state-specification tree (SST) which is a persistent data structure that supports the specification of states and other delta values through manipulation of a persistent data structure. Because the SST is a better interface for cooperative work applications, it is included in the current work, while the H2O DBPL is not (see [DHDD95] for a details on the H2O DBPL). • The system design cleanly separates the application development tasks of defining the delta forms for the application schema and defining the policies and procedures governing the access to and manipulation of the delta values and states. • The operational semantics of the Heraclitus[OO] implementation are defined by a formal theory that is included in the current work. Throughout this work, Heraclitus[OO] will be used to refer to the theoretical framework, and the abbreviation H2O will refer to the actual implementation. When used with the SST interface the system will be referred to as H2O/SST. The second result of the current research is in the use of H2O to build groupware applications. The unique features of H2O make it an ideal
4 data storage manager for applications such as those described above. This portion of the research includes the following: • A general framework for building groupware applications using H2O as a keeper module [EW94], where a keeper is defined as the module responsible for holding the artifact that is the focus of the collaboration. This framework, called the Coral architecture, defines the application specific development required to construct systems that utilize H2O. • A rudimentary prototype authoring system, called Reef, has been implemented with sufficient functionality to demonstrate that H2O properly implements the functionality needed to support the target class of groupware applications. • A hypothetical authoring system called Eniwetok has been designed, to demonstrate the application development required to construct an actual groupware application based on the Coral architecture. The basic functionality supported by Eniwetok is adapted from other established authoring systems, since the primary objective is to show how H2O/SST can be used to implement functionality that is proven to be useful, rather than suggesting novel new functionality. Nevertheless, the development of Eniwetok clearly shows how the flexibility inherent in the Coral architecture could be used to develop and experiment with new functionalities. While the more concrete results are focused on authoring systems as a representative example application in the target domain, the results are applicable to a wider domain. This is due to two key features of the Coral architecture and H2O/SST. First is that delta forms can be customized to match the requirements of the application artifact types. Second is that no particular procedures and policies related to access and control of the delta values and states are enforced. Rather, the architecture enables the development of application specific policies.
5 1.2
Background The section presents background material and context for the current
research and its presentation in this document. This includes related work that has motivated or supported the current work. It also includes general discussions of concepts in databases and cooperative work that are useful in understanding the current research. Heraclitus: Delta Based Data Storage: As mentioned above, the primary result of the current research is the development and implementation of the H2O DSM and the development of the Heraclitus[OO] theoretical framework. The family of Heraclitus DSMs focuses on the support of delta values as primary entities for the management of change. A delta value is a persistent object that captures in some way the effective semantics of a database state change. These delta values are first-class objects accessible to applications and users through a programming interface. This accessibility distinguishes delta values from other representations of change, such as transaction logs, that are typically used for system operation in DBMSs but are not available for application or user access. The Heraclitus principle of supporting delta values as first-class application entities was initially developed in the context of relational databases resulting in Heraclitus[Alg,C] [HJ91, GHJ96]. Heraclitus[Alg,C] defined the notion of a delta value over a relational instance and the language constructs for manipulating delta values in the context of the relational algebra. Heraclitus[OO] adapts the theory initially developed for relational databases to the support of deltas over object-oriented database states, allowing the functionality added by support of delta values to be immediately available to the broad class of applications which are better supported by the object-oriented model (see [ZM90b, BDK92] for general discussions of object-oriented databases and their application). In addition, Heraclitus[OO] substantially expands on the theory of deltas in the following ways. First, it allows delta values to be customized to support specific application semantics, by allowing for the development of application specific delta forms over the more robust object-oriented
6 data model. Second, it supports a mechanism for delta creation and state materialization that is significantly more general than that supported by Heraclitus[Alg,C] and that is adaptable to any programming language supporting the usual object-oriented concepts (see [ZM90a] for an overview of these concepts in terms of database systems). Third, it includes facilities for the identification of conflict between delta values. The process by which updates to the database state are converted to delta values is called deltafication. This process is entirely unique to Heraclitus[OO] and as such is an invention within the scope of the current research. Through deltafication, updates executed against the database state by some program written in an object-oriented programming language (OOPL) are directly stored in the database as delta values. This has the effect of bypassing the object locking usually associated with the native DBMS concurrency control, without giving up the other benefits, such as durability, provided by the concurrency control. As will be shown, this new functionality is key to supporting the target class of applications in a general application independent manner. Because updates are now stored as delta values, rather than as effected changes to object values, Heraclitus[OO] must also provide a run-time mechanism for reconstructing the object values which constitute the database state in a transparent manner at run-time. This second mechanism is referred to as state materialization. The combination of deltafication and state materialization as transparent run-time mechanisms allows Heraclitus[OO] applications to be written without specific awareness of the fact that the DSM is using delta values rather than materialized states. This has the practical benefit that such applications can be easily ported from a usual object-oriented environment to the Heraclitus environment. As will be shown, H2O is implemented as an application layer in terms of a standard interface, which has the advantage that the functionality it provides can be easily ported to any existing object-oriented database system (OODB) without requiring changes to the implementation of the host OODB. New delta values can be created from existing delta values through operations called smash and merge. Smashing delta values creates a new delta
7
Planning
Editing
Consolidation Figure 1.1: Work Process for Collaborative Authoring value which preserves the sequential semantics of the original deltas values. Merge combines the delta values in a manner that recognizes potential conflict between the updates encoded in the original delta values. The basic form of merge simply fails if conflict is detected, a more sophisticated operation, called partial merge, goes further in that it identifies and separates the particular causes of conflict. Importantly, H2O allows these delta value operators to be customized as part of the application schema, allowing for application specific specification of the granularity of delta values and the semantics of conflict detection. This can be done such that the application specific delta values maintain the properties required for the overall system semantics to be preserved. While the current research is focused on using Heraclitus[OO] to support a specific class of applications in the domain of cooperative work, it is a more general DSM that can be used for a wide variety of application requirements. For example, H2O has been used to support hypothetical queries in telecommunication customer service databases [DDD+96]. Data Storage for Groupware Applications: The second main result of the current research is the development of a general framework by which H2O/SST can serve as a data storage module for a specific class of groupware applications. This class of applications falls within the research domain called Computer Supported Collaborative Work (CSCW) (see [Bae93, EGR91] for overviews of CSCW). CSCW deals with the design of computer systems and applications to support the activities of groups of cooperating and collaborating humans. Groupware is generally used to refer to any computer based system or application designed to support cooperative or collaborative work.
8 The target class of applications is characterized by a particular kind of work process where the work is divided into distinct asynchronous and synchronous phases. This work process, shown in Figure 1.1, was identified and used in the context of the Collaborative Authoring System (CES) [GS88, MR96]. In the initial phase of the process, planning, authors jointly determine what work is to be done and subtasks are assigned to individuals. In the editing phase, each author works independently and asynchronously to execute their assigned subtasks, including updates to the shared artifact. In the consolidation phase, the authors work to remove any outstanding conflict resulting from their independent efforts, resulting in one or more meaningful versions of the document. (In [MR96], this phase was called ‘arguing’, the change to the more positive term ‘consolidation’ is deliberate.) The planning phase is not addressed in the current work, and is assumed to be an off-line activity, an assumption that has precedence in many authoring system designs [GS88, MM93]. This general application environment has also been called “semi-(a)synchronous authoring” [MM93]. The idea of using Heraclitus[OO] to support groupware applications is inspired in a large part by the suggestions put forth in an analysis of cooperative problem solving (CPS) from a database perspective given by Chakravarthy and others [CKNT93]. In that paper, they argue for a approach by which database technology can be applied to the outstanding problems of CPS. It is their contention that substantial contributions could be made through analysis of CPS system requirements and the migration of appropriate functionality, as defined by that analysis, into a DBMS. While that paper deals with a much broader class of problems than is addressed by the current research, this work follows their proposal by delivering a reusable data storage manager module for a broad class of groupware applications. Of particular relevance to the current research is their identification of the obstacles that traditional database concurrency control places in the way of the application of DBMSs to groupware applications. An acknowledged shortcoming of the proposal outlined in the previously cited paper is an assumption of “positive cooperation” between the end
9 users of the developed systems. This results in the omission of the possibility of conflict between the goals and actions of users. However, for a wide class of cooperative work applications including authoring systems, it is acknowledged that conflict is not only unavoidable but is, in fact, desirable. Greenberg and Morwood [GM94] discuss the range of possible concurrency control requirements for cooperative work applications. One class of problems which they identify are those for which there may be no requirement for concurrency control. In their words, With no concurrency control, sites are free to exchange events, leading to possible inconsistencies, because things occur out of order, or to competition for a resource that only one person should acquire. While this may appear unacceptable, it could be a reasonable strategy for some groupware situations. In particular, inconsistencies may not matter, or people will be able to mediate and repair their own actions and conflicts. While the situations in which inconsistencies do not matter are fully supported by H2O/SST, of greater interest to the current research are situations in which it is necessary to address conflict. As Greenberg and Marwood put it, Finally, if people do notice conflicts and problems, they are often quite capable of repairing their negative effects and consider it to be part of the natural dialog. The computer’s role can thus be seen as one that provides enough feedback and affordances of shared objects to support people’s natural abilities. While the argument given by Greenberg and Marwood is somewhat anecdotal and broad, research in the field of authoring systems has given more specific specifications of requirements related to concurrency control and conflict analysis. It is for that reason that the current research has focused on providing data storage for authoring systems. While Greenberg and Marwood point out that it is often quite natural for collaborators to resolve conflicts, if they notice them, in their discussion on issues in collaborative writing [SGB+ 93], Sharples et.al. point out that it is possible and useful for the computer system to do more: One possibility is for the computer system to extract some of the conflicts
10 embedded in a document and present them to the co-authors for discussion and resolution. From the general discussion above, the following specific requirements on the data storage manager are apparent. First, authors should be allowed to work independently and without interference or interruption due to the concurrent actions of others. This requires that concurrent, potentially conflicting updates must be allowed. Second, once authors are done with their individual updates, the system should support conflict resolution by identifying the sources of conflict and presenting them to the users in a manner useful for resolution. The development of authoring systems2 to support such activity has been well explored in CSCW literature [Sha93, Rad96, HW92b]. However, there has been little research in the development of general data storage managers to support the common requirements of these applications. While the delta values provided by Heraclitus[OO] are sufficient for the support of the target applications, it is recognized that delta values are not the proper conceptual unit for users of such systems. This issue will be further developed in the context of the example application outlined in Section 1.3 and will be resolved in Chapter 4 by showing that it is possible and desirable to develop application specific mappings from the user level conceptualization of tasks to the implementation level delta values. The system architecture proposed for groupware applications based on H2O (given in Chapter 4) follows the modular structure proposed by Ellis and Wainer [EW94]. Their proposal defines four types of modules: keepers, synchronizers, communicators and agents. Keepers are the data management modules, responsible for maintaining the artifacts which are the focus of the collaboration. Synchronizers coordinate the group activities, where activities are work tasks or procedure steps. Communicators support human-to-human communication. Finally agents are software modules that perform specific non-global tasks, often involving specialized domain knowledge. 2
The term ‘authoring system’ generally refers to both single user and multiple user applications. In the current work it will be assumed that authoring systems support multiple users collaborating on a shared document.
11
Versions
Alternatives
Alternative Merging
Figure 1.2: Example Authoring System Interface. As the focus of the current research is on data management, the module of interest here is the keeper. Throughout this document the terms ‘keeper’ and ‘data storage manager’ will be used interchangeably. Two technical issues regarding the design of keepers are identified by Ellis and Wainer: providing concurrent access to the artifact for groups and providing support for persistent group memory. The Coral architecture for groupware applications based on H2O addresses both these issues as shown in Section 5.3. 1.3
An Example Application In this section an example groupware application will be described.
This example application will be used in Chapter 4 to show how H2O/SST can be used in the context of the Coral to build groupware applications. This example has been deliberately based on the functionality provided by existing authoring systems, as described in CSCW literature. Rather than explore novel authoring system functionality that might arise from the use of H2O, the current research is focused on demonstrating the applicability of H2O in the context of established system designs and functionality. Because the current research is focused on system design, rather than user studies or interface design, this approach ensures that the results are applicable and useful in the target domain. The authoring system that will be used to exemplify the target class of applications is called Mj¨olner [MM93]. The Mj¨olner user interface is centered around a version evolution graph, the main constructs of which are shown in Figure 1.2. Mj¨olner supports the creation of document versions, the simultaneous creation of alternative versions and the merging of alternative versions.
12 The merging phase implies the need for the resolution of conflict between the alternatives. The graphical structure of the interface reflects the document history maintained by the system’s underlying storage manager. Mj¨olner also supports “active diffs” which identify differences between alternatives to allow authors awareness of the concurrent activities of other authors. The Mj¨olner interface supports the kind of authoring activity identified in Section 1.1 and shown in Figure 1.1. The creation of alternative versions corresponds to the asynchronous editing phase in which authors work independently on their individual tasks. The merging of alternatives corresponds to the synchronous consolidation phase in which authors work together to merge their individual efforts towards the common goal. While it is assumed that the authors are all working towards a common goal, it is possible that the portions of the document each needs to access and modify overlap. This may be due either to design or accident. Furthermore, imperfect task specification during the planning phase may result in unintended conflicts and inconsistencies between the individual authors updates. To understand the reason that traditional DBMSs are not well suited to support the data storage needs of such systems, suppose that the document is stored in such a DBMS. The usual concurrency control policies prevent conflicting updates by enclosing those updates in transactions, and ensure that each transaction cannot modify overlapping parts of the database, which in the case of the example is the shared document. This could have many undesirable effects on the work process of the authors. For example, it could lead to delays in the work of one author, if that author’s assigned task involves editing a portion of the document which is currently within the scope of an update underway by another author. A more severe consequence is the possibility that some unit of work will be lost if a transaction must be aborted in order to resolve a conflict between two updates. H2O overcomes these problems by converting updates directly into delta values and thereby bypassing the conflict prevention policies of concurrency control of the native database system. The most fundamental thing that must be provided by any data storage manager for an operating system is some persistent representation of the
13 document and any versions of that document. This means that the data storage manager must have some model for the document. Because Heraclitus[OO] is based on the object-oriented data model and fully supports that model, this is simply a matter of defining a document model in terms of an object schema. It will be shown that this is easily supported by constructing a schema that supports a hierarchical document model common to many existing authoring systems such as CES [GS88] and LINCKS [PL92]. The additional requirement imposed by Heraclitus[OO] is the specification of delta forms for the object classes in the application schema, such that delta values over the instances of that schema operate properly with the Heraclitus[OO] system. The development required for application specific delta forms is defined in Section 2.5 and illustrated in the context of authoring systems in Section 4.4. During the asynchronous editing phase, each author should see a single consistent version of the document. The document version selected for update by each author need not be consistent with the document versions seen by other authors, but any updates made by an author must be persistent and reflected in that author’s view of the document in a transparent manner. Ideally, the authoring interface used during the editing phase should support the functionality usually supported by common authoring and editing interfaces. From the perspective of the current research, which focuses on the data storage manager, it is sufficient to demonstrate that Heraclitus[OO] imposes no restrictions that would prevent the implementation of such an interface. As will be shown, Heraclitus[OO] operates in a manner such that application programs written against the database are unaware of the fact that delta values are being used to implement the persistent store. Thus any application that can be written in a common object-oriented language can be easily supported by Heraclitus[OO]. Furthermore, it will be shown that the delta value control interface can be easily used to implement an undo facility with application specific granularity. From the point of view of an individual author engaged in editing a selected version, there should be no difference in that activity whether the version is simply a new version or an alternative version. In other words, an
14 author engaged in editing should be able to focus on that task, without concern for the activities of other users. Thus the interface described in the previous paragraph should also suffice for authors editing alternatives. The ability of H2O to create persistent non-locking updates as delta values and to materialize states in application run-time is the key to supporting this. Any arbitrary editing session is reflected as an update to the document stored in H2O, and the resulting delta value(s) are sufficient to recreate the document version resulting from that editing session. Because the process by which updates are converted to delta values is independent for each author, there is no interference between the editing activities of each author. Furthermore, since the delta values are persistent and subject to the concurrency control of the underlying DBMS, there is no possibility of lost work due to system failures or other abnormal situations. The current research focuses on the most extreme case of the asynchronous editing mode, in which it is assumed that authors would never want to be interrupted by the concurrent activities of other authors. However there are also many cases in which some form of group awareness is desireable. The possibility of supporting group awareness during editing in a manner consistent with the Coral system architecture are discussed in the context of the example application in Section 4.3. Once authors have finished their individual editing activities, it is necessary for them to work together to consolidate their efforts to create a single consistent version of the document. This is represented in the Mj¨olner interface as alternative merging. This consolidation may require decisions on the part of the authors to resolve any conflicts or inconsistencies between their individual efforts. In extreme cases, this conflict resolution may be a lengthy task (either due to the complexity of the conflict or due to the decision making process chosen by the authors) so it is important that partially resolved merges be persistent so that they can be resumed as needed. It will be shown that Heraclitus[OO] can support merging and consolidation in manner that it can be divided into persistent segments of arbitrary duration. The final requirement is the support of the identification of the causes of conflict and the presentation of
15
Asynchronous User Interface
Synchronous User Interface
Artifact Editing Interface
User Level Tasks
Artifact Access
Delta Value Access
Deltas on Artifact H2O Figure 1.3: Overview of System Architecture. the conflict to the authors so that they may determine the required resolution. It will be shown that Heraclitus[OO] can isolate and identify the sources of conflict at the level of individual objects. The format for presentation of conflict is assumed to be application specific, for the current research it is sufficient to show that the information provided by Heraclitus[OO] is sufficient to support the presentation formats described in existing literature, such as those defined for the Prep editor [NCK+ 96] or the Mj¨olner editor [MM93]. This example application suggests a system architecture such as the one given in Figure 1.3 . At the bottom is H2O, with the artifact stored as a collection of delta values from which artifact versions can be reconstructed. To the upper left is the editing interface, in which authors should see a single consistent version of the artifact and through which they should be able to edit their selected version of the artifact as if it were a realized state. Each author will edit a different (but possibly equivalent) version of the artifact. To the upper right is the interface through which users can specify meaningful artifact versions and resolve conflicts between versions by manipulations of the SST.
16 This basic architecture will be further developed in Chapter 4, to demonstrate how Heraclitus[OO] can be used as a general keeper module. 1.4
Organization of the Dissertation The Heraclitus[OO] theory which defines delta values and how they
are used to construct multiple states in an object-oriented database is presented in Chapter 2. That chapter also contains a presentation of relevant features of the H2O data storage manager that implements the Heraclitus[OO] theory. Chapter 3 presents the State Specification Tree and explains how it is used to organize and manipulate the delta values stored in H2O and the states constructed from those delta values. Chapter 4 develops the Coral architecture which specifies how H2O/SST can be used in the development of groupware applications. That chapter continues with a description of the Reef prototype authoring system that validates the H2O/SST implementation in the context of the Coral architecture. It also expands on the use of Coral to design groupware applications through examples that illustrate the operation of the hypothetical Eniwetok authoring system. The Conclusion enumerates the contributions of this research, summarizes related research and suggests potential future research.
CHAPTER 2 H2O: A DELTA-BASED MULTISTATE DATA STORE This section presents the Heraclitus[OO] data storage manager which provides a multistate database service through the use of delta values to store state changes. A full theoretical model of Heraclitus[OO] is given, as well as an overview of the implemented system. To avoid ambiguity, the theoretical framework will be referred to as Heraclitus[OO], and the implemented system will be referred to by the abbreviation H2O. The theoretical model presented in Sections 2.2 through 2.4 expands significantly on the theory previously published [DHR96, DHDD95] by the inclusion of deltafication and state materialization. Deltafication is the mechanism by which updates encoded in the native programming language are converted into delta values, and state materialization is the mechanism by which database states are reconstructed from the stored delta values. Together these two mechanisms allow applications operating on a Heraclitus[OO] DSM to function properly, as if the underlying storage were implemented as a traditional state-based DBMS. Additionally, the inclusion of these mechanisms in the theoretical model allows for correctness proofs on the stored delta values. The remaining theory, concerning the semantics of individual delta values and the operations for combining them, is also modified from the previously published results, in order to correctly model the operational semantics of the implemented system. In addition to the theoretical model, the chapter opens with an informal example that illustrates the basic concepts of a delta-based DSM. This informal example provides a sufficient conceptual background for understanding the theory presented in the following sections. Section 2.5 discusses the implementation of the H2O data storage manager. This implementation is based directly on the theory presented in
18 the preceding sections. The presentation of H2O includes discussions of three technical issues related to the implementation. 2.1
An Example Delta Form This section introduces the Heraclitus[OO] concept of delta values
through a simple example. The example will use an integer class1 which supports a small number of operations on integer values. As the example is built up, it will show what delta values are and illustrate the operations which can be performed on those delta values. The presentation in this section will be informal, as the emphasis is on illustrating the basic concepts. The theory behind the example will be formalized in the following sections. Heraclitus[OO] is defined and implemented as an application layer over the ODMG-93 [Cat93] standard interface for object-oriented database. The object model defined by ODMG and used by Heraclitus[OO] is essentially the same as the object model of C++ [Str91]. Familiarity with C++ is sufficient for understanding the presentation in this document. In this object model, the fundamental notions are classes and objects. Classes define both the type and implementation of the objects which are instances of those classes. For the purposes of the current work it is sufficient to address two aspects of classes and objects: attributes and methods. Attributes2 are the data values stored in objects which maintain the state of the object. Methods are the functions which act on instances and potentially change the value of the attributes. Heraclitus[OO] enforces the rule that all attributes of an object are only accessible from the methods defined for the class of that object. (In terms of C++, this means that all attributes are ‘private’.) Consider the specification for class Integer given in Figure 2.1. The class specification states that all objects that are instances of class Integer 1
This example is taken from a demonstration telecommunications application called Smart-Talking [DDD+ 96], which used delta values to generate hypothetical future states of a customer service database. 2 In C++, these are called ‘data members’. The term ‘attributes’ is more commonly used database literature, and will be used throughout this dissertation.
19 class Integer { int x; int Set(int new_value); int Increment(int incr); int Get(); }; Figure 2.1: An Integer Class Specification in C++. int Integer::Set(int new_value) { x = new_value; return x; } int Integer::Increment(int incr) { x = x + incr; return x; } int Integer::Get() { return x; } Figure 2.2: An Integer Class Implementation in C++. have a single attribute, named x, which has type int, meaning that it can store one integer value. All objects of class Integer also have three methods which operate on the instance: • int Set(int new value) changes the value of x to new value and returns the new value of x, • int Increment(int incr) adds incr to x and returns the new value of x, and • int Get() returns the current value of x. The normal implementation of a C++ class would require supplying code to implement the methods defined in the class specification. An example of this is given in Figure 2.2. Both the Set and Increment methods make assignments to the attribute x, changing its value. This not only has the effect of storing the new value of the attribute, but also of destroying the old value. 1: 2: 3: 4:
i1->Set(6); i1->Increment(-2); i1->Set(12); i1->Increment(10);
Figure 2.3: A Program Fragment Operating on an Integer Object.
20 Consider the program fragment given in Figure 2.3. Assume the the variable i1 contains the object identifier3 (oid) of an object of class integer and that the value of that oid is i1 . After the execution of line 1, the value of the attribute x of the object with oid i1 is 6. This will represented as (i1 :< x : 6 >). The database state is composed of the values of the objects in the database; if the object with oid i1 is the only object in the database, then the database state is {(i1 :< x : 6 >)}. During the execution of the program fragment, the database state will have the following values: after line 1: {(i1 :< x : 6 >)}, after line 2: {(i1 :< x : 4 >)}, after line 3: {(i1 :< x : 12 >)}, after line 4: {(i1 :< x : 22 >)}. When operating with delta values, instead of changing an object’s state by changing the values of the object’s attributes, updates are converted directly into delta values through the process of deltafication. One possible implementation for this process is to replace the method implementations so that they compute a delta value that captures the state change that would have occurred had the original method been executed. In addition to computing the delta value, the new method implementation must also properly compute any side-effects, such as the return value and parameters. Figure 2.4 shows how replacement deltafication methods might be implemented for the current example. For the moment the means by which the delta values are stored and accessed will be ignored and abstracted though the system functions StoreDelta and EvalObj. The exact functionality of these system functions will be discussed in Section 2.4. The first system method, StoreDelta, tells the DSM to store the newly computed delta value, and the second, EvalObj, asks the DSM to reconstruct the current value of the object using the stored delta values. The replacement implementation for the method Set now constructs the delta value < set new value >, where new value is the parameter passed 3
For the reader who is unfamiliar with object identifiers, it is sufficient at this point to assume that it is a pointer, as in C++.
21 int Integer::Set(int new_value) { StoreDelta(); return new_value; } int Integer::Increment(int incr) { StoreDelta(); return EvalObj(); } int Integer::Get() { return EvalObj(); } Figure 2.4: Replacement Methods for Integer Class Deltafication. [Set(new value)]int =< set new value > [Increment(incr)]int =< add incr > [Get()]int = (none) Figure 2.5: Deltafication Operation for Example Integer Delta Form. to the method. The constructed delta value is passed to the DSM for storage and new value is returned, preserving the correct semantics of the method interface. Similarly, the replacement implementation for the method Increment constructs the delta value < add incr > and passes it to the DSM for storage. Since Increment must return the value that the object would have had had incr been added to the object’s original value, the system function EvalObj must be called to ask the DSM to reconstruct the object’s current value from the new delta value and previously existing delta values. Finally, since the last method Get is a read-only method (it does not change the object’s state), it does not construct a new delta value, rather it simply calls EvalObj to compute the object’s current value from previously stored delta values. The deltafication process represented by the replacement methods as shown in Figure 2.4 is shown again in Figure 2.5 using the notation that will be used throughout the current work. Note that the details related to the storage and evaluation of the delta values are omitted in the formal notation.
22 By inspecting the replacement methods, it can be seen that the set of possible delta values for the class Integer is {< set v >, < add v >}. In general, there are many possibilities for the set of legal delta values for a particular class. As will be shown in Section 2.2, the definition of the set of legal delta values for any class is part of the specification of a delta form for the class. The specification of delta forms, as part of the application schema, allows for the customization of the semantics of delta values for particular applications. Returning to the sample program fragment given in Figure 2.3 the following delta values are stored at each step of the program: at line 1: < set 6 >, at line 2: < add − 2 >, at line 3: < set 12 >, at line 4: < add 10 >. If it is assumed that H2O is storing each of these delta values in a time ordered sequence, then it can be seen that the delta values are recording the history of changes to the object, rather than the new states of the object. In fact, what actually occurs is that as the delta values are stored, H2O compresses the change history, and stores only the accumulated changes. This compression takes place through an operation called smash. The particular sequences of changes that are compressed through the smash operation is controlled by indicating the scope of each deltafication. One means of indicating this scope is through the deltafication operator of the H2O DBPL [DHDD95]. The deltafication operator [ ] encloses expressions that are to be deltafied as a unit. If the entire program fragment given in Figure 2.3 were enclosed by [ ], then the resulting delta value would be the result of the sequential smash of individual delta values constructed by the individual methods. Because the particular set of delta forms may be different for each class of objects, the smash operation must also be specified for each class. Figure 2.6 specifies the smash operation for the example Integer class in tabular form. If, as suggested above, the entire code fragment given in Figure 2.3 were enclosed in a single deltafication, then the actual delta value stored by H2O to represent
23 ∆2 ∆1
smashint (∆1 , ∆2 ) < add v2 > < set v2 > < add v1 > < add v1 + v2 > < set v2 > < set v2 > < set v1 + v2 > < set v2 >
Figure 2.6: Smash Operation for Example Integer Delta Form. ∆2 mergeint (∆1 , ∆2 ) < add v2 > < set v2 > < add v1 > < add v1 + v2 > fail ∆1
< set v2 >
fail
v1 = v2 : < set v2 > v1 = v2 : fail
Figure 2.7: Merge Operation for Example Integer Delta Form. that deltafication would have the following history: at line 1: < set 6 >, at line 2: < set 4 >= smash(< set 6 >, < add − 2 >), at line 3: < set 12 >= smash(< set 4 >, < set 12 >), at line 4: < set 22 >= smash(< set 12 >, < add 10 >). If, on the other hand, each line of the program fragment were individually enclosed in a separate deltafication, the H2O would store the four individual delta values as indicated above. Thus by controlling the boundaries of the deltafication, the application or user has complete control of the granularity of the change storage, with the most refined granularity being limited to the level of individual method calls. The smash operator is also available to applications for combining any two previously stored delta values to create a new delta value. Because smash is defined to preserve the sequential semantics of the original delta values, the new delta value defines the same change as the original delta values, in the particular order that they are smashed. Smash is not necessarily commutative. For example, smash(< add 3 >, < set 9 >) =< set 9 > which is not the same as smash(< set 9 >, < add 3 >) =< set 12 >. Intuitively, this is because setting a value to a particular new value overides and previous changes to the value, while modifying a value by addition does not.
24
object value v
∆ applyint (∆, v) < add v2 > < set v2 > v1 v1 + v2 v2
Figure 2.8: Apply Operation for Example Integer Delta Form. Noncommutativity, as in the above example, is one kind of conflict between delta values that must be addressed. Other kinds of conflict, specific to the semantics of particular applications and data types can be identified and dealt with with (see, for example, the examples in Section 4.4 and [DHDD95]). To deal with conflict, a second delta value combinator, called merge, is defined. In general, the result of merge is the same as smash if the smash operation commutes and no other conflict is detected between the delta values. If there is conflict, the result of a merge is the special delta value fail. The merge operator for the example Integer class is given in Figure 2.7. If the two delta values given in the smash example above are merged, the result in both cases will be the special delta value fail. Up to this point, the example has looked at operations that create and combine delta values. The final operation that is required effects the change called for by a delta value to an existing value. This operation is called apply and the definition of apply for the example Integer class is given in Figure 2.8. For example, apply(< add 3 >, 2) = 5 and apply(< set − 7 >, 12) = −7. The examples in this section have illustrated the basic concepts involved with delta values in the context of a single class of objects. Because Heraclitus[OO] uses delta values over entire database states, these notions must be extended to include all objects that make up the state. This will be formalized in Section 2.3, by viewing the database state as a single object that is essentially a set of objects. The next section formalizes the notion of a delta form and defines requirements on them that ensure the proper system semantics of Heraclitus[OO].
25 2.2
Delta Forms and Requirements This section presents a formal specification of delta forms, which en-
capsulate the information required to allow instances of a class to operate properly within the H2O system. The theory in this section is adapted from the theory previously published [DHDD95, DHR96] to better fit the semantics of the H2O system and to include the deltafication process. The delta form for each class can be programmed as part of the application schema, however, H2O provides default delta forms for any class as well as programmed delta forms for system supplied classes. As was shown informally in the previous chapter, there are five things which must be specified for any class4 : the domain of allowed delta values for the type; the apply, smash and merge operations that operate on values and delta values of the type; and the deltafication process which converts expressions that operate on values of the type into delta values. Delta Form Specification: Because delta values have to operate within the native database system and associated programming language, delta forms are defined with reference to that system and language. For any type, τ , there is the language, Lτ , that operates on values or instances of that type. More specifically, expressions in Lτ transform values of type τ . The evaluation of expressions by the native system is defined by some valuation function [ ] : Lτ × τ → τ . For any expression ∈ Lτ , [ ]] : τ → τ is a function from values to values. A delta form for a type τ is a five tuple < Dτ , [ ]τ , applyτ , smashτ , mergeτ >, where • Dτ is the domain of delta values for type τ . The value fail is in this domain for all types. • [ ]τ : Lτ × τ → Dτ is a mechanism for mapping expressions in the language Lτ to delta values. • applyτ : Dτ × τ → (τ ∪ error), such that for any value v of type τ , applyτ (fail, v) = error. Application is the mechanism by which a delta 4
In the spirit of C++, no distinction will be made between classes and the types they define.
26 value in the domain Dτ is used to effect a change to some value of type τ. • smashτ : Dτ ×Dτ → Dτ such that smashτ (fail, ∆) = smashτ (∆, fail) = fail for each ∆ ∈ Dτ . Smashing two delta values creates a new delta value which preserves the sequential semantics of the initial delta values. • mergeτ : Dτ ×Dτ → Dτ such that mergeτ (fail, ∆) = mergeτ (∆, fail) = fail for each ∆ ∈ Dτ . Merging two delta values creates a new delta value which has the same semantics as the two operand delta values, assuming they do not conflict. If two delta values do conflict, the value of the merge is fail. Requirements on Delta Forms: There are three fundamental requirements placed on delta forms by the Heraclitus paradigm (see also [GHJ96, DHR96]). These requirements are satisfied by H2O at the system level if they are satisfied at the object level by all system defined and application supplied classes. The first requirement is that a delta value computed by deltafication correctly captures the semantics of the expressions it was computed from, in the context of the particular value on which the deltafication was executed. This requirement defines the relationship between deltafication, application and the expression evaluation semantics of the native system. Let [ ] : Lτ × τ → τ denote the system’s normal mechanism for evaluating expressions, transforming values of type τ . As defined above, the deltafication operation, [ ]τ : Lτ × τ → Dτ , computes a delta value from an expression and a value. Existing delta values can be used to effect value changes through the application operation, applyτ : Dτ ×τ → τ . Together deltafication and application define the equivalent of the system’s normal expression evaluation function. Composing application and deltafication yields applyτ · [ ]τ : Lτ × τ × τ → τ , which is not identical in type to the system valuation functions, as it has an extra parameter of type τ . Intuitively, this is due to the fact that an intermediate step in the evaluation of expressions has been introduced. This step is the computation and storage of delta values. Because the delta values are
27 persistent, it is possible that the context in which the application is computed may be different from the context in which the deltafication was computed. The first requirement states that the composition of application and deltafication must be equivalent to the native language’s expression evaluation, if all are computed in the same context. Requirement 2.1:
For any type τ , any value v ∈ τ and any ex-
pression ∈ Lτ , applyτ ([]τ (v), v) = [ ]] (v). This requirement is weaker than requiring that for all values v, v ∈ τ applyτ ([]τ (v), v ) = [ ]] (v ) which would require that the delta values preserve the meaning of expressions in all contexts, not only in the context in which they are computed. To support this stronger requirement, it would be necessary for the domain of delta values, Dτ , to be computationally equivalent to the language of expressions, Lτ . For computationally complete languages such as C++, this is impractical, since proving program equivalence is intractable. For the same reason, testing for conflict between delta values would be impossible. The next requirement defines the relationship between application and smash. It states that smash must preserve the semantics of sequential application. Requirement 2.2:
For any type τ , value v ∈ τ , and delta values
∆1 , ∆2 ∈ Dτ , applyτ (smashτ (∆1 , ∆2 ), v) = applyτ (∆2 , applyτ (∆1 , v)). The final requirement restricts the semantics of merge and gives a minimum definition for conflict between delta values. It states that in cases of non-conflict, merge must be equivalent to smash (preserving sequential semantics), and that it must be commutative. In cases in which these requirements cannot be satisfied, the result of merge is by definition the delta value fail.
28 Dτ = {< v >}, v ∈ τ [fw ()]τ =< v >, where fw () is any write method [fr ()]τ = (none), where fr () is any read-only method
object value v
∆1
∆1
∆ apply(∆, v) < v2 > v1 v2
∆2 smash(∆1 , ∆2 ) < v2 > < v1 > < v2 >
∆2 merge(∆1 , ∆2 ) < v2 > < v1 > v1 = v2 :< v2 > v1 = v2 : fail
Figure 2.9: Delta Form for Object Granularity Deltas. Requirement 2.3: For any type τ and delta values ∆1 , ∆2 ∈ Dτ
smashτ (∆1 , ∆2 ) if smashτ (∆1 , ∆2 ) = smashτ (∆2 , ∆1 ) mergeτ (∆1 , ∆2 ) = . fail otherwise In general, a variety of application specific semantic issues can be considered when defining conflict between delta values and these application specific semantics can be encoded in the merge functions for the application classes. Checking for merges that result in fail is the primary means by which H2O test for conflict. Example Object Level Delta Forms: The formal definition of delta forms defines them in terms of classes or data types. As will be shown in Section 2.3 the delta form definition also applies to the state of an H2O database, and will be defined in terms of the delta forms for the classes of objects that may appear in the database. To further illustrate delta forms, two system defined delta forms will be described here. An object-granularity delta form is the default for all application defined classes for which no application specific delta form is defined. The
29 object-granularity delta form for any type τ is given in Figure 2.9. An objectgranularity delta value simply stores a value of the same type as the object, hence for any type τ , the domain of object-granularity delta forms is Dτ = {< v >}, where v ∈ τ . The deltafication process is defined generically over any methods defined for class τ . For methods that would normally modify the value of the object (write methods), the delta value that is constructed and stored is < v >, where v is the value of the object at the termination of the method call. For methods that do not modify the value of the object (read-only methods), no delta value is constructed or stored. Application of an object-granularity delta value changes the value of the object to which it is applied to the value stored in the delta value. Smashing two object-granularity delta values results in a new delta value with the same value as the second operand delta value. Merging two object-granularity delta values results in fail unless the values stored in both operand delta values are the same, in which case the resulting delta value has the same value as the operands. All delta forms must satisfy the three requirements stated above. Satisfaction of these requirements by the object-granularity delta form will now be shown. Proposition 2.4:
The object-granularity delta form satisfies Re-
quirements 2.1, 2.2 and 2.3. Proof:
Requirement 2.1: Let contain exactly one call to some
write method on an object of type τ with value v, then [ ]] (v) = v , where v is the value of the object at the termination of the method contained in . By the definition of deltafication, [](v) =< v >. Then apply([](v), v) = apply(< v >, v) = v . Requirement 2.2: Let ∆1 =< v1 > and ∆2 =< v2 >. Then apply(smash(∆1 , ∆2 ), v) = apply(∆2 , v) = v2 and apply(∆2 , apply(∆1 , v)) = apply(∆2 , v1 ) = v2 , satisfying the smash requirement.
30 ¯ Dset = PS(∆), ¯ ∈ {< ins v >, < del v >}, where ∆ v is an object value [insert(v)]set = {< ins v >} [remove(v)]set = {< del v >} occurrence of v in ∆ presence of v in S
applyset (∆, S) absent v ∈ S v ∈ S v∈S v∈S
ins(v) del(v) v ∈ S v ∈ S v ∈ S v ∈ S
occurrence of v in ∆2 occurrence of v in ∆1
smashset (∆1 , ∆2 ) absent ins(v) del(v)
absent absent ins(v) del(v)
ins(v) ins(v) ins(v) ins(v)
del(v) del(v) del(v) del(v)
occurrence of v in ∆2 occurrence of v in ∆1
mergeset (∆1 , ∆2 ) absent ins(v) del(v)
absent absent ins(v) del(v)
ins(v) ins(v) ins(v) f ail
del(v) del(v) f ail del(v)
Figure 2.10: A Delta Form for Sets. Requirement 2.3: Easily verified by inspection.
2
The satisfaction of Requirement 2.1 can be extended to expressions containing more than one method application on an object. This is because sequences of method applications within the same deltafication scope are handled by smashing together the individual delta values resulting from the deltafication of each method. Since Requirement 2.2 ensures that smash preserves sequential semantics, Requirement 2.1 is satisfied for sequences of method applications if it is satisfied for individual method applications. A delta form for sets is given in Figure 2.10. Sets support two operations, insert and remove, which add and delete elements from the set. If an element v is exists in a set S, the S.insert(v) does not change the set S, similarly, S.remove(v) does not change the set S if v is not present in that set.
31 The domain of delta values for sets is sets of elements of the form < ins v > or < del v >. The value v is called the target of the delta value element. A proper set delta value can have at most one element that targets any particular value v. The special delta value fail is a proper set delta by definition. The deltafication of the methods insert and remove results in delta values that are sets containing either a single < ins v > or < del v > respectively. Application of set delta values is defined in terms of the individual elements of the delta value and the set. If a < ins v > element appears in the delta value, then v appears in the result of the apply. If a < del v > element appears in the delta value, then v does not appear in the result of the apply. If an element v appears (does not appear) in the original set and there is no element in the delta value that targets v, then v does (does not) appear in the set resulting from the apply operation. The smash and merge operations on set delta values are also defined in terms of the elements of the operand delta values. If the element < ins v > or < del v > appears in the second operand, then that same element appears in the result of the smash. If an element targeting some value v appears in the first operand delta value and no element targeting that same value appears in the second operand, than the element from the first operand appears in the result. Finally, there can be no element targeting some value v in the result if there is no element targeting that value in either operand delta value. Merge is defined the same as smash, except for one case which results in failure due to conflict. That case is when an element < ins v > appears in one operand delta value and < del v > appears in the other operand, both targeting the same value v. In this case the result of the merge is fail. Note that the set of proper set delta values is closed under both smash and merge. Proposition 2.5:
The set delta form satisfies Requirements 2.1,
2.2 and 2.3. Proof:
Requirement 2.1: The execution of the insert method,
[ insert(v)]] , results in a set which contains the element v. Deltafication of the insert method, [insert(v)]set , results in a delta value containing {< ins v >
32 ∆1 ∆2 ∆3 = smash(∆1 , ∆2 ) s1 = apply(∆3 , s0 ) < ins v > < ins v > < ins v > v ∈ s1 < ins v > < del v > < del v > v∈ s1 < del v > < ins v > < ins v > v ∈ s1 < del v > < del v > < del v > v ∈ s1 < ins v > none < ins v > v ∈ s1 none < ins v > < ins v > v ∈ s1 < del v > none < del v > v ∈ s1 none < del v > < del v > v ∈ s1 ∆1 ∆2 s2 = apply(∆1 , s0 ) s3 = apply(∆2 , s2 ) < ins v > < ins v > v ∈ s2 v ∈ s3 < ins v > < del v > v ∈ s2 v∈ s3 < del v > < ins v > v ∈ s2 v ∈ s3 < del v > < del v > v ∈ s2 v ∈ s3 < ins v > none v ∈ s2 v ∈ s3 none < ins v > unknown v ∈ s3 < del v > none v ∈ s2 v ∈ s3 none < del v > unknown v ∈ s3 Figure 2.11: Analysis of Smash and Apply Operations on Set Delta Form. } and application of a delta value containing that element results in a set containing the value v. The execution of the remove method, [ remove(v)]] , results in a set that does not contains the element v. The deltafication of the remove method, [remove(v)]set , results in a delta value containing {< del v >} and application of a delta value containing that element results in a set that does not contain the value v. Requirement 2.2: The possible cases involving delta values elements for any value v are enumerated in Figure 2.11. For any set value s0 , the result of apply(smash(∆1 , ∆2 ), s0 ) is shown as set value s1 and the result of apply(∆2 , apply(∆1 , s0 )) is shown as set value s3 . Any particular value v will be present in set value s1 if and only if it is present in set value s3 . Requirement 2.3: Easily verified by inspection.
2
This section has formalized the notion of a delta form, through which arbitrary data types can be made compatible with the H2O system. The next
33 section shows how delta values on individual objects can be built up to define delta values on the entire database state. 2.3
The H2O State Delta Form This section adapts the general theory of delta forms to define the
particular delta form that implements delta values over the database state in a Heraclitus[OO] database. This delta form is defined by viewing the database state as an object that maps object identifiers to object values. Changes to the state change this mapping, hence the state delta form is defined in terms of that mapping. The state delta form assumes the existence of delta forms for all application defined and system supplied classes. Examples of such object level delta forms were given in Sections 2.1 and 2.2. The state delta form defined in this section formalizes the semantics of the implemented H2O DSM. Object-Oriented Database State: In an object-oriented database, the state is composed of the collected values of all objects currently existing in the database. Each object is uniquely identified by an object identifier5 (oid), which is a system generated value. The database state can thus be viewed as a map or dictionary associating oids with object values6 , that is, a state is a partial function with type state : oid → value. Every object is an instance of some class, which defines the object’s type. This class is immutable, and associated with the object’s oid at the time of oid allocation. To determine the class of an object, the partial function class : oid → type is defined. This model requires that the value of an object maintained by the state function must be of the type maintained by the class function, or, more formally, if τ = class(o) and v = state(o), then v ∈ τ . This 5
In fact, the set of consumed object identifiers must also be stored as part of the database state, in order to ensure uniqueness when new object identifiers are allocated. For the purposes of the current work, it can be assumed that object identifier generation is handled by the native DBMS, and does not impact the theory presented here. 6 In some cases, particularly when the semantics of the data model is of interest (see, for example, [BDK92]), it is more appropriate to view the database state as a tuple of classes, where the value of each class is a set maintaining the oids of the instances of that class. However, when implementing general object services, such as persistence or delta-based multistate, it is more appropriate to view all objects as generic values associated with object identifiers, and to access the class as a function of the object identifier.
34 class State { new(oid o, value v); delete(oid o) modify(oid o, value v); value value_of(oid o); }; Figure 2.12: Class Specification for an Object-Oriented Database State. required invariant on the state is usually maintained by the DBMS and the allowed operations on the objects. It will be proven that it is also properly maintained by Heracltus[OO] at the end of this section. There are three ways in which the database state can be changed: • Object construction: New objects are created and added to the database through object construction. This requires the specification of the class of the new object and results in the generation of a new oid, and an initial value for the object. The new oid and value are used to update the state and class functions. • Object deletion: An object is removed from the database through object deletion. Object deletion requires the specification of the oid of the object to be deleted, and results in the removal of that oid from the state function. • Object modification: The values of existing objects are modified through the application of the methods defined for the class of the object. The state function is updated with the new value of the object. In order to define a delta form over the database state, it is convenient to view the state as an object with a defined class and methods. The specification of this class, which will be called class State, is given in Figure 2.12. The first three methods defined on class State correspond directly to the three ways in which a state can be modified, as listed above. The fourth method value of returns the value currently associated with an object identifier. As was stated earlier, it is required that the value associated with an object identifier be of the type associated with that oid. The specification
35 ¯ DState = PS(∆), ¯ ∈ {< ins o v >, < del o >, < mod o ∆ >}, where ∆ o is an oid, v is an object value, and ∆ is an object level delta value Figure 2.13: Delta Values for Database State. of the class State as given here is insufficient to maintain that requirement. Rather, this requirement must be enforced at the level of the mapping from the native programming language to the state. To see that this can be easily accomplished, consider the operations in a typical OOPL such as C++. Constructors, which create new objects, would pass the allocated oid and initial value of the object to the method new. This is the only time a new oid is introduced into the state and is also the point at which the class method is updated, ensuring that the requirement is not violated. Destructors, which destroy objects, would pass the oid of the object to be destroyed to the method delete. As this does not associate a new value with the oid, it can not violate the requirement. The application of any class defined method would pass the oid and modified value of the object to which the method was applied to the method modify. Assuming that the new value of the modified object has the correct type (as enforced by the type system of the programming language), object modification will not violate the typing requirement on the state. With this view of the database state as an instance of class State, a delta form for the state can be developed in the same manner as was done for general object classes in the preceding subsections. The rest of this section will define that state delta form. H2O State Delta Values: An H2O state delta value is a set of atomic deltas, each of which is specific to one object. There are three types of atomic deltas, corresponding to the three types of state changes identified above. The domain of delta values for the database state is given in Figure 2.13. Insert atoms, < ins o v >, correspond to object construction, and store the oid of an object and an initial value for that object. Delete atoms, < del o >, correspond to object destruction
36 [new(o, v)]State = {< ins o v >}, v ∈ class(o) [delete(o)]State = {< del o >} [modify(o, v)]State = {< mod o ∆ >}, ∆ ∈ Dτ , τ = class(o) [value of()]State = (none) Figure 2.14: Deltafication Process for Database State. and store the oid of an object. Modify atoms, < mod o ∆ >, correspond to object modification and store the oid of an object and a delta value that can be applied to values of the type associated with that oid. The object whose oid is stored in an atomic delta will be referred to as the target object of that atomic delta. The following two definitions restrict the domain of state delta values. These restrictions ensure the correct operation of the state when implemented through the state delta form. It will be proven at the end of the section that the state delta form ensures that all state delta values are both proper and type correct. Definition: A proper state delta value contains at most one atomic delta that refers to any particular target object. The special delta value fail is a proper state delta value by definition. Definition: A type correct state delta value is one in which for all atomic deltas of the form < ins o v >, v ∈ class(o) and for all atomic deltas of the form < mod o ∆ >, ∆ ∈ Dτ , where τ = class(o). The special state delta value fail is type correct by definition. H2O State Deltafication: As was described in Section 2.2, the language that operates on instances of a class is the set of methods defined on that class. The language that operates on the H2O system state is the native programming language in which the object method calls are embedded. This means that the deltafication of application programs needs to account for any state changes resulting from arbitrary programs in that native programming language. However, as was shown above, that state is merely a mapping from object identifiers to object values. Thus, the only expressions in the application program that can change
37 the state are the constructors, destructors and method applications embedded in that program. Since these all result in calls to the methods new, delete and modify defined on the class State, it is sufficient to define deltafication in terms of these methods. This is done in the same manner that was done previously for object level delta forms. The deltafication of the State class methods is given in Figure 2.14. The method new results in an insertion atom, which holds the initial value of the object and the method delete results in a deletion atom. The method modify results in a modification atom, which holds a delta value applicable to values of the type of the reference object. Note that the method modify was defined to accept a new value for the object, rather than a delta value, as that value is what is of concern in the usual state-based implementation. The representation of the deltafication of the method modify appears to imply that the object level delta value is computed from this new object value. This is not the case, rather the object level delta value is passed from the deltafication method defined for the class of the object. (The passing of this delta value from the object to the state was implied by the call to system function StoreDelta in Figure 2.5 as part of the example Integer class delta form.) To simplify the presentation here, the details of this information exchange between the objects and the state will be ignored, and it will be assumed that the delta value called for in the state deltafication is correct. H2O State Smash: Recall from Section 2.2 that smash creates a new delta value preserving the semantics of sequential application of two operand delta values. For state delta values, the smash is computed by individually combining the atomic deltas that refer to the same target object. (This necessitates the restriction to proper state delta values.) The algorithm for smashstate is given in Figure 2.15. If an object is only referred to by one of the operand state delta values, the atomic delta targeting that object is copied to the new delta value. For any object referred to by both operand delta values, the atomic deltas targeting that object are combined according to the SmashAtoms table also given in
38 ¯2 ∆
¯1 ∆
¯ 1, ∆ ¯ 2) SmashAtoms(∆ < ins o v1 > < del o > < mod o ∆1 >
< ins < ins < ins < ins
o o o o
v2 ) v2 > v2 > v2 >
< del < del < del < del
o> o> o> o>
< mod o ∆2 > < ins o v3 > < del o > < mod o ∆3 >
where v3 = applyτ (∆2 , v1 ) and ∆3 = smashτ (∆1 , ∆2 ) smashstate (∆1 , ∆2 ) ∆new = {} ¯ | o is the target of ∆, ¯ ∆ ¯ ∈ ∆1 , ∆ ¯ ∈ ∆2 then if ∃∆ ¯ ∆new = ∆new ∪ ∆ ¯ | o is the target of ∆, ¯ ∆ ¯ ∈ ∆1 , ∆ ¯ ∈ ∆2 then if ∃∆ ¯ ∆new = ∆new ∪ ∆ ¯ ¯ ¯ 1 , o is the target of ∆ ¯ 2, if ∃∆1 , ∆2 | o is the target of ∆ ¯ 1 ∈ ∆1 , ∆ ¯ 2 ∈ ∆2 then ∆ ¯ 1, ∆ ¯ 2) ∆new = ∆new ∪ SmashAtoms(∆ return ∆new Figure 2.15: H2O State Smash Function. Figure 2.15. In all but two cases, the atomic delta from the second operand delta value is copied to the new delta value. In the case where the first operand delta value has an insertion atom and the second has a modification atom, an insertion atom is added to the new delta value. The value stored with the new insertion atom is computed using the apply function defined in the delta form for the class of the target object. In the case where both operand delta values have modification atoms referring to the same object, a modification atom is added to the new delta value. The delta value stored with the new modification atom is computed using the smash function defined in the delta form for the class of the target object. H2O State Merge: The merge of two state delta values is computed in the same manner as the smash, by comparing the atomic deltas referring to the same object. The algorithm for mergestate is given in Figure 2.16. The difference between
39
¯2 ∆
¯1 ∆
¯ 1, ∆ ¯ 2) MergeAtoms(∆ < ins o v1 > < del o > < mod o ∆1 >
< ins o v2 > < del o > < mod o ∆2 > ¯3 fail fail ∆ fail < del o > < del o > ¯4 fail < del o > ∆
¯ 3 = if v1 = v2 :< ins o v2 >, fail otherwise where ∆ ¯ 4 = if mergeτ (∆1 , ∆2 ) = fail : f ail, and ∆ < mod o smashτ (∆1 , ∆2 ) > otherwise mergestate (∆1 , ∆2 ) ∆new = {} ¯ | o is the target of ∆, ¯ ∆ ¯ ∈ ∆1 , ∆ ¯ ∈ ∆2 then if ∃∆ ¯ ∆new = ∆new ∪ ∆ ¯ ¯ ∆ ¯ ∈ ∆1 , ∆ ¯ ∈ ∆2 then if ∃∆ | o is the target of ∆, ¯ ∆new = ∆new ∪ ∆ ¯ ¯ ¯ 1 , o is the target of ∆ ¯ 2, if ∃∆1 , ∆2 | o is the target of ∆ ¯ 1 ∈ ∆1 , ∆ ¯ 2 ∈ ∆2 then ∆ ¯ new = MergeAtoms(∆ ¯ 1, ∆ ¯ 2) ∆ ¯ if ∆new = fail return fail else ¯ new ∆new = ∆new ∪ ∆ return ∆new Figure 2.16: H2O State Merge Function.
40 state
¯ ∆
¯ o) ApplyAtom(∆, < ins o v > < del o > < mod o ∆ >
o → ⊥ insert(o, v ) (none) (none)
o → v modify(o, v ) delete(o) modify(o, applyτ (∆, v))
applystate (∆, state) new state = state for all oids o referenced in ∆ ¯ ∈ ∆ | o is the target of ∆ ¯ let ∆ ¯ ApplyAtom(∆, o) Figure 2.17: H2O State Apply Function. smash and merge is that merge checks for conflict between the operand delta values for any object referred to by both. This is done by computing a new atomic delta for the object by the table also given in Figure 2.16. Deletions and insertions are in conflict and result in fail. Two insertions are in conflict unless both call for the same initial value. Two modifications are combined using the merge operator defined in the delta form for the class of the target object. That object level merge operation may also result in fail, causing the state merge to result in fail. H2O State Application: The application function is used to transform a value by applying the changes encoded in some delta value. This implies that apply needs to construct a new value. However, as will be fully explained in Section 2.4, it is not always necessary to construct full database states. As a result, a precise specification for application of state delta values cannot be given. However, to define the general semantics of state delta value application, Figure 2.17 gives an algorithm that assumes that materialized states are actually constructed. For any object referenced in the delta value that is to be applied, the atomic delta referring to the object is extracted from the state delta value. This atomic delta value is applied to the value of the object in the current state, according to the function ApplyDelta given in tabular form in Figure 2.17. The
41 s0 o → ⊥ o → v0 o → v0
s1 = [ ]] new(o, v1 ) o→ v1 delete(o) o→ ⊥ modify(o, v2 ) o → v2
∆ = [] s2 < ins o v1 > < del o > < mod o ∆ >
= apply(∆) o → v1 o → ⊥ o → v2
Figure 2.18: Analysis of Deltafication and Apply on State Delta Form. table shows the methods which would be executed against the state object to perform the application. Two cases here merit attention. If an insertion atom is applied when an object already exists with the same oid as the target of the insertion atom, the current value of the object is modified to the value associated with the insertion atom. Second, if a modification atom is applied when an object value exists, the apply operation specified in the delta form for the class of the target object must be used to apply the delta value stored in the modification atom to the current value of the object. Neither of these operations will cause a type error if the state of the delta value is type correct (and all object level delta forms are correctly defined). Correctness of H2O State Delta Form: Proposition 2.6:
The H2O state delta form satisfies Require-
ments 2.1, 2.2 and 2.3. Proof:
Requirement 2.1: The possible results of deltafication are
listed in Figure 2.18. In each case the value of the affected object in the state resulting from direct evaluation of the expression (s1 ) is the same as the value of that object in the state resulting from deltafication and application (s2 ). Requirement 2.2: The smash and apply relationship holds for the state if it holds for any pair of atomic deltas targeting the same object. Figures 2.19 and 2.20 enumerate all possible combinations of atomic deltas targeting the same object in two delta values. Figures 2.19 shows the value of that object if the delta values are first smashed, then applied to the initial state (s0 ) with the results listed under state s1 . Figure 2.20 shows the value of that object if the two delta values are applied sequentially, with the results shown under state s3 . In all cases state s1 equals state s3 .
42
s0 o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0
∆1 < ins o v1 > < ins o v1 > < ins o v1 > < ins o v1 > < del o > < del o > < del o > < del o > < mod o ∆1 > < mod o ∆1 > < mod o ∆1 > < mod o ∆1 > (none) (none) (none) < ins o v1 > < ins o v1 > < ins o v1 > < ins o v1 > < del o > < del o > < del o > < del o > < mod o ∆1 > < mod o ∆1 > < mod o ∆1 > < mod o ∆1 > (none) (none) (none)
∆2 (none) < ins o v2 > < del o > < mod o ∆2 > (none) < ins o v2 > < del o > < mod o ∆2 > (none) < ins o v2 > < del o > < mod o ∆2 > < ins o v2 > < del o > < mod o ∆2 > (none) < ins o v2 > < del o > < mod o ∆2 > (none) < ins o v2 > < del o > < mod o ∆2 > (none) < ins o v2 > < del o > < mod o ∆2 > < ins o v2 > < del o > < mod o ∆2 >
∆3 = s1 = smash(∆1 , ∆2 ) apply(∆3 , s0 ) < ins o v1 > o → v1 < ins o v2 > o → v2 < del o > o → ⊥ < ins o v3 > o → v3 < del o > o → ⊥ < ins o v2 > o → v2 < del o > o → ⊥ < del o > o → ⊥ < mod o ∆1 > o → ⊥ < ins o v2 > o → v2 < del o > o → ⊥ < mod o ∆3 > o → ⊥ < ins o v2 > o → v2 < del o > o → ⊥ < mod o ∆2 > o → ⊥ < ins o v1 > o → v1 < ins o v2 > o → v2 < del o > o → ⊥ < ins o v3 > o → v3 < del o > o → ⊥ < ins o v2 > o → v2 < del o > o → ⊥ < del o > o → ⊥ < mod o ∆1 > o → v4 < ins o v2 > o → v2 < del o > o → ⊥ < mod o ∆3 > o → v5 < ins o v2 > o → v2 < del o > o → ⊥ < mod o ∆2 > o → v6
where ∆3 = smashτ (∆1 , ∆2 ), v3 = applyτ (∆2 , v1 ), v4 = applyτ (∆1 , v0 ), v5 = applyτ (∆3 , v0 ) and v6 = applyτ (∆2 , v0 ). Figure 2.19: Analysis of Smash and Apply on State Delta Form (Part 1).
43
s0 o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0 o → v0
∆1 < ins o v1 > < ins o v1 > < ins o v1 > < ins o v1 > < del o > < del o > < del o > < del o > < mod o ∆1 > < mod o ∆1 > < mod o ∆1 > < mod o ∆1 > (none) (none) (none) < ins o v1 > < ins o v1 > < ins o v1 > < ins o v1 > < del o > < del o > < del o > < del o > < mod o ∆1 > < mod o ∆1 > < mod o ∆1 > < mod o ∆1 > (none) (none) (none)
∆2 (none) < ins o v2 > < del o > < mod o ∆2 > (none) < ins o v2 > < del o > < mod o ∆2 > (none) < ins o v2 > < del o > < mod o ∆2 > < ins o v2 > < del o > < mod o ∆2 > (none) < ins o v2 > < del o > < mod o ∆2 > (none) < ins o v2 > < del o > < mod o ∆2 > (none) < ins o v2 > < del o > < mod o ∆2 > < ins o v2 > < del o > < mod o ∆2 >
s2 = s3 = apply(∆1 , s0 ) apply(∆2 , s1 ) o → v1 o → v1 o → v1 o → v2 o → v1 o → ⊥ o → v1 o → v3 o → ⊥ o → ⊥ o → ⊥ o → v2 o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → v2 o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → v2 o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → v1 o → v1 o → v1 o → v2 o → v1 o → ⊥ o → v1 o → v3 o → ⊥ o → ⊥ o → ⊥ o → v2 o → ⊥ o → ⊥ o → ⊥ o → ⊥ o → v4 o → v4 o → v4 o → v2 o → v4 o → ⊥ o → v4 o → v5 o → v0 o → v2 o → v0 o → ⊥ o → v0 o → v6
where ∆3 = smashτ (∆1 , ∆2 ), v3 = apply(∆2 , v1 ), v4 = apply(∆1 , v0 ), v5 = apply(∆2 , v4 ) and v6 = apply(∆2 , v0 ). Figure 2.20: Analysis of Smash and Apply on State Delta Form (Part 2).
44 ∆1 ins v1 ins v1
∆2 (none) ins v2
ins v1 ins v1
del mod ∆2
del del del o del mod ∆1 mod ∆1
(none) ins v2 del mod ∆2 (none) ins v2
mod ∆1 mod ∆1
del mod ∆2
(none) (none) (none)
ins v2 del mod ∆2
smash(∆1 , ∆2 ) smash(∆2 , ∆1 ) merge(∆1 , ∆2 ) ins v1 ins v1 ins v1 ins v2 ins v1 if (v1 = v2 ) ins v1 else fail del ins v1 fail ins v3 ins v1 if (v1 = v3 ) ins v1 else fail del del del ins v2 del fail del del del del del del mod ∆1 mod ∆1 mod ∆1 ins v2 ins v4 if (v2 = v4 ) ins v2 else fail del del del mod ∆3 mod ∆4 if (∆5 = fail) mod ∆5 else fail ins v2 ins v2 ins v2 del del del mod ∆2 mod ∆2 mod ∆2
where v3 = applyτ (∆2 , v1 ), v4 = applyτ (∆2 , v2 ), ∆3 = smashτ (∆1 , ∆2 ), ∆4 = smashτ (∆2 , ∆1 ) and ∆5 = mergeτ (∆1 , ∆2 ). Figure 2.21: Analysis of Smash and Merge on State Delta Form. The case in which both delta values contain a modification atom targeting the same object (the case in which the final value of the object is v5 ) needs some clarification. In the first case, of smash and then application, v5 = applyτ (∆3 , v0 ) where ∆3 = smashτ (∆1 , ∆2 ). In the second case of sequential application, v5 = applyτ (∆2 , v4 ) where v4 = applyτ (∆1 , v0 ). These values are guaranteed to be equal by Requirement 2.2 on the delta form for the class τ of the object. Requirement 2.3: The smash and merge relationship holds for the state if it holds for any pair of atomic deltas targeting the smae object. All possible pairs are listed in Figure 2.21. In each case, the merge = fail if the smash does not commute.
2
45 Correctness of H2O State Delta Values: The following theorems state that all state delta values generated by the state delta form are both proper and type correct. Proper delta values ensure that the smash and merge operations will function correctly. Type correctness ensures that no type errors are introduced in constructed states by the apply operation. Proposition 2.7: All state delta values generated by the state delta form are proper. Proof: Null delta values are proper. Delta values created by deltafication are proper by the definition of deltafication. The set of proper delta values is closed under smash and merge because they generate only at most one atomic delta for any object, as shown by the algorithms in Figures 2.15 and 2.16.
2
Proposition 2.8: All state delta values generated by the state delta form are type correct. Proof: Object identifiers are assigned a class at object construction. Once the class function is updated at object construction the class assigned to the object identifier can never change, since object identifiers are never reallocated. Thus it is sufficient to prove that any generated insertion or modification atoms are type correct. (Deletion atoms are always type correct.) Insertion and modification atoms created through deltafication are type correct by the definition of deltafication, assuming that the object level delta forms are correctly defined. By the algorithm given in Figure 2.15, smash copies existing delta atoms in all but two cases. The copied atoms are type correct by definition. The first case, SmashAtoms(< ins o v >, < mod o ∆ >) =< ins o applyτ (∆, v) >, is type correct by the definition of applyτ . The second case, SmashAtoms(< mod o ∆1 >, < mod o ∆2 >) =< mod o smashτ (∆1 , ∆2 ) >, is type correct by the definition of smashτ .
46 By the algorithm given in Figure 2.16, merge copies existing delta atoms in all but two cases. The first case, MergeAtoms(< ins o v >, < mod o ∆ >) =< ins o applyτ (∆, v) >, is type correct by the definition of applyτ . The second case, MergeAtoms(< mod o ∆1 >, < mod o ∆2 >) =< mod o mergeτ (∆1 , ∆2 ) >, is type correct by the definition of mergeτ .
2
2.4
Delta Values and Multistates The presentation in the previous sections of this chapter has avoided
the issue of delta value storage and the manner in which states are constructed from delta values. This section defines the relationship between delta values and states, and the options available for the construction of states from delta values. This formal discussion will define an abstract store that implements the functionality called for by the class State that was defined in Section 2.3 and resolves the issues of delta storage that were abstracted in the specification of the deltafication methods presented in Sections 2.1 and 2.2. In addition to defining how a single state can be constructed from existing delta values, it will be shown that delta-based storage implicitly results in a multistate. In a multistate database (also called a multiversion database [GJ94]), multiple states are allowed to exist simultaneously. This ability to construct multiple simultaneous states from delta values is central to the use of Heraclitus[OO] in cooperative work applications, as it provides a natural and easily exploited mechanism for the support of versions. As was previously shown in Section 2.3, a database state can be modeled as a mapping from object identifiers to object values, which is a partial function of type state : oid → value. In a multistate or multiversion database, multiple states persist simultaneously. Accessing a single state in a multistate database therefore requires some form of state selector. This will be referred to in the current work as a state identifier, with domain sid. Thus a multistate
47
∆1 s0
∆2 s1
∆3 s2
s3
Figure 2.22: A State Specified by a Base State and Delta Values. is defined functionally as multistate : sid → state, where the domain state is the domain defined above for a single state. Expanding this definition yields multistate : sid → (oid → value). In some multistate databases, such as [GJ94], the equivalent definition, multistate : (sid × oid) → value is used, indicating that particular object values are accessed by a state identifier, object identifier pair. In the most general interpretation, the store in a multistate database with delta values could contain both materialized states and delta values, with domains state and delta respectively. Thus a multistore can be defined as multistore :< statemap, deltamap >, where statemap : sid → state and deltamap : did → delta. The new domain did is the domain of delta identifiers. Note that without the deltamap, the multistore consists of only a statemap which is equivalent to a multistate as defined above. Given this definition of a multistore, additional information is required to specify how the information stored in the multistore is to be used to construct states. This information is contained in a state specifier, domain statespec. For the discussion here, a delta value can be viewed as a function from states to states7 , delta : state → state. As indicated by the definition of smash, a sequence of delta values is equivalent to a single delta, which is obtained by smashing that sequence. This will be modeled by a stack of delta identifiers, stack[did], where a stack has the usual operations of push, pop, top and empty. A state specifier can now be defined as < sid, stack[did] >. The intuitive interpretation of this state specifier is that the specified state can be obtained by sequentially applying the deltas identified in the delta 7
This is technically incorrect. A delta value combined with the appropriate apply function defines a function from states to states, i.e., apply : delta → (state → state), apply(∆) : state → state. To simplify the presentation, the use of the apply operator to generate state transformation functions will be omitted.
48
∆0 empty
∆1 s0
∆2 s1
∆3 s2
s3
Figure 2.23: A State Specified by an Empty State and Delta Values. stack to the state identified in the specifier. The delta at the bottom of the stack is applied first, and the delta at the top of the stack is applied last. This is illustrated in Figure 2.22, which shows how the state specifier < s0 , [∆1 , ∆2 , ∆3 ] > specifies the state s3 . The type of a multistore can now be defined as multistore : statespec → state. While the specification of the multistore in terms of a base state and a sequence of delta values provides the most robust model, it can be simplified by observing that any base state can also be specified by a delta value from an empty state. A empty state is a state in which no objects exist, as when a database is initially created. With this observation, it is possible to define all potential states in terms of delta values and the same base state, namely the empty state. The base state can then be omitted from the state specification, significantly simplifying both the theoretical model and the system implementation. As will be described in Section 2.5, this is the model on which the current system implementation is based. From a theoretical perspective, this simplified model causes no loss of generality, however it does have an impact on system efficiency, as will be discussed in Section 2.5. By removing the base state, and defining all states in terms of the empty state, a multistore becomes simply a delta store, with domain deltamap, and a state specifier reduces to a delta stack, with domain stack[did]. This is illustrated in Figure 2.23, which shows the state s3 constructed from the empty state and the state specifier [∆0 , ∆1 , ∆2 , ∆3 ]. This is equivalent to the previous example, if ∆0 contains new object information (that is, insertion atoms) for all objects that existed in s0 in that example. Defining a multistore as both a delta store and a state generator, results in some ambiguity. Thus, from this point it will be assumed that only a delta store exists, and functions for extracting states from the delta store
49 StoreDelta(∆new , sspec, DStore) didact = top(sspec) ∆act = DStore(didact ) ∆act = smash(∆act , ∆new ) return [didact → ∆act ]DStore Figure 2.24: StoreDelta Function. will be defined. These state extraction functions will be called EvalState with type EvalState : statespec × deltamap → state or, equivalently, EvalState : stack[did] × deltamap → state. This function will be defined below, in the context of object value evaluation. In the previous sections of this chapter, the interaction between the object and state interfaces and the delta store was abstracted through two system functions: StoreDelta and EvalObj. Recall that the function StoreDelta took a delta value created through deltafication and stored it in the delta store. The function EvalObj determined an object’s value from an existing delta store. When these functions were first introduced (in Section 2.1), the contextual information, namely the delta store and the relevant state specifier, was omitted. This was because that information is, in a sense, global information from the point of view of the application program and delta forms implementing the deltafication and state materialization. Now that state specifiers and delta stores have been defined they will be included in the specifications of these functions. The delta store is a single global object, made persistent by the database system and accessible to all applications and users. State specifiers are unique to a single application. The mechanism by which a state specifier is selected to provide the context for an application program is deferred to the implementation details of Section 2.5 and Chapter 3. For this theoretical discussion it is sufficient to assume that some state specifier exists and is identified by the system, so that it can be accessed when the functions StoreDelta and EvalObj are executed. As was shown in Section 2.3, as the deltafication of the application
50 program proceeds, object level delta values are produced as object methods are invoked, and these object level delta values are captured in modify atoms by the state object. Additionally, insert and delete atoms are generated by the state object as constructors and destructors are executed. As these delta values are generated they are accumulated into the proper state delta values, and this accumulation is handled by the function StoreDelta. The changes defined by the new delta value must be immediately available in the rematerialization of the state against which the original update was apparently executed. (See, for example, the replacement implementation of the method Increment in Figure 2.5. The value of the object must be reconstructed by calling the function EvalObj immediately after the delta value is stored.) Thus the function StoreDelta modifies the delta store in the context of the state specifier for that apparent state, giving the function the following type: StoreDelta : delta × statespec × deltastore → deltastore. The proper place to store the new delta value is at the top of the stack of delta values in the state specifier. Recognizing that the smash operation preserves sequential semantics, the new delta value can be smashed with the delta value currently at the top of the state specification stack. The definition of the function StoreDelta given in Figure 2.24 shows how this is accomplished. The delta identifier of the active delta value is taken from the top of the state specifier stack and used to extract the current value of that delta value from the delta store DStore. The active delta value is then smashed with the new delta value and the delta store is updated8 such that the delta identifier of the active delta value maps to the new delta value resulting from the smash. Referring back to the previous example (see Figure 2.23) the state s3 was specified by the state specifier [∆0 , ∆1 , ∆2 , ∆3 ]. During the deltafication process, any changes should appear to affect the state s3 , hence the delta values generated by the deltafication should be smashed into delta value ∆3 , which is at the top of the state specification stack. 8 The notation used to update the map modeling the delta store is taken from denotational semantics (see, for example, [Sch88]). The [didact → ∆act ]DStore indicates that a new map is created that has the same value as the old map DStore, except that in the new map didact has value ∆act .
51 EvalObj(o, sspec, DStore) state = EvalState(sspec, DStore) return state(o) EvalState(sspec, DStore) < didtop , sspec >= pop(sspec) ∆top = DStore(didtap ) if empty(sspec ) then return applystate (∆top , empty) else return applystate (∆top , EvalState(sspec , DStore)) Figure 2.25: Eager Implementation of Object Evaluation Function. Given that atomic deltas are accumulated into existing state delta values, the initial creation of those state delta values must be specified. New delta values are simply empty sets, and are created when a deltafication process begins. The new delta value is pushed onto the top of the state specification stack and then selected as the active delta value. This issue is related to the question of how state specifiers are created and modified, and the details are dependent on the multistate control interface being used. As will be discussed at the being of of Chapter 3, there are two such interfaces available for H2O, the first being the H2O DBPL and the second being the State Specification Tree (SST). State specifier definition, new delta value creation and active delta value selection are discussed in more detail in Sections 2.5 and 3.2. The process of state and object materialization is handled by the function EvalObj, which extracts an object value from the delta store, given the object’s oid and a state specifier. Hence the type of this function is EvalObj : oid × statespec × deltastore → value. There are a number of possible implementations for the function EvalObj that computes an object value from a delta store and a state specifier. The first is to materialize the specified state, and then extract the object value in the usual single state manner. This is referred to as eager object evaluation, as shown in Figure 2.25. This method is obviously inefficient, as it will
52 EvalObj(o, sspec, DStore) state = EvalP artialState(o, sspec, DStore) return state(o) EvalP artialState(o, sspec, DStore) < didtop , sspec >= pop(sspec) ∆top = DStore(didtap ) ∆ = split(o, ∆top ) if empty(sspec ) then return = applystate (∆ , empty) else return applystate (∆ , EvalP artialState(sspec , DStore)) Figure 2.26: Lazy Implementation of Object Evaluation Function. force recomputation of the entire database state, every time an object value is accessed. However, it is the most conceptually simple implementation and adequate from a theoretical perspective. The second possible implementation for object evaluation makes use of the fact that delta values can be split to extract just the portions of the delta values identified by the state specifier that target the object of interest. This is referred to as lazy object evaluation, as shown in Figure 2.26. The function split extracts the portion of a state delta value referring to a particular object. (This will consist of at most one atomic delta.) In this way only the portion of the state that is required is evaluated. While these definitions of state evaluation are appropriate for defining the semantics of the operation, an implementation based directly on these algorithms will lead to obvious inefficiencies. These inefficiencies are due to recalculation of values that have not altered since the previous calculation. This is similar to the inefficiencies encountered during the expression evaluation process in pure functional languages, and can be overcome with techniques similar to those used in such languages. This issue will be discussed in the context of the H2O system implementation in Section 2.5.
53 2.5
H2O Implementation The H2O data storage manager has been fully implemented, accord-
ing to the theoretical definition given in the preceding sections of this chapter. In this section, an overview of the implementation is given and three technical issues are discussed. The first of these technical issues is the manner in which object delta forms are supported in the implementation. The second technical issues deals with the manner in which H2O supports non-locking updates through the deltafication process. This issue is essential to support the application development described in Chapter 4. The final issue is techniques to overcome inefficiencies inherent in the state materialization process. Overview of the H2O System Implementation: H2O is designed and implemented as an application layer operating over a native object-oriented database management system. This allows the unique functionality of H2O to be provided in an environment in which all the benefits of a full-fledged OODB are available. H2O is defined in terms of the interface defined by the ODMG-93 standard9 [Cat93] for object-oriented databases. The Object Database Management Group (ODMG) is a consortium of vendors and researchers working to standardize OODB interfaces to better facilitate the portability of applications across platforms. It was for this reason that H2O is defined in terms of that standard interface, rather than in terms of the interface of a specific product. The H2O system includes an implementation of a subset of the ODMG interface. As a result, H2O can be ported to other OODB platforms by reimplementing the mapping from the ODMG interface to the new platform, without requiring modifications to the functional modules of H2O itself. The native DBMS that currently supports H2O is the Versant Object Database Management System (version 5.0) [Ver97b]. An overview of the system architecture of H2O is shown in Figure 2.27. At the top of the figure are the run-time interfaces. At the left is the object access interface, by which application programs access objects in the same manner as a traditional state based DBMS. Application programs are written 9
Since the initial design of H2O, an updated standard, ODMG 2.0 [CB97] has been released. The changes in the new standard do not significantly impact the design of H2O.
54
Application Program
Multistate Control
Object Access Interface
Multistate Manager
ODMG Interface Method Application
Surrogate Objects
Object Value Materialization
State Specifier
Deltafication Delta Values Native OO DBMS
Figure 2.27: H2O System Architecture. in C++ binding of the ODMG object manipulation language (OML). As a result of the deltafication and state materialization processes, the portions of applications that deal with (apparent) single states of the database can be written exactly as they would be if they were operating on a single state database. At the top right is the interface to the multistate management module. This interface gives applications complete control over the deltafication and state materialization processes as well as the ability to access the delta manipulation functions such as smash and merge. The multistate management module itself implements the H2O state delta form (Section 2.3) and manages the delta store (Section 2.4). There are two available interfaces to the multistate manager, the H2O DBPL, which defines the interface in terms of operators integrated into the ODMG OML, and the State Specification Tree (SST), which provides the functionality through a persistent data structure. The SST is described in Chapter 3 and used to support the application of H2O
55 to cooperative work as described in Chapter 4. See [DHDD95] for details on the H2O DBPL. The current system implements the pure delta value model (no materialized states) and lazy object evaluation method, as defined in Section 2.4. The delta values are stored as objects in the Versant database, thus they are persistent and subject to Versant’s concurrency control. However, as will be discussed below, while the delta values are subject to concurrency control, the updates are non-locking in terms of application objects. The object delta forms (section 2.2) are supported by surrogate objects, which are also stored in the database. Surrogate objects are created as the result of object constructor invocation, and remain unchanged once created. When it is created, the surrogate object is assigned an object identifier by the native DBMS and this oid is the same oid used to identify the object in all delta values. The methods supported by the surrogate objects are the “replacement methods” suggested in Section 2.1 that implement the deltafication and state materialization processes at the object level. The specification of surrogate object classes is the means by which delta forms are defined as part of the application schema. This is discussed in detail below. The basic operation of H2O is also shown in Figure 2.27. The multistate manager supplies state specifiers, as defined by the application program, that are used to control the deltafication and state materialization processes. Through these processes, the surrogate objects maintain an apparent single state, while all information is actually stored in delta values. Object Delta Forms: The key to making H2O behave as an object-oriented database, while operating on stored delta values, is the implementation of the delta forms for the classes of all objects that can potentially appear in the database. This is supported in the H2O system through the subclassing mechanism of C++. This is shown for a Paragraph class in Figure 2.28, and will be described in detail below. This implementation of the delta form for a particular class may be coded in C++ by the application programmer or may be automatically
56
ms_value
ms_delta
Paragraph
Paragraph_ms_delta
ms_object
Paragraph_ms_object
Figure 2.28: Classes Required to Support Delta Forms. generated by H2O from a class specification written in the ODMG Object Definition Language (ODL). In addition, delta form implementations for system supplied classes (such as sets, lists and strings) are included in the H2O system library. Delta forms that are automatically generated from ODL class specifications all implement an object-granularity delta form for that class. A generic object-granularity delta form was defined in Section 2.2. For classes requiring application specific delta forms, the ODL generated delta form can be used as a template from which the new delta form can be coded. To illustrate the implementation of a delta form in this manner, the system generated objectgranularity delta form10 will be described. Figure 2.29 gives the ODL specification of the class Paragraph. Instances of this class have a string attribute to hold the value of the paragraph 10
In the code for this system generated delta form, shown in Figures 2.29 through 2.34, functions given in all capital letters, such as O NEW PERSISTENT, are functions specific to the Versant OODBMS. See [Ver97a] for details on these functions.
57 interface Paragraph (extent paragraphs) { void Value(in String new_value); readonly String Value(); attribute String value; }; Figure 2.29: ODL Specification of Paragraph Class. class Paragraph : public virtual ms_value { protected: d_String value; public: Paragraph(); virtual ~Paragraph(); virtual d_Ref Clone(); virtual d_Boolean Equals(d_Ref& other_Value); virtual void Print(ostream& os); virtual void Value(d_String new_value); virtual d_String Value(); }; Figure 2.30: ODL Generated Paragraph Class. and two operations to set and retrieve that string value. The first class generated from this specification is the C++ implementation of the class itself. This class, which has the same name as in the ODL specification, is shown in Figure 2.30. The class Paragraph inherits from the H2O defined class ms value (meaning multistate value), which allows it to be used in the construction of delta values. The class reimplements three methods, Clone, Equals and Print, that are used in processing delta values. As mentioned above, the objects that are actually accessed by the application program are surrogate objects. Surrogate objects inherit from the system class ms object, as shown in Figure 2.31. From the class ms object the methods Smash, Merge and Apply are inherited and reimplemented to define
58 class Paragraph_ms_object : public virtual Paragraph, public virtual ms_object { public: Paragraph_ms_object(); virtual ~Paragraph_ms_object(); virtual d_Ref Smash(d_Ref& d_Ref& virtual d_Ref Merge(d_Ref& d_Ref& virtual d_Ref Apply(d_Ref& d_Ref& virtual void Print(ostream& os);
d1, d2); d1, d2); d, v);
public: virtual void Value(d_String new_value); virtual d_String Value(); }; Figure 2.31: ODL Generated Paragraph Surrogate (Part 1). the delta form operations of the same name. These implementations are shown for the class Paragraph delta form in Figure 2.32. These three functions are accessed by the state delta form (in the multistate management module) as needed to process the state delta form functions (see Figures 2.15, 2.16 and 2.17). The deltafication process for the class Paragraph delta form is implemented through replacement of the two Value functions that were defined by the original ODL specification. The replacement methods are shown in Figure 2.33. In these implementations, the multistate manager methods EvaluateObject and ObjectModified respectively correspond to the EvalObj and StoreDelta methods of the state delta value that were defined in Section 2.4. The method ObjectModified leads to the generation of a modify atom in the state delta value. Thus these replacement methods operate as was suggested by the example given in Section 2.1. The constructor for the surrogate class, also shown in Figure 2.33, is modified to support object creation
59
d_Ref Paragraph_ms_object::Smash(d_Ref& d1, d_Ref& d2) { return d2->Clone(); } d_Ref Paragraph_ms_object::Merge(d_Ref& d1, d_Ref& d2) { if (d1 == d2) return d2->Clone(); d_Ref new_delta = O_NEW_PERSISTENT(Paragraph_ms_delta)(); new_delta->SetFail(); return new_delta; } d_Ref Paragraph_ms_object::Apply(d_Ref& d, d_Ref& v) { d_Ref delta_value = R_AS(Paragraph_ms_delta,d); Paragraph x = delta_value->Value(); d_Ref new_value = O_NEW_PERSISTENT(Paragraph)(x); return Ref_ms_value(new_value); } Figure 2.32: ODL Generated Paragraph Surrogate (Part 2).
60
void Paragraph_ms_object::Value(d_String new_value) { d_Ref obj = R_AS(Paragraph, ms_manager->EvaluateObject(d_Ref(this))); obj->Value( new_value); d_Ref delta = O_NEW_PERSISTENT(Paragraph_ms_delta)(*obj); ms_manager->ObjectModified(d_Ref(this), Ref_ms_delta(delta)); } d_String Paragraph_ms_object::Value() { d_Ref obj = R_AS(Paragraph, ms_manager->EvaluateObject(d_Ref(this))); d_String retval = obj->Value(); return retval; } Paragraph_ms_object::Paragraph_ms_object() : ms_object(), Paragraph() { VPP_CLASS_CONSTRUCTOR1(Paragraph_ms_object); d_Ref init_value = O_NEW_PERSISTENT(Paragraph)(); ms_manager->ObjectCreated(d_Ref(this), d_Ref(R_AS(ms_value,init_value))); } Figure 2.33: ODL Generated Paragraph Surrogate (Part 3).
61 class Paragraph_ms_delta : public virtual ms_delta { private: Paragraph value; public: Paragraph_ms_delta(); Paragraph_ms_delta(Paragraph& _value); virtual ~Paragraph_ms_delta() { } virtual d_Ref Clone(); virtual void Value(Paragraph& _value) { value = _value; } virtual Paragraph& Value() { return value; } virtual void Print(ostream& os); }; Figure 2.34: ODL Generated Paragraph Delta Value. by calling the method ObjectCreated on the multistate manager. This leads to the generation of a insert atom in the state delta value. Support for object deletion, is implemented in a generic manner and requires no support from the surrogate object. The final class generated for the delta form is the class that implements the delta value itself. This class, called Paragraph ms delta in the current example, is shown in Figure 2.34. All object delta values inherit from the system class ms delta, which allows them to be used in the atomic deltas in the state delta value. A number of utility functions are also inherited and reimplemented for each delta value class to be used in the generic manipulation of the delta values. Nonlocking, Persistent Updates: A key feature of H2O in the support of groupware application (see Chapter 4) is support for non-locking persistent updates. This means that users can execute updates against (apparent) database states, without the interference usually encountered due to the conflict prevention policies of the DBMS concurrency control. However, the updates are still durable and atomic. This is supported in H2O by capturing user’s updates as delta values, in a manner orthogonal to the transactions of the native DBMS. This means that user’s can begin and end deltafication processes completely independent of
62
application 1 ∆1 method calls
object
∆2 application 2 Figure 2.35: Deltafication of Concurrent Applications. start and completion (through commit or abort) of transactions. For example, a user can start an update (begin a deltafication) and commit or abort any portions of the ongoing update, then end the update, resulting in a delta value that captures committed portions of the update. The deltafication process, and therefore the updates captured by it, does not induce locking at the level of application objects because of the use of surrogate objects and delta values. The surrogate objects are created with a unique oid and never changed once created, so they are read-only objects. Because the native DBMS guarantees unique oid generation at every object creation, surrogate object creation cannot cause conflicts. Besides surrogate object creation, the delta values themselves are the only things being modified in the database. As long as all concurrent updates are captured in different delta values (this is enforced at the multistate manager control interface, see Section 3.2) there is no possibility of conflicts that may require locking. This process is shown graphically in Figure 2.35 showing method calls from concurrent applications against the same object being diverted into different delta values. The support for non-locking updates assumes a delta-centric storage of delta values, as in the current implementation. This means that the set of atomic deltas that makes up a delta value is stored in the database as a
63 single object. Another possible storage option, called object-centric storage, stores all the atomic deltas (from all existing state delta values) that target a particular object as part of the surrogate object for that target object. While object-centric storage is not effective in the kinds of applications addressed in the current research, it has benefits in terms of efficiency in the state materialization process and was used in the implementation of an earlier version of H2O. Run-time Efficiency: The deltafication and state materialization implemented in H2O are based directly on the theoretical definitions given in Section 2.4. While that definition is convenient for defining the semantics of the process, implementation directly from the algorithms given here will lead to obvious inefficiencies. This is because they require recomputation of the smash of a stack of delta values for every object access. There are two particular situations when this inefficiency could become critical. The first is when state specifiers consist of relatively large delta stack, requiring many smash computations to yield a single object value. The second is when there are very large delta values that are relatively stable, requiring large numbers of redundant recomputations. A particular example of the second situation is when relatively small delta values are being used to specify temporary hypothetical states11 relative to a large base state. If, as suggested above, the base state must be specified by a delta value from the empty state, the amount of unneeded recomputation of stable object values can become intolerable. The inefficiency due to the use of delta values is similar to that encountered in expression evaluation in functional programming languages. In such languages, a pure specification of the semantics assumes that there is no state and that all expressions are evaluated as needed. To overcome the inefficiencies due to redundant evaluations, implementation of functional programming languages use a mechanism called memoization to store and reuse previously computed expression values, without changing the semantics of the 11
This is the kind of situation that the H2O DBPL operator when is designed to support. See [DHDD95, DDD+ 96] for examples of this sort of application.
64 language. A similar process is possible for H2O, in that object values can be memoized once computed and held as long as a particular state specifier is stable. Particular strategies for such object memoization have not been explored in the context of the current research, but the indications are that it can be effectively used so that the semantics of a pure delta value data store can be implemented with acceptable efficiency.
CHAPTER 3 SST: AN INTERFACE FOR COLLABORATION CONTROL This chapter describes the State Specification Tree (SST), which serves as an interface to the H2O multistate manager. The SST is a persistent data structure that organizes delta values to specify states and exports the control functions of the multistate manager. The SST serves the same function as the H2O DBPL [DHDD95] in which the control interface to the multistate manager was embedded in language constructs of the H2O OML interface. The H2O DBPL supplies operators for deltafication, apply, smash, merge and when1 . The first four of these operators implement the functionality described in Section 2.2 and the when operator implemented a hypothetical or temporary application. The usability of the H2O DBPL was proven when it was used in the implementation of a prototype telecommunications customer service application [DDD+96], although the development of that application did indicate some awkwardness in the use of the deltafication expression. In attempting to design groupware applications using H2O, a number of problems and shortcomings of the H2O DBPL interface became apparent. As these problems led directly to the system redefinition that resulted in the SST, they will each be briefly described here. The first problem was the tight integration of the H2O DBPL operators and the H2O OML interface through which applications accessed objects stored in H2O. In applications, such as the telecommunications application mentioned above, where the sequencing of the operations on the multistate manager and delta values can determined and programmed at application development time, this integration is appropriate. However, in groupware applications such as authoring, the decisions as to when and why such control 1
Two additional operations, compose and reverse deltafication, were also defined in theory but not included in the implementation.
66 operations should be invoked is generally best made by system users at runtime. (This observation also influences the interface design of such systems, as shown in Section 4.1.) For this reason it was necessary to decouple the multistate control interface from the application program. Because of the nature of the H2O DBPL operators, any control that affected the internal state of the multistate manager is transient when using that interface. While any delta values generated through that interface are persistent, any states constructed from the delta values are transient. For example, the when operator can be used to define a sequence of delta values that define a temporary state. But that state can only exist within the scope of the when operator. Similarly, any states created in that manner are not accessible to other programs operating on the same database. Sharing such information between programs would require defining application specific persistent data structures to pass delta values sequences between programs. Those sequences could then be used to reconstruct the states of interest. Because cooperating users need to be able to share such information about the internal state of the multistate manager it was necessary to develop a new interface in which such information was persistent. The final shortcoming of the H2O DBPL that had to be addressed to support cooperative applications was the limited ability to identify conflict between delta values (and the updates they represent). Using the H2O DBPL, conflict detection was limited to the use of the merge operator. As described in Section 2.2, merge fails if there is a conflict between the deltas involved in the merge. This is sufficient if conflict detection is adequate, but in cooperative applications it is generally important to be able to assist the users in conflict resolution by isolating and identifying the causes of conflict. As a result of the above analysis, the following requirements were placed on the design of a new interface to the multistate manager: • It must allow for the specification of deltafication scope and the specification of states to be materialized from existing deltas. These specifications must be persistent2 . 2
This persistent state specification is equivalent to the notion of an alternative, first
67
∆6 ∆4 ∆1
σ4
∆3 σ1
∆2
σ3
σ6
∆5
σ5
σ2
Figure 3.1: A State Specification Tree • It must support the delta combination operations of smash and merge. • It must facilitate the isolation and identification of the source of conflicts between delta values. This identification should be persistent. • Access to the functionality required by the first three points should be decoupled from application programs, so that interfaces to that functionality can be easily built for direct user access. The SST is designed as a control mechanism, and is not intended as a user interface. Rather, it is a low-level abstraction upon which such interfaces can be defined and constructed. The issue of proper user interface abstractions is dealt with in the context of an example application in Section 4.3. 3.1
SST Definition As a result of the requirements defined above, the State Specification
Tree (SST) was developed as an alternate interface to the H2O multistate manager. The SST was designed around the observation that any state can be specified by a delta value and another state (see Section 2.4 for more on the relationship between delta values and states). With states specified in that manner, a collection of states naturally forms a tree. Encapsulation of that tree as a persistent object and defining the required functionality as methods on the class of the object yields a simple and elegant solution to the problems outlined above. For example, the tree shown in Figure 3.1 specifies six states, in terms suggested in [BDD+ 95].
68 of six delta values. The states are represented by the nodes of the tree and the delta values are represented by the edges of the tree. The addition of a particular root state is required, since any tree has one node with no entering edges. This root state is naturally an empty state3 , in which no objects exist. The empty root state is identified in the figure by a square node. Formally, the SST is a set of state specifiers. A state specifier has the form < σ2 , ∆, σ1 >, where σ1 and σ2 are state identifiers (sids) and ∆ is a delta value. The first element, σ2 , is called the target state, the last element, σ1 , is called the parent state, and the middle element, ∆, is called the defining delta value. Thus a state specifier says that the value of the target state is equivalent to the result of applying the defining delta value to the value of the parent state, or value(σ2 ) = apply(∆, value(σ1 )), where apply is the H2O application operation as defined in Section 2.2. The root state, designated by the special sid root always has value empty, where empty is a state in which all oids map to “does not exist”. Referring back to the SST given in Figure 3.1, the value of that SST is
< σ1 , ∆1 , root >, < σ2 , ∆2 , σ1 >, < σ3 , ∆3 , σ1 >,
< σ4 , ∆4 , σ1 >, < σ5 , ∆5 , σ4 >, < σ6 , ∆6 , σ4 >
.
The empty state, with oid root is implicitly included in all SSTs, and is omitted when writing down SSTs. Thus an empty SST, with value {}, is a tree consisting of a single root node. With the addition of the SST (replacing the H2O DBPL interface) the system architecture is as shown in Figure 3.2. This system will be referred to as H2O/SST, and will be the assumed architecture for the remained of this document. From this architecture, it can be seen that the interface to the SST and the multistate manager control functions is cleanly decoupled from the object access interface. While this decoupling is preferable for certain applications, 3
The SST is defined with the philosophy of a ”pure delta” data store, in which no materialized states exist. It is possible to define the data store such that materialized states exists, in which cause the definition of the root state would have to be modified. See Section 2.4 for details on such variations.
69
Application Program
Users
Users
Application Runtine
SST Interface Multistate Control
Object Schema
Method Interface Delta Base
Delta Forms
Multistate Manager
State Specification Tree
Figure 3.2: System Architecture of H2O/SST. it does require that such applications implement any necessary synchronization between the two interfaces. This synchronization will be discussed in the context of groupware applications in Chapter 4. 3.2
SST Operation
Update Control: For single state operation, such as editing, an active state, σact , must be selected. The defining delta value for that state becomes the active delta, ∆act , for the H2O multistate manager (see Section 2.4). The delta values in the path back to the root of the tree become the stack of deltas which H2O uses to define context for object evaluations. The active state is denoted graphically as an encircled node (see, for example, the state σ5 in Figure 3.1). It is not possible to select the empty state as the active state, as it can not be modified. When an update is executed against the selected state that update is converted into a delta value by the deltafication process described in the previous chapter and smashed into ∆act , effectively updating the state σact . The active state is not part of the state specification tree, as a different active state must be selected by all concurrent users. This is necessary to prevent
70 interface { boolean boolean boolean
SST ParentState(in sid source, out sid result); DefiningDelta(in sid source, out Delta result); AffectedObjects(in sid source, out Set objects);
boolean NewState(in sid where, out sid result); boolean Copy(in sid source, in sid where, out sid result); boolean Delete(in sid source); boolean Split(in sid source, in Set target_objects, out sid result1, out sid result2); boolean Smash(in sid source, out sid result); boolean Merge(in sid source1, in sid source 2, out sid result, out boolean failed); boolean PartialMerge(in sid source1, in sid source2, out sid result1, out sid result2); }; Figure 3.3: State Specification Tree Interface. corruption of the active delta value during the deltafication process, and implies that a lock of some sort must be placed on the active state and delta value (see Section 3.3 for more on locks on the SST). Since the SST is persistent, it is possible to return to previously created states and to continue to update that state, with the update stored in the defining delta value for the state. Implicitly, the new update is combined with the existing delta value using smash semantics. (As shown in Section 2.3 deltafication is implemented through a method granularity smash.) This is in contrast to the H2O DBPL, in which the language syntax required that deltafication against a delta value could not be continued once the deltafication in which the delta value was initially created was completed. This combined with the fact that the deltafication process is orthogonal to concurrency control gives applications and users considerable flexibility in defining the boundaries of updates. SST Manipulation Methods: As stated above, the SST is a persistent object that is an instance of
71 a system supplied class. The methods available for manipulating the SST are defined by the SST class interface, which given in Figure 3.3. These methods were defined with the philosophy that the SST should make no assumptions about what are correct or incorrect manipulations of the states and delta values it manages. It is fully possible to construct SSTs in which “corrupted” states exist. However, since all the states are virtual, it is also easy to reverse undesirable manipulations. Because all SST methods (except Delete) add information to the SST, without destroying any information, there is no corruption of existing delta values or states. The following naming conventions are used for the state identifier parameters: • source is used for existing states which are used as the inputs to the operations, • where is used for existing states that identify where states created by the operations are to be attached, and • result is used for states created by the operations. For the sake of readability, in the text these will be referred to simply as “source states”, “where states” and “result states”. For correctness, these terms should be interpreted as “the state identified by the sid in the parameter source, (where,result)”. In cases where there are two parameters of the same kind, they will be referred to as the ‘first’ or ‘second’ of that kind. For example, “the first source state” refers to parameter source1 and “the second source state” refers to parameter source2. All the methods return a boolean value. This value is true if the preconditions for the operation are valid and false otherwise. A general precondition for all operations is that source states must exist in the SST. Additional preconditions for particular operation are given with the general descriptions below. In all cases, if the operation returns false, the SST is unchanged. Each of the SST methods will now be informally described. The first set of methods extract information from the SST. The methods ParentState and DefiningDelta extract the components of the state specifier associated with the source state. If the source state is σ2 , then the
72
where
result ∆
Figure 3.4: SST NewState Method. state specifier < σ2 , ∆, σ1 > must exist in the SST (by the precondition stated above). The method ParentState will return σ1 as the result state and the method DefiningDelta will return a copy of ∆ as the result delta value. The method AffectedObjects determines the oids of all objects that appear as targets of any atomic delta in the defining delta of the source state and returns those oids in the set valued parameter objects. The method AffectedObjects is instrumental in conflict resolution, as will be shown in Section 4.3. The remaining methods restructure the SST and in some cases add new states and deltas to the tree. The effect of each of these methods is depicted graphically, with the parameter names used as state identifiers in the graphs. Information that is added to the SST by the execution of these methods is indicated by a box with a dashed outline, and information that is removed from the SST is indicated by a hashed box. New states, with null defining delta values, are added to the SST by the method NewState (Figure 3.4). The result state is a new state attached at the where state. Since the defining delta value of the new state is null (has no effect when applied), the result state has the same value as the where state. This method will generally be used to capture isolated updates resulting in new versions of the state. For example, if a new state is added to the SST, then selected as the active state, the new state will initially have the same value as the source state and then any updates to that state will be captured in the defining delta value of the new state. The method Copy (Figure 3.5) creates a new state (the result state) and makes a copy of the defining delta value of the source state the defining delta value of the new state. The parent state of the new state is the where state. This has the effect of applying the changes which define the source state to the where state. The source state cannot be the empty state, since it has
73
source ∆1 where
result ∆1
Figure 3.5: SST Copy Method.
source ∆ Figure 3.6: SST Delete Method. no defining delta value. States and their defining delta values can be removed from the SST by the method Delete (Figure 3.6). The source state is removed from the SST, and its defining delta value is removed from the delta base. The method Delete is the only method that removes information from the SST. The state to be deleted cannot be the parent of any other states, since that would result in an unrooted tree. Also, the state to be deleted cannot be the root state. The method Split (Figure 3.7) creates two new states by breaking the parent delta value of the source state into two new delta values. One of the new delta values contains the portion of the original delta value that affects a set of target objects (identified by the parameter target objects). The other new delta value contains the remaining portion of the original delta value. If the delta values resulting from a split were to be merged, the resulting delta value would be identical to the one from which they were created. The new delta values become the defining delta values for two new result states and the parent state of the source state becomes the parent state for both result states. The source state cannot be the empty state, since it has no defining delta value. The methods Smash (Figure 3.8) and Merge (Figure 3.9) export the same functions defined for H2O in Section 2.2. However, through the SST they are restricted in their applicability (the general H2O operators could be applied
74
∆1 σ1
result1
∆0
source
∆2 result2 Figure 3.7: SST Split Method.
grandparent
∆ 1 parent ∆ 2
source
∆3 result Figure 3.8: SST Smash Method. to any delta values). This restriction ensures that the results are predictable. The Smash method smashes the defining delta values of the source state and its parent state. The delta value resulting from the smash becomes the defining delta value of a new result state. The parent of the new state is the grandparent state of the source state. By the definition of smash, the result state has the same value as the source state. The source state and its parent cannot be the root state, since it has no defining delta value. The method Merge has two source states, that must have the same parent state. The defining delta values of the two source states are merged and the resulting delta value becomes the defining delta value a new result state. The new result state has the same parent state as the source states. If the merge fails, the SST is unchanged and the parameter failed is set. The final method PartialMerge (Figure 3.10) overcomes the limitation of merge with respect to conflict detection. This method creates three states. The defining delta value of the first new state contains the portions of the defining delta values of the source states that do not conflict. The parent state of this new state is the same as the parent state of the source states.
75
source1 ∆1 parent
∆2
source2
∆3 result Figure 3.9: SST Merge Method.
source1 ∆1 ∆ 2 source2 ∆4
∆3
result1 result0
∆5 result2
Figure 3.10: SST PartialMerge Method. This new state also becomes the parent state of two new result states. The defining delta values of these two new result states contain the portion of the defining delta values of the two source states that do conflict. The two result states have the same value as the two source states, respectively. As with the method Merge, the two source states must have the same parent state. The use of the method PartialMerge, along with the method AffectedObjects, for the purpose of conflict identification, will be illustrated in Section 4.3. 3.3
Notes on Concurrency The SST is a single persistent data structure that is intended to serve
as a common interface to a group of users sharing a multistate. For the most part, the operations on the SST are nondestructive and do not conflict with each other. This means that users can concurrently make modifications to
76 portions of the SST without corrupting the multistate. However, there are a number of cases where it is necessary or desirable to implement some locking on the SST, either to prevent corruption of the multistate or to provide support for application defined control policies. This section outlines a possible locking scheme supporting three kinds of locks: a delete lock, a modify lock and a read lock. A delete lock on any node means that the Delete method cannot be invoked with that node as the target state. This lock is naturally intended to prevent specific states and their defining delta values from being removed from the SST. All interior nodes in the tree implicitly have delete locks, and delete locks can be assigned to any leaf node by the application or users. A modify lock on a node means that that node cannot be selected as the active state. This ensures the defining delta value of that state cannot be modified through deltafication. All interior nodes implicitly have modify locks, because modification of the delta values associated with those nodes could induce conflict with the delta values in the subtree of that node. When a node is selected as the active state a modify lock is placed on that node to prevent multiple deltafication processes from being applied to that node simultaneously, which could corrupt the defining delta value of the node. Modify locks can also be applied by applications or users to make certain states read-only states. A read lock is intended to be used to prevent access to the values of certain states. This might be used to keep the values of certain states hidden until they are released by the person who created them. They might also be used to prevent certain states from being used in the construction of new states, if, for example, it is known that the value of that state is likely to change. A read lock on a node prevents that node from being used as the source or where node in any SST method invocation except ParentState and also prevents the node from being selected as the active state. The method ParentState is allowed on read locked nodes because it doesn’t give any information on the value of the state, only on the structure of the SST.
CHAPTER 4 INCORPORATING H2O/SST INTO SYSTEMS TO SUPPORT COLLABORATIVE WORK This chapter addresses the second goal of the current research, the application of H2O/SST to cooperative work applications. As will be shown, H2O/SST can be easily used to provide data storage for a broad class of groupware applications. Moreover, this can be accomplished in a systematic manner, guided by a general architecture, allowing for reuse of system design results across various applications. This general architecture, called Coral, indicates the application specific design required to develop a groupware application using H2O/SST. The Coral architecture generalizes on the design principles developed during the implementation of a prototype authoring system called Reef, which is described in Section 4.2. While Reef supports sufficient functionality to test H2O/SST in the context of an authoring system, it lacks the high level interface support required to make a usable authoring system. The functionality that is lacking in Reef is centered around the user’s conceptions of interesting units of work and the change they represent. To demonstrate the support of this functionality in the Coral architecture, a proposed application called Eniwetok is described in Section 4.3. Eniwetok incorporates the basic user interface of Mj¨olner, namely the version evolution graph, as a better interface for user interaction with the changes represented by the delta values stored in H2O and structured by the SST. A series of examples based on Eniwetok operations will show how user’s activities map down through the Coral architecture to H2O/SST. In addition to showing how Eniwetok supports the Mj¨olner version evolution graph, the examples also illustrate additional functionality, such as variable granularity undo and collaboration awareness, that is commonly required in groupware applications and easily supported by the Coral architecture.
78 The example driven presentation of the Eniwetok application is followed by the development of a possible delta form for documents. This delta form has been implemented to support the Reef prototype and is directly usable as an application schema for Eniwetok. Moreover, the document delta form provides a concrete and realistic example of delta form development, illustrating the application of the theory of delta forms that was presented in Section 2.2. 4.1
The Coral Architecture for Delta-Based Groupware The Coral1 reference architecture for delta-based groupware applica-
tions defines a general system architecture for building applications that use H2O/SST as a data storage module. This architecture suggests a generic application design that abstracts and modularizes application specific semantics. This section defines Coral and outlines the development tasks required to build an application based on Coral. The Coral architecture is designed around the use of H2O/SST as a delta-based multistate data storage manager. In terms of the modular architecture for groupware systems proposed by Ellis and Wainer [EW94], H2O/SST fills the role of a keeper, responsible for maintaining the artifacts which are the focus of the collaboration. Throughout this chapter, the terms ‘data storage manager’ and ‘keeper’ will be used interchangeable. In addition to the keeper, their proposal defines three other modules: synchronizers, communicators and agents. The module most relevant in the context of Coral is the synchronizer, which coordinates the activities of users. As will be shown, the synchronizer functionality related to coordinating the user’s access to the data artifact must be handled by any groupware application built from the Coral architecture. An example of an agent module will be given in Section 4.3. Communicator modules are not addressed in the current work. The Coral architecture is given graphically in Figure 4.1. H2O/SST 1
The name ‘Coral’ is intended to invoke the notion of infrastructure in aquatic environments.
79
Users
Editing Interface
synchronization signals
Object Control
Delta Base
Control Interface
Delta Control
Multistate Manager
Transaction Control
State Specification Tree
H2O Figure 4.1: Coral Architecture for Cooperative Work Applications. is the foundation of the architecture. The remaining modules manage the control and translation between the H2O/SST keeper module and the application users. As with any groupware application, the functionality and interface presented to the users is of prime importance, thus Coral accounts for the necessary translation between the data driven functionality of H2O/SST and the conceptual constructs of potential application users. It is because the requirements of specific applications and groups of users are highly variable that Coral does not define specific functionality for the modules outside of the keeper. Rather Coral defines a framework in which specific functionality can be implemented. Induced by the separate asynchronous and synchronous activities in the specification of the target work cycle (as defined in Section 1.1) there are two distinct interfaces defined. The editing interface, supporting the asynchronous editing activity, is responsible for presenting an an apparent single version of the artifact and providing operations to manipulate that artifact
80 version. The control interface supports the synchronous consolidation activity in which users recombine their individual editing efforts. Thus the right side of the architecture is concerned with the presentation and manipulation of the representations of those individual and combined tasks. While the two interfaces are mostly responsible for the two different work phases, the correspondence between the editing and control interfaces and the editing and consolidation activities is not perfectly separable. Both activities require some access to the functionality and objects supported by both interfaces. As a result, some synchronization signals must be communicated between the two interfaces. As an example, an operation at the control interface that results in a change to the selected state on the SST requires that a refresh signal be sent to the editing interface to reload object values from the database. This point will be illustrated in the context of the application examples in Section 4.3. Below the interfaces are control modules, which interact directly with the H2O/SST system. The transaction control module, at the far right of Figure 4.1, allows users access to the transaction control of the native database system through the ODMG interface of H2O. This allows users to specify arbitrary transaction boundarys and to chose if particular transactions are to be committed or aborted. Thus, any activity occurring in either the editing or control interfaces is session independent in terms of the database concurrency control. Because the user interface to this module is simple and of limited interest, it is omitted from the architecture. Between the editing and control interfaces and H2O/SST are the object and delta management modules. These modules are responsible for maintaining the mapping from the objects and deltas at the H2O and SST interfaces to the appropriate objects at the user interfaces. They are also responsible for controlling access to those objects. The object control module accesses the objects of the artifact through the ODMG OML interface of H2O. This access takes the form of method calls on existing objects as well as calls to constructors and destructors. Because H2O presents an apparent single state to this interface (based on the current state of the SST, as described in Sections 2.4 and 3.1) the object control module
81 does not have to account for the fact that the underlying storage is implemented through delta values. The object control module must translate the objects accessed from the database into whatever representation is required at the editor interface, and must also account for migration to the native programming language of the editing interface module. This notion of making a distinction between the objects representing the artifact in the data store and the objects representing the artifact in the user interface is derived from the distinction between data objects (storage) and visual objects (user interface) made by Cortez [Cor95] as part of his notion of a groupware system definition (GSD). (See Section 5.2 for a description of a GSD.) This is a useful notion in the context of Coral, in that it allows the representation of the artifact in the user interface to be specified separately from the schema specification for H2O. This means that the interface can be designed according to user’s requirements, while the schema specification and associated delta form can be defined for optimal operation of the delta values. Distinguishing between visual objects and data objects, along with H2O’s ability to present an apparent single state has the added advantage that known authoring and editing interface implementation techniques can be directly applied to Coral editing interfaces. This is important since such interfaces are highly complex and substantial research has been done into the development of these interfaces. By not imposing any restrictions on how the editing interface can or should be constructed, the Coral architecture allows H2O/SST to be used in a wide variety of applications. The delta control module has a similar role to that of the object control module. It accesses the SST and the operations defined on the SST to determine the apparent state presented at the H2O ODMG interface and to organize the SST into meaningful delta configurations and states. Experiments with the prototype authoring system Reef (described in Section 4.2) made it apparent that the SST is a poor interface for direct user manipulation. Thus the delta control module also has to make a translation between the objects in the database (delta values and the SST) and the objects in the user interface (tasks, versions, alternatives, etc.). Since the appropriate user interface objects
82 are highly dependent on the requirements of specific applications, the general Coral architecture makes no suggestions as to what these objects should be or how the translation from the SST should take place. This will be discussed in depth in the examples in Section 4.3. In addition to maintaining the mapping from the SST and delta values to whatever are the appropriate user-level constructs for the particular application, the delta control interface must also implement any policies or procedures for controlling accesses to the SST. Such policies and procedures fall in the synchronizer module in the architecture of Ellis and Wainer and in the scope of the interaction rules of Cortez’s GSD. These policies and procedures will be called ‘coordination policies’ in the current work. Since such coordination policies are highly variable across applications and user groups, they are not specified by the Coral architecture nor by the SST (see Section 3.2). This is in distinct contrast to the usual database approach of implementing concurrency control policies as part of the DBMS, and further expands the range of applicability of H2O/SST and the Coral architecture. An interface that has been omitted from the Coral specification in Figure 4.1 is schema and delta form specification. This interface to H2O was previously indicated in Figure 2.27 and described in Sections 2.2 and 2.5. An example schema and delta form for documents is discussed in Section 4.4. 4.2
Reef: A Prototype Authoring System To test the functionality of H2O/SST in the context of the Coral
architecture, a prototype authoring system has been implemented. This prototype is called Reef (since it is constructed from Coral). While the Reef prototype was initially intended simply as a test and demonstration platform for H2O/SST, its development was instrumental in the design of the Coral architecture. Since Reef is intended as a prototype test platform, the implementation is limited to providing the necessary functionality for testing and demonstrating H2O/SST as a document storage manager. Consequently, it does not
83
∆6
The delta value access interface for Reef is also implemented in Python/Tk. As mentioned above, it implements no coordination procedures or policies, not does it support an translation from the delta values to specific task representations.
∆4 ∆1 σ0
σ4
∆3 σ1
∆2
σ3
σ5
σ2
Object Control
Delta Base
∆5
σ6
Transaction Control
Multistate Manager
State Specification Tree
H2O Figure 4.2: The Reef Prototype Authoring System. have sufficient functionality to make it a truly useful and practical authoring system. Nevertheless it has been sufficient for testing H2O/SST and for demonstrating the potential of the Coral architecture. The limitations will be discussed in the description of Reef below, and addressed in the context of a more realistic authoring system in the next section. The implementation of Reef is shown in Figure 4.2. As can be seen by comparison with Figure 4.1, it follows the Coral architecture, with separate interfaces for editing, update and version control and transaction control. The editing interface is implemented in the language Python [vR95], using its library access to the Tk interface to X windows. This interface consists of a graphical representation of the document and a text editor which can be loaded with any paragraph (by clicking on that paragraph in the upper
84 window). The document can be restructured at any level by graphical manipulation. (This hierarchical document model, which is based on the model used by several established authoring systems, and the associated delta form are described in Section 4.4.) While this interface is obviously lacking in the sophisticated editing operations expected of modern authoring and editing systems it is sufficient for executing all operations at the H2O interface. From the perspective of the current research, this limitation has no consequence, since the Coral architecture is specifically designed to support any editing interface and document format. The artifact access module in Reef maintains the mapping between the (virtual) objects stored in H2O and the visual objects manipulated by the authoring interface. In the current implementation, there is a direct oneto-one mapping between the operations supported by the authoring interface and the methods defined on the data objects. While this is a natural result in the current prototype, it is possible that the artifact access layer could be used to implement a more complex relationship between these operation sets, as described in the previous section. The object control module also acts as the interface between the Python environment of the editing interface and the C++ environment of the H2O interface. Python/Tk was chosen as the implementation language for this interface after experiments proved it to be a vastly superior platform for graphical user interface design than other platforms such as Tcl/Tk or C++ class libraries such as Interviews. This choice of Python as the user interface implementation language forced an initial division between the stored data objects, which exist in the C++ environment of H2O/SST, and the interface objects in the Python environment. Since this interface was developed prior to the Coral architecture, this implementation requirement forced a reconsideration of the roles of these objects in the Coral framework. This lead to the adoption of the separation of data objects and visual objects that was described in the previous section. The Reef update and version control interface is also implemented in Python/Tk. Since Reef is intended to test H2O/SST, the most natural interface is to export a direct graphical representation of the SST. While this interface
85 is ideal for the purposes of Reef, experimentation with this interface indicated that the SST is not an ideal interface for most applications as it is centered on the low level delta values appropriate for the H2O/SST keeper, rather than the application specific constructs appropriate for system users. This observation led to the requirement for an update and version control interface in Coral to allow for the proper user-level abstractions The Reef delta control module implements no coordination policies. Rather it simply allows all SST operations to be accessed through graphical manipulation. It is assumed that users will develop and enforce such policies on their own, external to the system. While this is in fact a viable policy in itself (for example, the Milo system [Jon95] was implemented with this philosophy) it does not illustrate the possibilities intended by the design of the Coral architecture. These possibilities are further explored in the discussion of a hypothetical authoring system in the following section. The final portion of the Reef implementation is the transaction control interface. This interface allows the user to specify the end points of transactions and to decide if particular transactions are to be committed or aborted. This is particularly important in that it tests H2O/SST to prove the orthogonality of updates, as supported by delta values, and transactions, as units of persistence. 4.3
Eniwetok: A Coral Application While the Reef prototype authoring system, as described in Sec-
tion 4.2, is sufficient for testing the H2O/SST implementation and for demonstrating the usefulness of the basic principles of the Coral architecture, it stops short of the implementation of application specific coordination policies and appropriate user-level conceptualizations. In this section, the design of an application with these features is developed, in order to show how such applications can be developed and as a further illustration of the usefulness of the H2O/SST keeper and the Coral architecture. This application, called Eniwetok2 , supports functionality typically required by authoring systems. As mentioned in the previous section, the editing interface was also 2
Eniwetok is a coral reef located in the South Pacific.
86 limited in the functionality required to make it a practical system. As this interface can make use of all existing techniques for editing interface design, and is unaffected by the use of H2O/SST as the keeper module, it will not be further addressed in the current work. The focus of the Eniwetok application is the functionality that was missing from the Reef prototype, namely the interface to the SST and control over the operations at that interface. Rather than define specific implementation details, this section will focus on a series of examples that illustrate the kind of functionality that might be provided at this interface. The examples are derived from the user interface of the Mj¨olner authoring system, as was described in Section 1.3 and illustrated in Figure 1.2. The primary user concepts support by Mj¨olner are versions, alternatives and alternative merging. The examples presented here will show how these interface constructs can be supported by defining a mapping from these constructs to the deltas and operations supported by the SST. Additionally, the examples will show how Eniwetok can support other functionality typically required by authoring systems and other groupware applications. While these examples are derived from authoring system and document artifacts, they are adaptable to other applications by replacing the artifact model (the H2O schema and delta forms) and editing interface. Example 4.1: Editing a Version with Undo: The most fundamental operation required of any authoring system is the ability to edit the document. In general, all the high level operations (such as copy and paste or replacement) can be easily mapped to direct changes on the artifact value in the database, and are therefore not of interest in the current research. However, the undo operation does require some interaction with the system’s version control (or, in the case of Coral based systems, delta control) system. In this example, it will be shown how Eniwetok supports a single user editing a single version with undo. This example maps to the ‘version’ creation option in the Mj¨olner interface. Without undo, supporting versions is simply a matter of adding a new delta value to the SST and selecting the state associated with that delta value
87
undoable operations
completed edit
smash
Figure 4.3: SST Operations to Support Editing With Undo. as the active state. Then all changes executed by the author at the editing interface are stored directly in the new delta value by H2O, resulting in a new state that properly reflects the version being updated. (See Sections 2.4 and 3.1 for details.) Supporting undo requires maintaining a sequence of changes that can be removed or unapplied. The specification of what determines an undoable edit operation is an application specific problem. However, this is easily handled in a generic manner by Coral based applications. In essence, the specification of the size of undoable operations is a problem in specifying the time granularity of deltafication. How this works in Eniwetok is illustrated in Figure 4.3. The delta control module can continually add new delta values to the SST, each originating from the version existing at the end of the previous undoable operations. Each delta value in the sequence then represents one undoable edit, and executing undo requires simply deleting the latest delta value and making the preceding state the new active state. Once the author is satisfied with the result of the editing session, the sequence of undo delta values can be smashed together to yield a single delta value representing the cumulative effects of the edit session. (Recall from Section 2.2 that smash preserves the sequential semantics of delta values.) By smashing and removing the undo delta values, the SST does not become unduly cluttered with transient information. However, this is an example of a policy decision chosen for Eniwetok, and other application could choose to keep such delta values in the SST. Note that the exact specification of what constitutes an undoable
88 operation has not been specified, nor can it be without knowledge of the primitive operations supported by the editing interface. This is an example of a case where some synchronization information must be exchanged between the editing interface and the update/version control interface. Assuming that the specification of what constitutes an undoable operation is part of the implementation of the editing interface, operation boundary signals must be sent from the editing interface to the update/version control interface to synchronize the interfaces and determine when the delta control module should add or delete delta values from the SST. Due to the mechanism by which H2O implements deltafication (see Section 2.5), the minimum size of undoable operations is limited to single method calls against the data objects stored in H2O. This example illustrates the need for a mapping between user concepts and delta values by the delta control module, which is lacking in the Reef prototype. The creation of undo delta values by direct user manipulation of the SST is obviously not desirable. However, if the delta control maintains a persistent mapping from the sequence of undo delta values to the more appropriate version object in the update/version control interface the mechanism is easily used without burdening the author. Additionally, because the creation of delta values is orthogonal to concurrency control in H2O, the decision to end an editing session by smashing the transient undo delta values is independent of the decision to commit or abort a transaction. Thus an author can safely commit an ongoing edit session and return at any time with the sequence of undo operations intact. Example 4.2: Concurrent Update of Alternatives: The second user construct supported by Mj¨olner is the creation of alternatives. Alternatives are simultaneously existing versions, which can be concurrently edited. This is easily handled in Eniwetok, repeating the process described above for single versions. Because H2O ensures that simultaneous delta values can be created on the same objects without locking, multiple delta values (or sequences of undoable delta values) and associated states can be created in the SST. Example 4.3: Merging Alternatives with No Conflict:
89 The third user construct in the Mj¨olner interface is the merging of existing alternatives to create a new version. Assuming that the alternative versions are represented by single delta values (that is, any undo delta values have been smashed out) the merge operation of the SST can be used to create a new delta value that constructs the merged alternative. However, as merge is defined to fail in the case of conflict, this will only be successful if there is no conflict between the delta values that specify the alternatives to be merged. (This is equivalent to saying that there is no conflict between the alternatives, from the authors’ points of view.) In the case of conflict, a more sophisticated procedure is required to account for the resolution of the conflict. Such a procedure is illustrated in the next example. Example 4.4: Merging Alternatives with Conflict: To merge alternatives when there is conflict requires a multiple step procedure, since there is no single operation on the SST that successfully merges conflicting deltas. This is because the nature of conflict resolution requires human intervention. Any merge that can be computed automatically by H2O is, by definition, not conflicting. This example will show how the conflict can be isolated, how user resolution of the conflict is supported and how the resolution is encorporated to yield the desired merged alternative. The process is illustrated in Figures 4.4 through 4.8. To avoid clutter, the delta values are not labeled in these figures. In the text the delta values will be referred to by the state that they define. Thus ∆2 in the text refers to the delta value that defines (points to) state σ2 . The identification of conflict is handled by the partial merge operation defined on the SST. As defined in Section 3.2, partial merge creates three new delta values, as shown in Fig 4.4. Delta value ∆3 contains all parts of delta values ∆1 and ∆2 that could be merged without conflict. Delta values ∆4 and ∆5 contain the conflicting portions of delta values ∆1 and ∆2 , respectively. By the definition of partial merge, the values of states σ1 and σ4 are the same, as are those of states σ2 and σ5 . Thus partial merge can be used to solve the first part of the conflicting merge problem, that of conflict identification and isolation.
90
σ1 σ0
σ2 partial merge
σ4
σ3
σ5
Figure 4.4: Conflict Resolution: Partial Merge. Once the conflict has been isolated, it is necessary for the authors to determine how the conflict should be resolved. This is accomplished by presenting the conflict to the authors and allowing them to make edits to resolve the conflict. These resolution edits are captured by the same deltafication process used for version editing above. The problem of conflict presentation is not addressed by the current research. In general this problem is dependent on the data model used for a particular application. For text documents, existing conflict presentation techniques such as the flexible diff used by the Prep editor [NCK+ 96] can be used for conflict presentation. Eniwetok supports conflict presentation to the extent of extracting the oids of the objects involved in the conflict (through the SST operation AffectedObjects) from the delta values created by partial merge. When conflict has been presented to the authors, they can chose to edit one or both of the alternatives to resolve the conflict. This is shown in Figure 4.5. As with any editing, the results are captured in the delta values ∆6 and ∆7 , resulting in two new states. (As in Example 4.1, the sequence of delta values to support undo could also be used to capture the resolution editing.) After the conflict has been resolved, the resulting states, σ6 and σ7 , can be merged. This first requires getting them defined in terms of two delta values from the common parent state σ0 , introducing some intermediate (but automated) SST manipulations. First, the resolution deltas can be moved to
91
σ1 σ0
σ2 partial merge
σ4
σ3
σ5
σ6
conflict resolution
σ7
Figure 4.5: Conflict Resolution: Manual Resolution.
σ1 σ0
σ8
σ2 partial merge
σ4
σ3
σ5
σ9 copy
conflict resolution
σ6 σ7
Figure 4.6: Conflict Resolution: Copy. the original alternatives, σ1 and σ2 , as shown in Figure 4.6. This is a safe operation, because these states have the same value as σ4 and σ5 , as a result of the partial merge. Once this is done, states σ8 and σ9 have the same values as states σ6 and σ7 and therefore have no conflict. In the next step the delta values defining states σ8 and σ9 can be smashed, as shown in Figure 4.7. This yields the desired non-conflicting delta values, ∆10 and ∆11 , defining states with the same values as the states resulting from the conflict resolution. These two delta values are then merged, as in Figure 4.8, to produce the final result. The state σ12 is the alternative representing the desired merge and incorporating the conflict resolution.
92
σ 10 σ 11 σ1 σ0
smash σ8
σ2 partial merge
σ4
σ3
σ5
σ9 copy
conflict resolution
σ6 σ7
Figure 4.7: Conflict Resolution: Smash. The process just described again illustrates that the SST is generally a poor interface for end users. User intervention was only required at the conflict resolution step and requiring users to execute (or even understand) the other steps is unreasonable. Again, if the delta control manager maintains a mapping from the delta values involved in this process to the user interface construct (alternative merge) appropriate to the users, this detail can be hidden from the users. The conflict resolution step can be potentially lengthy. The authors attempting to resolve the conflict may need to consult with others, or the conflict may simply be too complicated to resolve in one session. This illustrates another reason why the orthogonality of updates and delta values is important. As with single version editing above, a partially complete conflict resolution session can be committed to the database at any time, and returned to later with no loss of work. Example 4.6: Group Memory: Group memory has been identified by Ellis and Wainer [EW94] as
93
σ 10
σ 12
σ 11
merge σ1 σ0
smash σ8
σ2 partial merge
σ4
σ3
σ5
σ9 copy
conflict resolution
σ6 σ7
Figure 4.8: Conflict Resolution: Merge. the second responsibility of the keeper module (the first being storage of the artifact). In general terms, group memory refers to the storage of the history of actions of a group as well as reasons for those actions. Exactly what historical information needs to be kept is obviously application dependent. Much of this information may be unrelated to the artifact updates and is uninteresting in the context of the current research. What is of relevance in the current work is the history of the actions that do affect the artifact. In this example, it will be shown how Eniwetok might support the policy of an authoring system such as Collaborwriter [MG94] in which it is necessary to record all versions in the history of a document. Since all operations on the SST, with the exception of delete, strictly add information to the SST, it is a natural record of the updates and operations performed on the artifact, as well as all versions of the artifact. What is missing from the current implementation is a record of the sequence in which the operations where executed. Support for such additional information could be added by augmenting each state with pointers to the states from which
94
Users
Control Interface
Annotate Active Deltas
Delta Control
Notify Authors if Conflict Conflict Detection Agent Monitor Active Deltas
Delta Base
Multistate Manager
State Specification Tree
H2O Figure 4.9: System Operation to Support Conflict Detection Agents. they were computed. As an example, this would allow each step in the conflict resolution process (Example 5.5) to be traced back to the delta values on which the merge was originally intended. Additional historical information, such as who was responsible for an action and their reasons for doing it are highly application dependent and therefore not best supported directly by the SST. Nevertheless, a generic annotation facility on the SST would be useful for such support. Other policies for group memory can also be supported as procedures on the SST. For example, the extreme opposite of the Collaborwriter policy is no memory beyond the last interesting version. This can be implemented by smashing any delta values from the SST root to the state of interest and deleting any extraneous states and delta values. Example 4.7: Conflict Detection Agents: The Coral architecture is designed to support applications in which
95 the editing phase is completely asynchronous and authors are deliberately unaware of the updates being created by other authors. However, in many applications it is useful to have some awareness of the concurrent actions of others. In particular, it is often useful to discover potential conflicts between updates as soon as they develop. This can be supported in Coral applications such as Eniwetok through the introduction of conflict detection agents, as shown in Figure 4.9. Such agents could use the merge operation to test all pairs of active updates for conflict. (This would require allowing the agents to override some of the SST locks that were defined in Section 3.3.) Application specific policies such as what should be done when conflict is detected and how often states should be tested can be designed into the agents. 4.4
A Delta Form for Documents This section defines a document model and associated delta forms
that can be used in the application of H2O/SST to develop authoring systems. The model presented here is based on the hierarchical document models commonly used in authoring systems such as LINCKS [PL92, LSP93] and CES [GS88]. The presentation will begin with the document model that has been implemented for the Reef prototype authoring system (see Section 4.2). The capabilities and limitations of that basic model will be explored through a number of examples, and then potential extensions to overcome the limitations will be discussed. Basic Document Model: The hierarchical document model is defined in terms of components and paragraphs. A component is a structuring element that consists of a sequence of paragraphs of text and a sequence of subcomponents. This is illustrated by the example document instance shown in Figure 4.10. The objects marked with c are components, the objects marked with p are paragraphs of text and the objects marked with t and s are sequences of paragraphs and components, respectively. For example, if component c1 represents a section, then sequence t1 holds the initial text of that section (paragraphs p3 , p4 and p5 ) and sequence s1 holds the subsections (components c1.1 and c1.2 ). This
96
t 0
p1 p2
c0
t 1 s0
p3 p4 p5
c1
t 1.1 s1
c 1.1
p8 p9
s 1.1 t 1.2
p 10
c 1.2 s 1.2 t 2 c2
p6 p7
s2
t 2.1
p 11
c 2.1 s 2.1 t 2.2 c 2.2
p 12 p 13
s 2.2 Figure 4.10: An Example Document. model is more simple than would be required for the implementation of an actual authoring system, omitting many details (such as section titles) that are uninteresting in the context of the delta form design. Note that the sequences are distinct objects and not part of the component objects, rather each component object holds the oids of the sequence objects that maintain the text and subcomponents of that component. This is necessary in the context of the current H2O implementation and its basis in the object-oriented data model. This allows the sequence objects to have their own oids, and to utilize the system defined delta form for sequences (that delta
97
c
t
s
p P Figure 4.11: LDM Schema for Hierarchical Documents. form is fully described in Appendix A). This is in distinct contrast to the complex value approach, as used in [DHR96], in which the component type would be modeled as a tuple of two sequences with the delta form for a component defined in terms of the template delta forms for tuples and sequences. The object-oriented approach taken here is driven by the H2O state delta form (as described in Section 2.3) which requires that objects have oids in order to be maintained by the H2O system. With this approach, the H2O multistate manager is capable of directly handling the delta values for the individual objects that make up the document, while in the complex value approach the delta forms for container objects (such as sequences) would need to be defined with awareness of the delta forms for the objects they contain (such as paragraphs). The approach used here (and in the Reef implementation) has limitations related to the ability to detect violations of global constraints, as will be illustrated in the examples. This flat object-oriented approach is consistent with the logical data model (LDM) [KV93]. The document schema can be represented in LDM as shown in Figure 4.11. An LDM schema is a labeled directed multigraph, where each node has a particular type. Leaf nodes, denoted by
2 must be
98 of a basic type, for example, the one leaf node in this schema has the basic type paragraph. Interior nodes have structured types. The types used here are tuples, denoted by ⊗, and sequences, denoted by . The edges between the nodes indicate that instances of the type at the tail of the edge hold oids3 of the type at the head of the edge. The node marked c is the component class, modeled as a tuple containing the oids of two sequences. Sequences of paragraphs (the node marked t) contain oids of paragraphs, and sequences of components (the node marked s) contain oids of components. The equivalent document model can be represented by the ODL4 schema given Figure 4.12. The Ref construct in ODL indicates that the associated variable or parameter is an object identifier, called a reference in ODL. The class Paragraph has a single string valued attribute that holds the paragraph text, and methods to read and modify that string value. The class Sequence maintains a list of oids, in the attribute value and has methods for modifying and accessing that list. The class Component has attributes to hold the oids of the sequences maintaining its text and subcomponents, and methods to retrieve those oids. Note that the class Component has no methods to modify the values of the sequence oid attributes, making instances of that class read-only objects. (As was indicated in Chapter 2, H2O enforces the rule that all attributes are accessible only through the class’ methods.) Delta Forms for Basic Document Model: A delta form for the class Component is not required since instances of that class are read-only objects. Creation and destruction of instances of that (and any other) class are handled as changes to the H2O state value, and captured in the H2O state delta values (see Section 2.3). Delta forms for particular classes need only deal with changes to object values. An object-granularity delta form will be used for the class Paragraph. Object-granularity delta forms were defined generically in Section 2.2 and the default implementation of this delta form for the class Paragraph was given in 3
In [KV93] oids are called names. ODL is the ODMG object definition language, used for the specification of H2O schemas. See Section 2.5 for details. 4
99
interface Paragraph { attribute String value; void Value(in String new_value); String Value(); }; interface Sequence { attribute List> value; boolean move_to_head(in Ref move_oid); boolean move_to_tail(in Ref move_oid); boolean move_before(in Ref move_oid, in Ref target_oid); boolean move_after(in Ref move_oid, in Ref target_oid); boolean insert_at_head(in Ref ins_oid); boolean insert_at_tail(in Ref ins_oid); boolean insert_before(in Ref ins_oid, in Ref target_oid); boolean insert_after(in Ref ins_oid, in Ref target_oid); boolean remove(in Ref del_oid); Ref element_at(in integer position); }; interface Component { attribute Ref paragraphs; attribute Ref subcomponents; Ref Paragraphs(); Ref Subcomponents(); }; Figure 4.12: Object Schema for Hierarchical Documents.
100 Section 2.5. The delta form for the class Sequence is fairly complex, and is described in full in Appendix A. Details of this delta form will be described in this section as needed. As indicated in Figure A.2, sequence delta values have the form D, I, P, where D is a set of oids deleted from the sequence, I is a set of oids to be added to (inserted in) the sequence, and P is a set of anchored subsequences (called islands). The set of islands indicates relative positioning of the oids in the sequence, by defining disjoint subsequences relative to some anchor which might be an oid or the head or the tail of the list. Each island has the form [o1 , . . . , om , | a |, om+1 , . . . , on] where the | . | bracketed oid is the anchor of the island. The symbol ∧ will be used for the sequence head and the symbol ∨ will be used for the sequence tail. For example, the island [| ∧ |, o23 , o92 ] indicates that the subsequence [o23 , o92 ] should be positioned after the sequence head. The islands may included “phantom oids” which act as place holders for the oids that have been deleted or moved. The following series of examples will illustrate the basic usage of the proposed document model and delta forms. Example 4.8: Inserting a Component: As was indicated above, components are read-only objects. The constructors defining how objects should be initially created were omitted from the schema specification, but, since components are read-only objects, the values of their attributes must be determined when the object is constructed. This means that the construction of a component also requires the construction of two sequences. Suppose a component with oid c3 is created with a sequence of paragraphs having oid t3 and a sequence of subcomponents having oid s3 , and suppose the new component is attached to the document instance given in Figure 4.10 as the third subcomponent of the component with oid c0 . This means that the oid of the new component will be placed after subcomponent
101 c2 in sequence s0 . These changes are captured by the following delta value:
< ins c3 < t3 , s3 >>,
< ins s3 [ ] >,
< ins t3 [ ] >, < mod s0 < { }, {c3 }, {[| c2 |, c3 ]} >>
.
The modify atom at the end of the delta value indicates the change to sequence s0 . The delete set is empty and the insert set contains the oid of the component c3 that was added to sequence. The island in the positioning set indicates that the inserted component oid (c3 ) was positioned after the oid c2 , which is therefore the anchor of the island. Example 4.9: Modifying and Moving Paragraphs: Suppose two authors are creating concurrent updates, both working from the document state illustrated in Figure 4.10. The first author modifies the value of paragraph p8 and the second author also modifies the value of paragraph p8 and moves that paragraph to subsection c1.2 . This results in the following delta values: ∆1 = {< mod p8 “some string” >} ,
< mod p8 “a different string” >,
∆2 = < mod t1.1 < {p8 }, {}, {} >>, < mod t1.2 < {}, {p8 }, {[| ∧ |, p8 ]} >>
.
By the definition of the merge operation for the object-granularity delta form used for paragraphs, the two modify atoms targeting paragraph p8 conflict. The two modify atoms targeting the sequences t1.1 and t1.2 do not conflict. Thus if the authors attempt to merge these delta values, the merge will fail. If the partial merge operator is applied to these delta values, the following delta values will result:
< mod t1.1 < {p8 }, {}, {} >>, ∆3 = , < mod t1.2 < {}, {p8 }, {[| ∧ |, p8 ]} >> ∆4 = {< mod p8 “some string” >} ,
102 ∆5 = {< mod p8 “a different string” >} . Here, ∆3 contains the non-conflicting portions of the two source delta values and ∆4 and ∆5 contain the conflicting portions of ∆1 and ∆2 respectively. The delta value ∆3 will define the parent state of ∆4 and ∆5 . The conflict is now identified and isolated and conflict resolution can proceed as indicated in Example 4.4. Example 4.10: Conflicting Local Moves: Suppose again that two authors are creating concurrent updates from the state state shown in Figure 4.10. The first author moves paragraph p4 to the head of sequence t1 and the second moves paragraph p5 to be in front of paragraph p4 . This results in the following delta values: ∆1 = {< mod t1 < {}, {}, {[| ∧ |, p4]} >>} , ∆2 = {< mod t1 < {}, {}, {[p5, | p4 |]} >>} . By the definition of the merge operator for the sequence delta form (see Appendix A) these two delta values conflict and the authors will be required to manually resolve the conflict. The preceding two examples indicated cases where the given delta forms for documents successfully detects conflict between updates resulting from modifications and moves. These are local conflicts, as they could be detected from the changes to the values of individual objects. This is sufficient for ensuring consistency in the values of individual objects, however, it is not sufficient for ensuring consistency over the overall document. This problem, and potential solutions, will be explored in the following. Defining Conflict in Terms of Constraints: There are inherent limitations in the ability to analyze potential constraint violations when using delta values.
This is best illustrated by an
example taken from a telecommunications application, called Smart Talking [DDD+96], using the integer delta value described in Section 2.1. In that application, delta values were used to predict future states of the database, where one type of object was a terminal box. Objects of this type are constrained
103 in the number of lines that can be attached to the box. This problem can be abstracted by saying that there is a constraint on the upper value of some integer object. Suppose the constraint says that the value of the integer object with oid i9 has an upper bound of 100. Also suppose that the following delta values targeting that integer object exist: ∆1 =< mod i9 < add 10 >>, ∆2 =< mod i9 < add 20 >> . By the definition of the integer delta form, these two delta values do not conflict and can be merged resulting in ∆2 =< mod i9 < add 30 >> . However, in the context of the stated constraint, the definition of what is conflict is less clear. The problem here is that the constraint is defined in terms of allowed values of the object, or more generally, allowed states of the database. Thus it is not possible, in general, to determine if two delta values conflict with out the context of some state. In the context of a state in which the value of integer i9 is 10, there is no reason to consider ∆1 and ∆2 to be in conflict, but in the context of a state in which the value of that object is 75, it may be considered a conflict. In the Smart Talking application, this problem was dealt with by use of the H2O DBPL operator when, which permitted hypothetical application of delta values. If a hypothetical application resulted in a state which violated a constraint then that application was rejected. Similarly, sequential hypothetical application of two delta values, (or the hypothetical application of the merge of the two delta values) could be used to determine if two delta values conflict in the context of some state. The rest of this section will explore similar problems in dealing with constraints over the proposed document model. A Constraint on Paragraphs: In many cases it may be necessary to define a constraint stating that no paragraph can be included in the document in more than one position in
104
p1 p2
t 0 c0
p3
s0
∆1 t 1
c1 s1 t 2 c2
p3 p4 p5
∆2
p3 p6 p7
s2 Figure 4.13: Updates Violating Paragraph Constraint. the document. The following example indicates that this constraint cannot be maintained by the current document model and delta forms. Example 4.11: Undetected Constraint Violation: Suppose that two authors are creating concurrent updates from the state shown in Figure 4.13. The first moves paragraph p3 to be after paragraph p2 in sequence t0 and the second moves the same paragraph to be at the head of sequence t2 . This results in the following delta values:
< mod t1 < {p3 }, {}, {} >>, ∆1 = , < mod t0 < {}, {p3 }, {[| p2 |, p3 ]} >> < mod t1 < {p3 }, {}, {} >>, ∆2 = . < mod t2 < {}, {p3 }, {[| ∧ |, p3 ]} >> The modify atoms targeting sequence t1 do not conflict because they have the same effect. The other modify atoms, targeting different sequence objects and they also do not conflict (by the definition of the merge operation on the H2O state delta form). Thus conflict between these delta values cannot be detected, even though their combined application will cause a violation of the stated constraint on all states.
105 The reason that the constraint violation can not be detected in the preceding example is that the document model being used interprets the position of paragraphs as a property of the sequences, rather than the paragraphs. Thus the moves show up in the delta values as changes to the values of paragraphs. The solution is to modify the document model so that the position of a paragraph is also a property of the paragraph. Suppose an attribute contained in of type Ref were added to the class Paragraph and the appropriate modifications were made to the methods on the class Sequence to maintain that value when paragraphs are moved among the sequences. Example 4.12: Detection of Constraint Violation: Consider the same two updates described in Example 4.11. If the class Paragraph is modified as described above, then the updates result in the following delta values:
< mod t1 < {p3 }, {}, {} >>,
∆1 = < mod t0 < {}, {p3}, {[| p2 |, p3 ]} >>, < mod p3 < “value of paragraph”, t0 >>
< mod t1 < {p3 }, {}, {} >>,
∆2 = < mod t2 < {}, {p3}, {[| ∧ |, p3 ]} >>, < mod p3 < “value of paragraph”, t2 >>
,
.
In this case, the conflict will show up in the modification atoms targeting the paragraph p3 . A Constraint on Document Structure: It is natural to assume that the document is a tree. This can be enforced by a constraint stating that there will be no loops in the structure of the document. This is a more global constraint than the constraint on paragraph positions described above, as it cannot be maintained through the inspection of the values of single objects. For the discussion of the implications of this constraint it is convenient to simplify the model of the artifact to include only sequences of sequences. The results obtained from this simplification can easily be generalized back to
106
state σ 0
s1
s3
s0 s2 state σ 1 s0
s1
s3
s2
s2
s1
s3
state σ 2 s0
∆1 σ0
σ1
∆2 σ2
Figure 4.14: Document Structure Constraint Example. the full data model. This means that the type of all objects in the database will be Sequence (with a slight abuse of ODL syntax). Example 4.13: Delta Value Application Violating Structure Constraint: Consider the SST and three states shown in Figure 4.14. The delta value defining state σ1 from σ0 is
< mod s0 < {s2 }, {}, {} >> ∆1 = . < mod s3 < {}, {s2 }, {[| ∧ |, s2 ]} >> Now suppose that delta value ∆1 is copied to state σ2 , resulting in
107
s3
state σ 3 s0
s1 s2
∆1 σ0
σ1
∆2
∆1 σ2
σ3
Figure 4.15: Document Structure Constraint Example. state σ3 , as shown in Figure 4.15. State σ3 is an illegal state, because it contains a loop. In this example, a delta value that produced a legal state in one context produced an illegal state when applied in a different context. Note that the copy operator on the SST is the only operator that can result in such unrestricted application. This is the same as the situation described for the Smart Talking application above. In that application the solution was to use hypothetical application (the when operator) to test for a valid state before applying the delta value. Here the equivalent solution can be used. This amounts to a policy decision in the design of the authoring system. The required policy is to test the state resulting from any copy on the SST, and if the resulting state violates any constraints, the copy is disallowed and the the new state and its defining delta value are deleted. Example 4.14: Modifying Sequence Delta Form to Detect Cycle Conflicts: The problem of detecting inadvertent cycles due to delta application is equivalent to testing for conflict between the two delta values. Using the
108 normal sequence model, the delta values from the example given in Figure 4.14 are:
< mod s0 < {s2 }, {}, {} >> ∆1 = , < mod s3 < {}, {s2 }, {[| ∧ |, s2 ]} >> < mod s0 < {s1 }, {}, {} >> ∆2 = . < mod s2 < {}, {s1 }, {[| ∧ |, s1 ]} >> No conflict is detected bewteen these two delta values, despite the fact that they have been shown to violate the global constraint when applied together. This is because each of the individual sequences is consistent. The delta atoms for the only sequence targeted by both delta values, sequence s0 , can be merged with no conflict (see the details of the sequence delta value given in Appendix A). Better conflict detection to avoid loops can be achieved in a manner similar to that used to prevent multiply assigned paragraphs in Example 4.12. Two delta values that do not individually cause a loop in the document structure cannot violate that same constraint when combined, unless the two delta values assign different parent sequences to some sequence. Thus this constraint violation can be avoided by adding the parent sequence of each sequence to the delta values:
< mod s0 < {s2 }, {}, {} >, null >
∆1 = < mod s2 < {}, {}, {} >, s3 > < mod s3 < {}, {s2 }, {[| ∧ |, s2 ]} >, s1 >
∆2 =
< mod s0 < {s1 }, {}, {} >, null > < mod s1 < {}, {}, {} >, s2 > < mod s2 < {}, {s1 }, {[| ∧ |, s1 ]} >, s0 >
.
.
Now a conflict is detected at sequence s2 , since the delta atom targeting that sequence in delta value ∆1 assigns that sequence to parent sequence s3 , and delta atom targeting that sequence in delta value ∆2 assigns that sequence to parent sequence s0 .
CHAPTER 5 CONCLUSION 5.1
Summary of Contributions The current research has made substantial contributions in the sup-
port of applications in which the management of change is of primary importance. The H2O/SST data storage manager and supporting Heraclitus[OO] theory provide a general database solution in which change is given equal status with state. H2O/SST has been fully implemented and tested to ensure that it operates properly according to the theory presented in Chapter 2. The potential range of application for H2O/SST is quite broad, because the system and theory are integrated into the general object-oriented model. The research goes further than the general database solution by establishing an application space within the domain of cooperative work that can immediately benefit from the use of Heraclitus principles. Furthermore, a simple and flexible system architecture by which H2O/SST can be used to support such applications has been designed, and this architecture has been verified through the development of an example application. Contributions to Database Theory and Technique: The major technical contributions of this research, in terms of database application support are summarized in the following paragraphs. Each of these results is supported through theoretical development and system design and implementation. • Delta Values Over Object States: H2O is unique in the ability to provide delta values as first class objects in an object-oriented database. Existing systems supporting multiple states or versions of the database may use a mechanism equivalent to delta values to implement the multistate, but do not include the deltas in the theory defining the multistate (see, for example, [GJ94]), nor to they make
110 the delta values available to applications and users. • Application Semantics Through Delta Forms: Unlike the delta values designed for Heraclitus[Alg/C], which were designed around the static relational model, Heraclitus[OO] delta values need to accommodate the various object forms and semantics possible in the objectoriented model. This necessitated the development of delta forms to allow application specific delta value specification as part of the application schema. • Run-time Deltafication and State Materialization: The ability to compute delta values from updates and to rematerialize states in a run-time environments is a significant new technology for object-oriented systems. This result is applicable to both database systems and programming languages. This specific implementation techniques are portable to any object-oriented environment supporting classes and inheritance. This technology is fundamental in enabling the functionality described in the next two paragraphs. • Updates Orthogonal to Transactions: The creation of delta values through deltafication is orthogonal to the concurrency control mechanisms of the native database system. This allows the scope of updates to be defined in terms of meaningful units, from the perspective of applications or users, rather than being defined in terms of transactions. As a result, updates can be executed free from the consistency constraints imposed on transactions, while maintaining the other benefits of concurrency control, such as durability and atomicity. This functionality is useful for supporting applications in any domain where noninterference between user’s updates is desirable. • Transparent Application Interface: Because Heraclitus[OO] is fully integrated into the object-oriented data model, and because the mechanism of deltafication and state materialization are encapsulated behind the interfaces that objects present to applications, the process is transparent at the application interface. As a result, the portions of application programs that concentrate on updates to single states, can
111 be written with no awareness of the deltafication and state materialization processes. • Conflict Identification: Heraclitus[OO] supports application specific conflict semantics between delta values. This is supported through the specification of the merge operation in application supplied delta forms. Through the partial merge operation available through the SST, the identification of the sources of conflict, at the granularity of objects, is automated. • Multistate Control Interface: The system design cleanly separates the application development tasks of defining the delta forms for the application schema and defining the policies and procedures governing the access to and manipulation of the delta values and states. The control interface is the state-specification tree (SST) which is a persistent data structure that supports the specification of states and other delta values through manipulation of a persistent data structure. As demonstrated in Section 4.3, the SST is quite useful in defining application specific procedures for managing change and associated conflict by using the SST operations as atomic steps in change management. Contributions to Groupware Application Development: In addition to the specific developments in object-oriented database design noted above, the current research makes the following contributions in the area of groupware application development: • H2O/SST Keeper Module: The H2O/SST data storage manager has been shown to be a useful, general keeper module [EW94], for a range of related applications in groupware. H2O fully supports the object-oriented data model, with delta values defined in terms of the application schema, thus it can be used across a wide range of artifact types. Because of the nonlocking updates provided by the deltafication process, which effectively bypass the transaction policies of the native DBMS, H2O/SST is useful for supporting applications in which the policies regarding update concurrency are fluid or redefinable. It
112 is believed that this makes H2O/SST uniquely well qualified to provide data storage in the development of experimental research applications. A reference architecture, called Coral, guides the development of groupware applications using H2O/SST, and further motivates the separation of object access and delta and state control in groupware applications. • Prototype Authoring System: The Reef prototype authoring system tests and demonstrates the functionality supported by H2O/SST in the context of authoring systems. While Reef is far from a practically useful system, the current implementation can be useful in the development of more sophisticated systems, since it provides the basic required functionality in a manner that can be easily extended consistent with the Coral architecture. • Example Application Development: A hypothetical authoring system called Eniwetok has been designed, to demonstrate the application development required to construct an actual groupware application based on the Coral architecture. The development of this example system focuses on the control environment needed to interface to the SST to support a hypothetical authoring system interface. • Example Document Delta Form: A possible delta form has been developed for supporting documents. This delta form is developed in theory to support the Eniwetok application and has also been implemented to support the Reef prototype. In addition to illustrating the use of delta forms in application development, the document delta form stimulated the development of a new delta form for sequences that is of some theoretical interest. 5.2
Related Work The research outlined in this section is related to the various aspects
of the current research. This related research is organized according to the general research area in which it was developed. Deltas and Difference Computation:
113 Heraclitus[OO] follows from the earlier related work which focused on techniques for managing updates in relational databases. The idea of of developing generic programming constructs for the explicit manipulation of delta values, which capture state changes as first-class database values was first introduced in [HJ91]. This culminated in the development of an algebra of deltas along with a database programming language (DBPL) called Heraclitus[Alg,C] that incorporates the algebra into a prototype database environment [GHJ96] implemented on top of the Exodus storage manager toolkit [CDRS86]. Delta values, or “differentials”, have been used in numerous database and systems applications, for example, materialized views [BLT86, GL95], version management [SL76, Kat90], active databases [GHJ96, WC95], and datalog evaluation algorithms [BR88]. In those investigations, the deltas are supported in a variety of different ad hoc fashions, and are not directly accessible to programmers or users. The key novelty of the Heraclitus data storage managers is that deltas can be directly created, accessed and manipulated by programmers. Applications based on the Coral architecture capture the net difference implied by a user edit session as a delta value. References [ZS89, CRGMW96] develop algorithms for computing the net differences in connection with updates against ordered trees; these can be applied in the context of documents. Importantly, the algorithms are cognizant of subtree moves (i.e., cut-and-paste), in addition to inserts, deletes, and content-modifies. In contrast to H2O, in which the delta values are computed as the updates are executed, these algorithms compute the differences from materialized versions. Similarly, a number of algorithms have been developed for computing differentials over text or code versions [Mye86, SL76, HM75]. Database Concurrency Control: Although the success of traditional database concurrency control techniques for coordinating and isolating the actions of independent users has been well established, it has been recognized that these techniques are not adequate for the support of many modern applications [BK91, Elm92]. In particular, the isolation imposed by the assumption of competition between the users is counter to the needs of cooperative work systems [CKNT93, JE93].
114 The requirements of Computer-Aided Design (CAD) systems have lead to some research into new approaches to concurrency control [BKK85, PKH88] that address similar problems to those encountered in groupware. The design activities typical of CAD applications are generally open-ended in that they are of uncertain duration, contain unanticipated developments and interact with other concurrent activities [PKH88]. Although the first of these characteristics may not be true in cooperative work environments, the second two generally are. The first approach [BKK85] is to define a hierarchical structure of different kinds of transactions to support different needs. For example, a project transaction may consist of various cooperating transactions, which in turn contain client transactions. The protocol for concurrency control varies at each level according to the semantics of that level. The second approach [PKH88] is to use split-transactions to allow part of an open-ended transaction to be committed, thus releasing any resources it holds while keeping the rest of the transaction active. The ideas for adding application specific information to define potential conflict between updates is related to research in adding semantic information to concurrency control protocols [HW88, RAA94]. The main distinction is that conflict analysis in H2O is performed on the delta values which represent partial or complete updates, while the semantic concurrency control is based on traditional notions of locking. Also, H2O provides for the possibility of modifying proposed updates to resolve conflict, rather than simply aborting the conflicting updates. Version Management: Version management focuses on applications where the continued existence of multiple versions is assumed and the primary task of the system is to assist in the identification and maintenance of these versions. This is a distinctly different requirement than that addressed by the current work in which it is assumed that the shared goal of all authors is the eventual creation of a single consistent document. Examples of things which versions may represent are: (a) refinements to existing designs, (b) alternative interchangeable
115 designs, or (c) variant designs of components for specific requirements. In addition to providing facilities to create and access versions, version management also provides means for structuring the versions. This includes the maintenance of relationships between versions (such as derivations) and methods to select among component versions to define different configurations of the design artifact. The version management facilities are generally incorporated into the object-oriented data model as new kinds of relationships between objects. In this way, versioning is integrated as part of the design process, rather than being grafted on as a separate activity. There has been extensive research in the representation and management of versions in design environments; a survey of this research is given in [Kat90]. This research has been predominately motivated by the need to provide database support for versions of design objects in computer-aided design (CAD) systems [KCB86, KSW86, AN91, Sci94]. This research acknowledges the fact that the design of a complex system is not a linear process that can be supported by a single evolving representation of the design artifact. Multiple versions of the components of the artifact may be generated and stored during the design process for various reasons. The degree to which the version management support is integrated with the data manipulation and query languages is also an issue. The EXCESSV language [Sci94] is a good example of a tight integration of the language constructs to support versioning features with an existing DBPL. This allows versions to be easily accessed and queried, without separate specifications to select the desired versions. However, since the language constructs for versioning require that the programmer/user be aware of the individual objects which are versioned, they are not easily adaptable to CSCW. The H2O DBPL language constructs, with their emphasis on the representation and manipulation of updates rather than individual object versions, more naturally supports cooperative work, where the emphasis is more on coordinating actions. Configuration Management and Software Engineering: Research to support computer-aided software engineering (CASE) [HPR89, HR92] and configuration management [Dar91, HOS90, AHM89] has
116 investigated how to manage the possible conflicting actions of different designers. While this work is focused on the editing of text files containing program source code, the methodology for detecting and resolving conflicts can be generalized for the support of cooperative work. Early configuration management systems, such as RCS [Tic85], generally relied on a “check-out/check-in” policy, which locks files which are being modified to prevent conflicting updates. To allow for more parallel editing, this policy is modified in some systems, such as the RPDE3 librarian [HOS90] and the Sun Network Software Environment (NSE) [AHM89], to a copy-modifymerge policy. Under copy-modify-merge, users check out copies of files which they can edit independently of the parallel modifications of other users. Once editing is complete, the updated file is then merged with other updated copies. Conflicting updates are handled by a reconciliation activity, which is usually handled by invoking an editor which allows a user to resolve the conflicts. A problem with policies such as check-out/check-in or copy-modifymerge is that it requires that the objects which are to be modified be identified (checked out) in advance. While this is acceptable for supporting file access in a programming environment, it is not desirable in a cooperative work environment where the user’s attention is focused on the task he or she is trying to accomplish, rather than the individual objects which model the environment in which the task takes place. As was suggested by Cort´es [Cor95], many cooperative work environments require a distinction between the objects stored in the database and the objects manipulated in the user interface. This makes the application of techniques requiring explicit object identification at check-out problematic. Program dependence graphs [HPR89, HR92] have also been used to analyze and manage conflict between updates for program source text files. This work automates the process of conflict resolution by utilizing knowledge of application semantics. The conflicting text files are parsed into a semantically meaningful form (program dependence graphs) and these graphs are used to identify and (where possible) resolve the conflicts. After the merge, the graph is then used to construct a new text file. Although the goal of automated merging
117 is similar to the goals outlined in this proposal, the techniques are highly dependent on the application (program text files) and not easily generalizable. The “change sets” used in the commercial software engineering system Aide-De-Camp [Sof89, Dar91] are similar to delta values and can be used for constructing unique versions of software artifacts. These change sets are based on program text and not generalizable as are delta values. Computer Supported Cooperative Work: A general survey of CSCW issues and applications is given in [Bae93]. Issues related to concurrency control and using databases to support groupware applications are enumerated in [GM94, CKNT93]. An example of a concurrency control protocol specifically designed to support groupware is the Gordion object base management system [EE87]. Gordion anticipates the needs of CSCW by providing design oriented concurrency control primitives from which application specific protocols can be built. The concurrency control primitives consist of hard and soft locks on objects. Soft locks can be broken, but the holder of the lock is notified of other accesses, which allows the holder of the lock to both define and enforce the concurrency control policy. More recent work [JMR91, RKT+ 95] attempts to define models in which transactions are replaced by shared processes or activities with flexible and adaptable control protocols. There has been considerable work in the Computer Supported Collaborative Work (CSCW) community on techniques for merging updates and detecting conflict between updates, for example, see [HOS90, KP90]. The flexible object merging framework described by Munson and Dewan [MD94] is very similar to the general Heraclitus framework. Because the delta values of H2O are supported directly by the DBMS, the Coral architecture is more general and flexible. The DCWPL (Describing Collaborative Work Programming Language) project [CM96, Cor95] suggests a separation of tools for artifact manipulation and tools to implement policies that is similar to that suggested by the Coral architecture. The DCWPL work focuses more on the collaboration policy definition, while the current research focuses more on the underlying artifact
118 management. The groupware system definition (GSD) suggested by Cort´es [Cor95] inspired the aspects of the Coral architecture dealing with the separation of data objects and visual objects. A GSD is a tuple < D, V, U, C >, where D = data artifacts, V = visual artifacts, U = users, C = interaction rules. The procedures defined for the Eniwetok operations in Section 4.3 fail into the coordination tasks subcategory of a GSD’s interaction rules. Authoring Systems: The general principles and requirements for authoring systems are well researched [Sha93, Rad96, HW92b]. The particular requirements for collaboration policies in authoring systems are highly volatile. A survey of collaborative authoring systems and a description of the wide spectrum of considerations affecting their design is given in [MR96]. Interestingly, of about 40 systems for collaborative authoring surveyed in that article, only two are listed as addressing the issue of conflict management in the system design. Both of these are focused on editing of hypermedia systems. One of them, CoAUTHOR [HJ+ 89], uses a carefully structured knowledge base to record and access all editing changes. The other, Sepia [HW92a], uses the hypertext storage medium to store multiple versions of hypertext nodes. Further, the version manager can identify clusters of node versions that were created in the context of the same editing session. This closely parallels the notion of delta values supported by Coral. The Mj¨olner authoring system [MM93] was used for the development of the Eniwetok application. Its user interface is centered around a version evolution graph. Unlike the SST, this graph is static once created and cannot be manipulated by users. Mj¨olner also supports “active-diffs” that can be used to support awareness of conflict developing between the editing of concurrent versions. The document model used to support the Reef and Eniwetok applications is directly based on the hierarchical models commonly used in authoring systems such as LINCKS [PL92] and CES [GS88]. While the presentation of change is outside the scope of the current research, the interface of the PREP editor [NCK+ 96] could be used in the context of a Coral application. The PREP differencing algorithm (flexible-diff) is strictly text based, and focused
119 strictly on local textual changes. 5.3
Future Work
Further Database Theory and Techniques: The current research can be extended in a number of ways furthering the current results. • Selective Locking: The current research has focused on the most extreme example of the plan-edit-consolidate work process, in which authors are completely unaware of the activities of other authors during the asynchronous editing cycle. The possibility of using conflict detection agents to reintroduce some collaboration awareness was discussed in Section 4.3. The possibility of introducing selective locking on particular objects also exists. This can be easily accomplished at the H2O implementation level, by adding locks on objects that inhibit the deltafication mechanism (see Figure 2.35). The impact of policies to support selective locks on the general system architecture is yet to be addressed. • Support for Interobject Relationships and Global Constraints: The current research has focused on text documents, with no support for semantic links between the constituent objects of the document. To extend this research beyond this limitation, the development of new delta forms which account for general inter-object relationships is required. In addition to modeling semantic links across text documents, these relationships could also be used to support constraints in other artifact types, such as VLSI designs. The addition of these new delta forms to the H2O system would also require modifications to the deltafication and state materialization mechanisms. A potential mechanism for supporting these modification is the implementation of relationships as active objects [DPW93]. This would allow H2O to identify the relationships potentially affected by updates and thereby include the objects which implement the relationships in the appropriate delta values.
120 • Adding Read Sets for Conflict Analysis: Heraclitus[OO]’s support for conflict analysis could be extended by adding read sets to the delta values. Such read sets would be used semantically in much the same manner as the interpretation of reads in usual database concurrency control policies. There are three issues to be addressed in terms of read sets. The first is how the definition of conflict is changed. This affects the specification of merge operations for the H2O state delta form as well as application defined delta forms. The second is how read sets are stored in relation to delta values. It is unclear if it is most beneficial to store the read sets as part of the object level delta values or the state delta value. Finally, is the issue of how read sets are generated at application run time. In usual concurrency control policies, reads are defined as any object accessed during a transaction by the computation executed by the application program. This is a significant problem in terms of authoring systems and related groupware applications, because the application program (the module labeled “Application Program” in Figure 2.27 or “Object Control” in Figure 4.1) does little, if any, computation. Rather, the computation generally occurs in the minds of the system users, and as a result, the system has no awareness of what is a relevant read in terms of a particular edit or update. This indicates that user studies are required to determine how an author’s conceptions of read objects can be captured in the database. Application of Coral and H2O/SST to Support CSCW Research: It is hoped that the general Coral architecture and H2O/SST keeper module will be used to support more systematic design and research into the development of groupware applications. In this manner it is possible that results and developments can be more easily ported between application areas. • Change and Conflict Presentation: The presentation of change and conflict at the user interface is essential to effect conflict resolution by authors and other potential application users. While H2O/SST is effective in the identification of conflict, it yields no support for
121 presenting that conflict in a manner that is understandable to users. Nevertheless, it may be possible to use the notions of conflict as supported by Heraclitus[OO] to develop new theories for the presentation of conflict. This could lead to the development of the equivalent of the delta form to support application specific presentation semantics in the context of a general system framework. • Collaboration Policy Manager: The current research has shown that the SST interface can be effectively used to define application specific policies and procedures for concurrency. It would be useful to also provide a better framework for the specification of such policies in terms of the concepts (tasks, users, updates, roles, etc.) of particular applications. As such it would provide a general framework for a synchronizer module. A promising route towards such a framework could come from the adaptation of collaboration policies such as DCWPL [CM96] or COACT [RKT+ 95]. By extracting the portions of their specification languages that relate to the data store, and adapting that to interact with the SST, a major step could be taken towards developing a general groupware architecture. • Planning Phase Support: The current research has focused on the later phases of the plan-edit-consolidate work process. The planning phase might also be supported through the insertion of empty delta values in the SST, mapped by the application to the tasks being planned. These planning delta values could also be used to predict where potential conflict might arise during editing, by accessing the objects projected to be changed by the planned updates. New Application Areas: In addition to using H2O/SST and the Coral architecture to support a wider range of applications within the domain of groupware, it may be possible to apply it to similar applications from other domains. One potentially interesting area is Software Engineering. • Software Engineering: The success of the Aide-de-Camp product [Sof89] in using “change-sets” to support software development (as well
122 as configuration management systems such as [HOS90, AHM89]) in much the same manner as delta values are used to support groupware in the current research, indicates that there may be some potential in exploring the use of H2O/SST to support configuration management and software engineering. One environment that might benefit from the direct application of the results of the current research is environments in which application development is driven by customer motivated enhancements to a well defined baseline product. Because such an environment necessitates the development of new versions of the product, without interference from the parallel development of enhancements for other customers, it creates a situation similar to the asynchronous editing phase of document authoring. Similarly, migration of developed enhancements from the resulting customer specific products into the baseline product is similar to the consolidation phase of authoring. Adaptation of the current research would require the development of new delta forms to support existing models of the software artifacts, which are program code. Because of the richer and more syntactically apparent relationship between components of program artifacts, the definition of read sets in that context may be more apparent than in general documents, possibly motivating the exploration of read sets in conflict analysis mentioned above.
BIBLIOGRAPHY [AHM89]
Evan W. Adams, Masahiro Honda, and Terrence C. Miller. Object management in a CASE environment. In Proc. of Eleventh ACM Intl. Conf. on Software Engineering, pages 154–163, 1989.
[AN91]
Rafi Ahmed and Sham B. Navathe. Version management of composite objects in CAD databases. In Proc. ACM SIGMOD Symp. on the Management of Data, pages 218–227, May 1991.
[Bae93]
Ronald M. Baecker, editor. Readings in Groupware and Computer-Supported Cooperative Work. Morgan Kaufmann Publishers, Inc., San Mateo, CA, 1993.
[BDD+95]
O. Boucelma, J. Dalrymple, M. Doherty, J. C. Franchitti, R. Hull, R. King, and G. Zhou. Incorporating Active and Multidatabase-state Services into an OSA-Compliant Interoperability Framework. In The Collected Arcadia Papers, Second Edition. University of California, Irvine, May 1995.
[BDK92]
F. Bancilhon, C. Delobel, and P. Kanellakis, editors. Building an Object-Oriented Database System: The Story of O2 . Morgan Kaufmann, San Mateo, California, 1992.
[BK91]
N. S. Barghouti and G. E. Kaiser. Concurrency control in advanced database applications. ACM Computing Surveys, 23(3):269–317, September 1991.
[BKK85]
Fran¸cois Bancilhon, Won Kim, and Henry F. Korth. A model of CAD transactions. In Proc. of Intl. Conf. on Very Large Data Bases, pages 25–33, 1985.
[BLT86]
J.A. Blakeley, P.-A. Larson, and F.W. Tompa. Efficiently updating materialized views. In Proc. ACM SIGMOD Symp. on the Management of Data, pages 61–71, 1986.
[BR88]
F. Bancilhon and R. Ramakrishnan. An amateur’s introduction to recursive query processing strategies. In M. Stonebraker, editor, Readings in Database Systems, pages 507–555. MorganKaufmann, Inc., 1988.
124 [Cat93]
R.G.G. Cattell. The Object Database Standard: ODMG-93. Morgan Kaufmann Publishers, San Mateo, California, 1993.
[CB97]
R.G.G. Cattell and Douglas K. Barry. The Object Database Standard: ODMG 2.0. Morgan Kaufmann Publishers, San Francisco, California, 1997.
[CDRS86]
M. J. Carey, D. J. DeWitt, Joel E. Richardson, and E. J. Shekita. Object and file management in the EXODUS extensible database system. In Proc. of VLDB, pages 91–100, 1986.
[CKNT93]
Sharma Chakravarthy, Kamalakar Karlapalem, Shamkane B. Navathe, and Asterio Tanaka. Database supported cooperative problem solving. International Journal of Intelligent and Cooperative Information Systems, 2(3):249–287, 1993.
[CM96]
Mauricio Cort´es and Prateek Mishra. DCWPL: A programming language for describing collaborative work. In Proc. of the ACM Conf. on Computer Supported Cooperative Work, 1996.
[Cor95]
Mauricio Cort´es. Control and coordination in groupware applications. Technical Report TR-012-95, Department of Computer Science, State University of New York at Stony Brook, 1995.
[CRGMW96] S. S. Chawathe, A. Rajaraman, H. Garcia-Molina, and J. Widom. Change detection in hierarchically structured information. In Proc. ACM SIGMOD Symp. on the Management of Data, pages 493–504, 1996. [Dar91]
Susan Dart. Concepts in configuration management systems. In Proceedings of the Third International Workshop on Software Configuration Managemant, pages 1–18, June 1991.
[Dat86]
C.J. Date. An Introduction to Database Systems, 4th ed. Addison-Wesley, Reading, Mass., 1986.
[DDD+96]
M. Derr, J. Durand, M. Doherty, R. Hull, and M. Rupawalla. Applications of Heraclitus in telecommunications information processing. International Journal of Engineering Intelligent Systems Special Issue on Databases and Telecommunications, July 1996.
[DHDD95]
M. Doherty, R. Hull, M. Derr, and J. Durand. On detecting conflict between proposed updates. In Proc. of Intl. Workshop on
125 Database Programming Languages. Springer-Verlag, September 1995. [DHR96]
M. Doherty, R. Hull, and M. Rupawalla. Structures for manipulating proposed updates in object-oriented database. In Proc. ACM SIGMOD Symp. on the Management of Data, pages 306– 317, June 1996.
[DPW93]
M. Doherty, J. Peckham, and V. Fay Wolfe. Implementing relationships and constraints in an object-oriented database using a monitor construct. In Rules in Database Systems: Proceedings of the 1st International Workshop on Rules in Database Systems, pages 347–363. Springer-Verlag, 1993.
[EE87]
Aral Ege and Clarence A. Ellis. Design and implementation of GORDION, an object base management system. In Proceedings of the IEEE International Conference on Data Engineering, pages 226–234, 1987.
[EGR91]
C. A. Ellis, S. J. Gibbs, and G. L. Rein. Groupware: Some issues and experience. Communications of the ACM, 34(1):39– 58, 1991.
[Elm92]
Ahmed K. Elmagarmid, editor. Database Transaction Models for Advanced Applications. Morgan Kaufmann Publishers, Inc., San Mateo, California, 1992.
[EW94]
C.A. Ellis and J. Wainer. Goal-based models of collaboration. Collaborative Computing, 1(1):61–86, 1994.
[GHJ96]
S. Ghandeharizadeh, R. Hull, and D. Jacobs. Heraclitus: Elevating deltas to be first-class citizens in a database programming language. ACM Trans. on Database Systems, 21(3):370– 426, 1996.
[GJ94]
S. Gan¸carski and G. Jomier. Managing entity versions within their context: a formal approach. In Proceedings of the Database and Expert Systems Applications Conference, Athens, Greece, 1994.
[GL95]
T. Griffin and L. Libkin. Incremental maintenance of views with duplicates. In Proc. ACM SIGMOD Symp. on the Management of Data, San Jose, California, May 1995.
126 [GM94]
Saul Greenberg and David Marwood. Real time groupware as a distributed system: Concurrency control and its effect on the interface. In Proc. of the ACM Conf. on Computer Supported Cooperative Work, pages 207–217, Chapel Hill, North Carolina, October 1994.
[GS88]
Irene Greif and Sunil Sarin. Data sharing in group work. In Irene Greif, editor, Computer-Supported Cooperative Work: A Book of Readings. Morgan Kaufmann, San Mateo, 1988.
[HJ+ 89]
U. Hahn, M. Jarke, et al. CoAUTHOR: A hypermedia group authoring environment. In Proc. of First European Conf. on Computer Supported Cooperative Work (EC-CSCW’89), pages 226–244, 1989.
[HJ91]
R. Hull and D. Jacobs. Language constructs for programming active databases. In Proc. of Intl. Conf. on Very Large Data Bases, pages 455–468, 1991.
[HM75]
J. W. Hunt and M. D. McIlroy. An algorithm for differential file comparison. Technical Report 41, Bell Laboratories, New Jersey, 1975.
[HOS90]
William H. Harrison, Harold Ossher, and Peter F. Sweeney. Coordinating concurrent development. In CSCW ’90: Proceedings of the Conference on Computer Supported Cooperative Work, pages 157–168, October 1990.
[HPR89]
S. Horwitz, J. Prins, and T. Reps. Integrating non-interfering versions of programs. ACM Transactions on Programming Languages and Systems, 11(3):345–387, July 1989.
[HR92]
S. Horwitz and T. Reps. The use of program dependence graphs in software engineering. In Proc. of Fourteenth ACM Intl. Conf. on Software Engineering, pages 392–411, 1992.
[HW88]
M. P. Herlihy and W. E. Weihl. Hybrid concurrency control for abstract data types. In Proc. ACM Symp. on Principles of Database Systems, pages 201–210, 1988.
[HW92a]
J.M. Haake and B. Wilson. Supporting collaborative writing of hyperdocuments in SEPIA. In Proc. of ACM Conf. on Computer Supported Cooperative Work (CSCW’92), pages 138–146. ACM Press, 1992.
127 [HW92b]
Patrik O’Brian Holt and Noel Williams, editors. Computers and Writing - State of the Art. Kluwer Academic Publishers, Dordrecht, The Netherlands, 1992.
[JE93]
Matthias Jarke and Clarence A. Ellis. Distributed cooperation in integrated information systems. International Journal of Intelligent and Cooperative Information Systems, 2(1):85–103, 1993.
[JMR91]
Matthias Jarke, Carlos Maltzahn, and Thomas Rose. Sharing processes: Team coordination in design repositories. International Journal of Intelligent and Cooperative Information Systems, 1(1):145–167, 1991.
[Jon95]
Steve Jones. Identification and use of guidelines for the design of computer supported collaborative writing tools. Computer Supported Collaborative Writing, 3(3-4):379–404, 1995.
[Kat90]
R. H. Katz. Toward a unified framework for version modeling in engineering databases. ACM Computing Surveys, 22(4):375– 408, December 1990.
[KCB86]
Randy H. Katz, Ellis Chang, and Rajiv Bhateja. Version modeling concepts for computer-aided design databases. In Proc. ACM SIGMOD Symp. on the Management of Data, pages 379– 386, May 1986.
[KP90]
Michael J. Knister and Atul Prakash. DistEdit: A distributed toolkit for supporting multiple group editors. In CSCW ’90: Proceedings of the Conference on Computer Supported Cooperative Work, pages 343–355, October 1990.
[KSW86]
Peter Klahold, Gunter Schlageter, and Wolfgang Wilkes. A general model for version management in databases. In Proc. of Intl. Conf. on Very Large Data Bases, pages 319–327, August 1986.
[KV93]
Gabriel M. Kuper and Moshe Y. Vardi. The logical data model. ACM Trans. on Database Systems, 18(3):379–413, 1993.
[LSP93]
P. Lambrix, M. Sj¨olin, and L. Pagdham. LINCKS – a platform for cooperative information systems, 1993. unpublished manuscript, available at http://www.ida.liu.se/∼lincks/papers.html.
128 [MD94]
J. P. Munson and P. Dewan. A flexible object merging framework. In Proc. of the ACM Conf. on Computer Supported Cooperative Work, pages 231–242, October 1994.
[MG94]
Keith McAlpine and Paul Golder. A new architecture for a collaborative authoring system. Collaborwriter. Computer Supported Cooperative Work, 2(3):159–174, 1994.
[MM93]
Sten Min¨or and Boris Magnusson. A model for semi(a)synchronous collaborative editing. In Proceedings of the European Conference on Computer-Supported Cooperative Work, pages 219–231, Milan, Italy, September 1993.
[MR96]
Antonios Michailidis and Roy Rada. A review of collaborative authoring tools. In Roy Rada, editor, Groupware and Authoring, pages 9–43. Academic Press, San Diego, CA, 1996.
[Mye86]
E. W. Myers. An O(ND) difference algorithm and its variations. Algorithmica, 1:251–266, 1986.
[NCK+ 96]
C. M. Neuwirth, R. Chandhok, D. S. Kaufer, P. Erion, J. Morris, and D. Miller. Flexible diff-ing in a collaborative writing system. In Roy Rada, editor, Groupware and Authoring, pages 189–204. Academic Press, San Diego, CA, 1996.
[PKH88]
Calton Pu, Gail E. Kaiser, and Norman Hutchinson. Splittransactions for open-ended activities. In Proc. of Intl. Conf. on Very Large Data Bases, pages 26–37, 1988.
[PL92]
L. Padgham and J. L¨owgren. User interface management for an object oriented database system. In Proceedings of the First International Conference on Information and Knowledge Management, Baltimore, Maryland, 1992.
[RAA94]
R. F. Resende, D. Agrawal, and A. El Abbadi. Semantic locking in object-oriented database systems. In Proc. ACM Conf. on Object-Oriented Programming Systems, Languages, and Applications, pages 388–402, 1994.
[Rad96]
Roy Rada, editor. Groupware and Authoring. Academic Press, San Diego, CA, 1996.
[RKT+ 95]
M. Rusinkiewicz, W. Klas, T. Tesch, J. W¨asch, and P. Muth. Towards a cooperative transaction model – the cooperative activity
129 model. Technical report, German Research Center for Computer Science, March 1995. GMD Technical Report No. 899. [Sch88]
David A. Schmidt. Denotational Semantics: A Methodology for Language Development. Wm. C. Brown Publishers, Dubuque, Iowa, 1988.
[Sci94]
Edward Sciore. Versioning and configuration management in an object-oriented data model. The Journal of Very Large Data Bases, 3(1):77–106, January 1994.
[SGB+ 93]
M. Sharples, J. S. Goodlet, E. E. Beck, C. C. Wood, S. M. Easterbrook, and L. Plowman. Research issues in the study of computer supported collaborative writing. In Mike Sharples, editor, Computer Supported Collaborative Writing, pages 9–28. Springer-Verlag, London, 1993.
[Sha93]
Mike Sharples, editor. Computer Supported Collaborative Writing. Springer-Verlag, London, 1993.
[SL76]
D.G. Severance and G.M. Lohman. Differential files: Their application to the maintenance of large databases. ACM Trans. on Database Systems, 1(3), September 1976.
[Sof89]
Software Maintenance & Development Systems, Inc., Concord, MA. Aide-De-Camp Software Management System, Product Overview, 1989.
[Str91]
Bjarne Stroustrup, editor. The C++ Programming Language. Addison-Wesley Publishing Company, Reading, Massachusetts, 1991.
[Tic85]
W. F. Tichy. RCS: A system for version control. Software: Practice & Expererience, 15(7):637–654, July 1985.
[Ull82]
Jeffrey D. Ullman. Principles of Database Systems (2nd edition). Computer Science Press, Potomac, Maryland, 1982.
[Ver97a]
Versant Object Technology. Versant ODBMS 5.0 C++ Reference Manual. Menlo Park, CA, June 1997.
[Ver97b]
Versant Object Technology. Versant ODBMS 5.0 Concepts and Usage. Menlo Park, CA, June 1997.
130 [vR95]
Guido van Rossum. Python reference manual, 1995. available by anonymous ftp at ftp.python.org or by world-wide web at http://www.python.org.
[WC95]
J. Widom and S. Ceri. Active Database Systems: Triggers and Rules for Advanced Database Processing. Morgan-Kaufmann, Inc., San Francisco, California, 1995.
[ZM90a]
S. Zdonik and D. Maier. Fundamentals of object-oriented databases. In S. Zdonik and D. Maier, editors, Readings in Object-Oriented Database Systems, chapter 1, pages 1–32. Morgan-Kaufmann Publishers, Inc., San Mateo, CA, 1990.
[ZM90b]
S. Zdonik and D. Maier. Readings in Object-Oriented Database Systems. Morgan-Kaufmann Publishers, Inc., San Mateo, CA, 1990.
[ZS89]
K. Zhang and D. Shasha. Simple fast algorithms for editing distance between trees and related problems. SIAM Journal of Computing, 18(6):1245–1262, 1989.
APPENDIX A A DELTA FORM FOR SEQUENCES This appendix gives an overview of a delta form for a sequence data type, which can be used to refine the document model given in Section 4.4. This sequence delta form is supported in the current H2O implementation and is used in the implementation of the Reef prototype. A sequence is an ordered lists of elements with relative positioning operations. This makes the sequence different from other kinds of lists, such as arrays, in which the positioning is absolute. For example, when an element is removed from a sequence, all other elements maintain their respective relative positions, without leaving gaps in the list. The sequence data type is defined by the ODL class specification given in Figure A.1. As defined here, the sequence operates on object identifiers (oids), but the results presented here are equally applicable to sequences of values. A sequence value will be written [om , . . . , on ], where om , . . . , on are the oids in the sequence. A sequence may also contain phantom oids, which are used as place holders for oids that have been moved or removed. Phantom oids will be indicated by a superscript p; op34 would be a phantom oid used to hold the former location of oid o34 . The phantom oids are required for the operation of the sequence delta form, as will be shown. These phantom oids are system generated values and are not apparent or available to application programs. Where they appear in sequence values, they must be removed by the system in the application’s apparent view of the sequence. Thus the sequence [o7 , op23 , o19 ] would appear to the application as [o7 , o19 ]. (This hiding of phantom oids can be handled in the implementation of the method element at, defined below.) The methods which change the value of a sequence consist of various operations to move, insert and remove elements from the sequence. The values in the sequence can be retrieved by the method element at which returns the
132 interface Sequence { boolean move_to_head(in Ref move_oid); boolean move_to_tail(in Ref move_oid); boolean move_before(in Ref move_oid, in Ref target_oid); boolean move_after(in Ref move_oid, in Ref target_oid); boolean insert_at_head(in Ref ins_oid); boolean insert_at_tail(in Ref ins_oid); boolean insert_before(in Ref ins_oid, in Ref target_oid); boolean insert_after(in Ref ins_oid, in Ref target_oid); boolean remove(in Ref del_oid); Ref element_at(in integer position); } Figure A.1: Class Specification for Sequence Data Type. value at any position in the sequence. The method remove removes an element from the sequence, leaving the rest of the elements in the same relative order. The move and insert methods are all defined relative to some element already existing in the sequence or to the head or tail of the sequence. That position or element is referred to as the target of the operation. For example, the method move to head moves the element identified by the parameter move oid to the head of the sequence, thus the head of the sequence if the target of the operation. Similarly, the method insert before adds the element identified by the parameter ins oid to the sequence and positions it before the target element, identified by the parameter target oid. The methods that modify the sequence value all return a boolean value indicating whether the operation was executable. The operation could fail to be executable if the target is not present in the sequence, if an element to be moved or removed is not present, or if an element to be inserted is already present in the sequence. For the discussion in this section, it is sufficient to assume that the operations do not modify the sequence if these preconditions are not met and the return value can be ignored. The targets are critical in the definition of the delta form for sequences. In the delta form, the targets will translate to anchors in subsequences called islands. It is in this way that the intended relative positions are captured
133 Dseq = D, I, P where D is a set of oids to be deleted, I is a set of oids to be inserted, and P is a set of anchored subsequences (islands) indicating the relative positions of the oids. Figure A.2: Delta Values for Sequences. in the delta values. The domain of delta values for the sequence delta form is given in Figure A.2. A sequence delta value consists of D, a set of oids to be deleted or removed, I, a set of oids to be inserted and P a set of islands that define a subsequence relative to some anchor which might be an oid or the head or the tail of the list. Each island has the form [o1 , . . . , om , | a |, om+1 , . . . , on ] where the | . | bracketed oid is the anchor of the island. The symbol ∧ will be used for the sequence head and the symbol ∨ will be used for the sequence tail. For example, the island [| ∧ |, o23 , o92 ] indicates that the subsequence [o23 , o92 ] should be positioned after the sequence head. As with sequence values, the oids in an island may include phantom oids. Definition: A proper sequence delta value must satisfy the following properties: • There can be at most one island in P anchored at the sequence head and at most one island in P anchored at the sequence tail. • If an island is anchored at the sequence head (tail) there can be no oids before (after) the anchor in that island. • If o ∈ D, then o ∈ / I and o cannot appear in any island in P. • Each o ∈ I must appear as a non-anchor in exactly one island in P. • An oid can appear in at most one island in P. The last requirement on proper sequence delta values is important because it means that the oids referenced by each island are disjoint. If instead of viewing a sequence as a specialized list, it is viewed as a set with an ordering on the elements, then the set of islands in a proper sequence delta value defines a partially specified sequence. A sequence value is a fully specified sequence because it defines an ordering for all elements, but a set of islands is a partially
134 [move to head(o)]seq =< {}, {}, {[| ∧ |, o]} > [move to tail(o)]seq =< {}, {}, {[o, | ∨ |]} > [move bef ore(o1 , o2 )]seq =< {}, {}, {[o1, | o2 |]} > [move af ter(o1 , o2 )]seq =< {}, {}, {[| o2 |, o1 ] >} > [insert at head(o)]seq =< {}, {o1}, {[| ∧ |, o]} > [insert at tail(o)]seq =< {}, {o1 }, {[o, | ∨ |]} > [insert bef ore(o1 , o2 )]seq =< {}, {o1 }, {[o1 , | o2 |]} > [insert af ter(o1 , o2 )]seq =< {}, {o1 }, {[| o2 |, o1] >} > [remove(o)]seq =< {o}, {}, {} > [element at(i)]seq = null Figure A.3: Deltafication Operation for Sequence Objects. specified sequence because it defines an ordering only over some subsequences. The deltafication operation for the sequence delta form is given in Figure A.3. The method remove creates a delta value with the oid that was removed in the set D, and the insert methods create delta values with the oid that was inserted in the set I. The move methods and the insert methods also place an island in the set P that captures the position of the moved or inserted oid relative to the specified target. The deltafication operation produces a proper delta value on a sequence, which can be easily verified by inspection of Figure A.3. The key to the specification of the apply and smash operations for the sequence delta form is the relationship between the sequence value and the set of positioning islands. Any sequence value can be viewed as an island anchored at the sequence head. For example, the sequence [o87 , o4 , o21 , o77 ] is equivalent to the island [| ∧ |, o87 , o4 , o21 , o77 ]. Ignoring the sets of deleted and inserted oids for the moment, the apply and smash operations can be defined as similar operations, where smash operates on a set of islands, and apply operates on a single island anchored at the sequence head. For another perspective, a set of islands defines a partially specified sequence, and smash has to do the same operation on the partially specified sequence that apply does on a fully specified sequence. The apply operation for the sequence delta form, applyseq , is defined
135 Algorithm Apply(S,∆): Input: S, a sequence object; ∆, a valid delta on sequences Output: A sequence with value applyseq (∆, S) begin (1) if ∆ = fail (2) return error; (3) let D, I, P = ∆; (4) for each o ∈ D if o is in S, replace it with op ; (5) for each o ∈ I if o is not in S, append o to S; (6) for each [. . . , | a |, . . .] ∈ P (7) if a is a phantom oid with value op and o ∈ / D and o is in S p (8) replace o with o (in S); (9) append o to S; (10) for each [o1 , . . . , om , | a |, om+1 , . . . , on ] ∈ P (11) if a is in S (12) for j = 1, m (13) if oj is in S (14) move oj immediately before a in S; (15) for j = n, m − 1 (16) if oj is in S (17) move oj immediately after a in S; (18) return S; end Figure A.4: Apply Operation for Sequence Delta Form. by the algorithm given in Figure A.4. At line (4), all oids which are in the deletion set of the delta value are replaced by phantom oids. This essentially deletes the oids, leaving a placeholder in case that oid is used for relative positioning in some future delta value application. In the following line, all oids in the insertion set of the delta value are appended to the sequence. Note that by the fourth property of proper sequence delta values, all these oids must appear as non-anchors in the positioning set of the delta value, and will be moved to their correct position later in the algorithm. In the next seqment, lines (6) through (9), all islands in the delta value’s positioning set are checked for phantom anchors. If a phantom anchor exists and that oid is not in the deletion set, then that phantom must have arisen from a repositioning during a smash operation (as defined below). To handle this, the oid in the sequence is
136 replaced with the phantom, and the oid is moved to the end of the sequence to be repositioned later as if it were a new oid. Finally, in lines (10) through (17), the subsequence called for by each island in the positioning set is performed on the sequence. If the anchor for any island is not present in the sequence, that island’s subsequence is not performed. Also, if any non-anchor is not present in the sequence, it is omitted form the resulting subsequence. Note that since each island is disjoint, there is no possibility of nondeterministic results due to the order in which the individual islands are processed. The smash operation for the sequence delta form, smashseq , is given in in Figure A.5. This algorithm uses two auxiliary functions which are given in Figure A.6. At lines (5) and (6), the deletion and insertion sets for the delta value are computed. This computation has exactly the same semantics as the smash for the set delta form given in Section 2.2. In lines (8) and (9), all oids that appear in the new delta value’s deletion set are replaced by temporary phantoms in the positioning sets that are to be smashed. These temporary phantoms, indicated with a superscript q will later be removed from the resultant positioning set at lines (23) and (24). Lines (10) through (15) prepare copies of the positioning sets from the delta values to ensure that the anchor of each island in the second set will properly find an island in the first set. Finally, in lines (16) through (22), the Cut and Paste algorithms given in Figure A.6 are used to apply each of the islands from the second set to the islands in the first set. Note that this cutting and pasting against the set of subsequences represented by the first set of islands is the same as the repositioning done on a full sequence in lines (10) through (17) of the Apply algorithm given in Figure A.4. The following claims are made without proof: • The set of proper sequence delta values is closed under this smash operation. • Requirements 2.1 and 2.2 are satisfied by the deltafication, apply and smash operations given here. This claims can be verified as follows. If a sequence is viewed as a set of elementss and an ordering on those elements, then a sequence delta value can
137
Algorithm Smash(∆1 ,∆2 ): Input: ∆1 , ∆2 , two valid deltas on sequences Output: ∆3 , where ∆3 = smashseq (∆1 , ∆2 ) begin (1) if ∆1 = fail or ∆2 = fail (2) return fail; (3) let D1 , I1 , P1 = ∆1 and D2 , I2 , P2 = ∆2 ; (4) ∆3 = D3 , I3 , P3 ; (5) D3 = D2 ∪ (D1 − {o | o ∈ D1 , o ∈ I2 }); (6) I3 = I2 ∪ (I1 − {o | o ∈ I1 , o ∈ D2 }); (7) P3 = P1 ; P3 = P2 ; (8) for each o ∈ D3 (9) replace each occurrence of o in P3 and each occurrence of op in P3 by oq ; (10) for each i = [. . . , | a |, . . .] ∈ P3 (11) if a never appears in P3 (12) if ap does appear in P3 (13) replace a by ap in i; (14) else (15) P3 = P3 ∪ {[| a |]}; (16) for each i = [o1 , . . . , om , | a |, . . . , om+1 , . . . , on ] ∈ P3 (17) for j = 1, m (18) Cut(P3 ,oj ); (19) Paste(P3 ,oj ,a,before); (20) for j = n, m + 1 (21) Cut(P3 ,oj ); (22) Paste(P3 ,oj ,a,after); (23) for each o ∈ D3 (24) cut(P3 ,oq ); (25) return ∆3 ; end Figure A.5: Smash Operation for Sequence Delta Form.
138
Algorithm Cut(P ,o): Input: P , a set of positioning islands; o, an oid to be extracted from P Output: P , a set of positioning islands, reflecting P with o removed begin (1) P = P ; (2) if there is an island in P with o as a anchor (3) replace o by op in that island; (4) for any island in P where o is a non-anchor (5) remove o from that island; (6) delete any islands in P that consist of only an anchor; (7) return P ; end Algorithm Paste(P ,o,l,where): Input: P , a set of positioning islands; o, an oid to be inserted in P ; l, the oid of relative locator; where = before or after Output: P , a set of positioning islands, reflecting P with o inserted begin (1) P = P ; (2) if there is an island i ∈ P with l in it (3) if where = before (4) put o in i, immediately before l; (5) else (6) put o in i, immediately after l; (7) else (8) if where = before (9) P = P ∪ {[o, | l |]}; (10) else (11) P = P ∪ {[| l |, o]}; (12) return P ; end Figure A.6. Positioning Set Modification Algorithms for Sequence Delta Form.
139 be viewed as a set delta value (stored in the deletion and insertion sets D and I) and a partial order (stored in the positioning set P). The properties related to insertion and deletion of elements can be verified in a manner analogous to the proofs given for the set delta value in Section 2.2. This leaves verification of the properties related to positioning. The positioning properties hold because the apply and smash operations perform essentially the same action, where apply realizes the partial orders called for by a delta value in the context of the full order defined by a sequence and smash realizes the partial order called for by a delta value in the context of another partial order called for by the other delta value. Let S0 be some sequence value and let ∆1 and ∆2 be two sequence delta values. Also let S1 = applyseq (∆1 , S0 ), S2 = applyseq (∆2 , S1 ), ∆3 = smashseq (∆1 , ∆2 ) and S3 = applyseq (∆3 , S0 ). Suppose p1 is an island appearing in the positioning set of ∆1 and suppose that p2 is an island appearing in the positioning set of ∆2 such that the anchor of p2 appears in p1 . In this case, the subsequence defined by p1 will be present in sequence S1 and then the subsequence defined by p2 will be realized in sequence S2 relative to the subsequence created due to p1 in S1 . If the delta values are first smashed them applied, an inland will appear in the smashed delta value created by realizing the subsequence called for by p2 in the subsequence defined by p1 . When applied to the sequence S0 , this will result in the same subsequence in S3 that appears in S2 . A proper proof of these properties would require addressing a number of details, such as the interaction between islands that have more intricate overlaps between the elements and the complications caused by the phantoms due to deletions. A particular merge operation has not been defined for sequence delta values in the current implementation of H2O. The default definition of merge is used, as given in Section 2.2.
