Flexible Merging for Asynchronous Collaborative ... - Semantic Scholar

Flexible Merging for Asynchronous Collaborative Systems Haifeng Shen and Chengzheng Sun School of Computing and Information Technology Griffith University Brisbane, Qld 4111, Australia {Hf.Shen, C.Sun}@cit.gu.edu.au

Abstract. Version control systems are widely used asynchronous collaborative systems in team-working environments, where document merging is a key function. However most existing systems only support limited semantic merging, and techniques for supporting semantic merging are strictly bound up with the merging algorithms that do syntactic merging. In this paper, we propose a flexible merging framework in which semantic merging policies are separated from the syntactic merging mechanism for asynchronous collaborative systems. In this framework, semantic merging policies are not restricted by the merging algorithms used in the syntactic merging mechanism, and the syntactic merging mechanism is flexible to support a wide range of semantic merging policies. This framework can be used to describe and compare a range of existing merging policies and mechanisms, and to guide the design of new merging policies and mechanisms. The proposed framework has been applied to the design of a flexible merging component in FORCE (Flexible Operation-based Revision Control Environment) prototype, which uses a single syntactic merging mechanism to support a range of semantic merging policies. Keywords: Collaborative system, version control system, document merging, operational transformation

1

Introduction

It is very common that multiple developers are involved in developing a large software project. These developers may be dispersed in different offices or even scattered geographically over the world, but they need to simultaneously modify the code of the shared project. Version control systems are therefore widely used nowadays to facilitate collaborative work in team-working environments. The configuration of a version control system can be illustrated by Figure 1: a repository server maintains all evolving versions of a software project and developers have network connections with the repository server. A version control system is a typical asynchronous collaborative system, which works in the manner of Copy-Modify-Merge [5]: each developer checks out a separate working copy of R. Meersman, Z. Tari (Eds.): CoopIS/DOA/ODBASE 2002, LNCS 2519, pp. 304–321, 2002. c Springer-Verlag Berlin Heidelberg 2002

Flexible Merging for Asynchronous Collaborative Systems

305

the original copy from the repository; then modifies her/his working copy independently; and finally merges her/his working copy with other copies. Merging is the process of integrating multiple documents to generate a new document, which is an essential function in asynchronous collaborative systems. As shown

Original copy and versions

Server Working Copy 1

Working Copy 2

Network Site 2

Site 1

Fig. 1. The configuration of a version control system

in Figure 2, merging may happen at two stages. One is at the committing stage 1 when a working copy (Wm ) is committed into the repository to form a new ver1 1 sion R1 = Wm . At this stage, updates made between W01 and Wm are merged into its original copy R0 in the repository. The other is at the updating stage when a working copy Wn2 is updated by a committed working copy (or version) R1 from the repository. At this stage, updates made between R0 and R1 are 2 . merged into Wn2 to generate a new document state Wn+1

[O W

1

11,...,O 1m

] W

0

1 m

Version control command

Edit operation commit (merging happens)

checkout R0

Site 1

R1

Repository server update (merging happens)

checkout W 20

[O

21,...,O 2n]

W 2n

W 2 n+1

Site 2

Fig. 2. Merging may happen at two stages

Merging at the committing stage can be easily achieved by re-applying those 1 ) to updates made in the working copy (i.e., updates made between W01 and Wm 1 its original copy R0 in the repository because R0 = W0 on which those updates were generated. In contrast, merging at the updating stage is far more difficult. Updates made in a committed working copy (i.e., updates made between R0

306

H. Shen and C. Sun

and R1 ) cannot be simply re-applied in Wn2 because Wn2 = R0 on which those updates were generated. Because of the difficulties involved in merging at the updating stage, most existing systems only support limited semantic merging (i.e., semantically merging two documents), and techniques for supporting semantic merging are strictly bound up with the merging algorithms that do syntactic merging ((i.e., syntactically merging two documents). Those systems usually adopt ad hoc techniques to glue semantic merging and syntactic merging together. For example, in RCS (Revision Control System) [14] and CVS (Concurrent Versions System) [3], the semantic merging policy defines that concurrent updates made within the same line are semantically conflicting while concurrent updates made in different lines are not. This policy, although simple, is very restrictive. On one hand, concurrent updates made within the same line do not necessarily imply they must conflict. For example, suppose a document contains a line “You are student”. A user inserts an article “a” between “are” and “student” to fix a grammar error while another user appends a full stop mark “.” to the line to make it a complete sentence. Although these two updates are made within the same line, they are not semantically conflicting at all. On the other hand, concurrent updates made in different lines do not necessarily imply they would not be conflicting either. For example, suppose a document contains two lines: “You are” and “student.”. A user appends an article “a” to the first line while another user inserts character “s” between “t” and “.” in the second line to make “student” plural. Although these two updates are made in different lines, they are semantically conflicting. The main reason for adopting this kind of semantic merging policy is due to the limitation of the syntactic merging algorithm diff3 [9], which is unable to merge concurrent updates made within the same line. In this paper, we propose a flexible merging framework in which semantic merging policies are separated from the syntactic merging mechanism for asynchronous collaborative systems. In this framework, semantic merging policies are not restricted by the merging algorithms used in the syntactic merging mechanism, and the syntactic merging mechanism is flexible to support a wide range of semantic merging policies. This framework can be used to describe and compare a range of existing merging policies and mechanisms, and to guide the design of new merging policies and mechanisms. We will show how to apply the proposed framework to the design of a flexible merging component in FORCE (Flexible Operation-based Revision Control Environment) prototype, which uses a single syntactic merging mechanism to support a range of semantic merging policies. The rest of the paper is organized as follows. After the introduction, a flexible merging framework is described. Then the next two sections systematically present a flexible merging component for a version control system, which uses a single syntactic merging mechanism to support multiple semantic merging policies. Finally the paper is concluded with a summary of our major contributions and future work.


2

307

A Flexible Merging Framework

As shown in Figure 3, in the proposed merging framework, semantic merging and syntactic merging are separated in two components: Policy component and Mechanism component.

Making policies Semantic merging

Executing merging Syntactic merging

repository

documents

Policy Component

Mechanism Component

logs

Fig. 3. A flexible merging framework

2.1

The Policy Component

The policy component makes various semantic merging policies. These policies are application-dependent. For example, if the shared document is an article, the semantic merging policies could specify a set of spell and grammar checking rules, and possibly the roles of participants. If the shared document is a computer program, the semantic policies could specify the programming language’s syntax parsing rules, and possibly how the work is divided among those participants. Our observation is if the work is properly divided among participants in the way that different participants play different roles, it is possible that their concurrent updates made to the shared document do not semantically conflict with each other, and therefore a new document could be generated to integrate all updates made by different participants. For example, two authors are jointly revising a scientific paper, one correcting spell/grammar errors while the other adding references. Their concurrent updates made to the paper could be as close as within the same line. But these updates would not cause any semantic conflict. A semantic merging policy is specified as a set of Semantic Merging Rules (SMR) with which a function Semantic-Conflict (SMR, Or , Ol , CT) determines whether two concurrent updates Or and Ol are semantically conflicting according to the context CT on which they were generated. On one extreme, the function could automatically return true/false without human intervention if the SMR

308

H. Shen and C. Sun

has been well formulated. For instance, Or and Ol were made concurrently by two users, both aiming to make the sentence “You are student.” grammatically correct. Or inserted an article “a” between “are” and “student” while Ol made “student” plural by appending character “s” between “t” and “.”. If the SMR specifies the grammar checking rules, Semantic-Conflict (SMR, Or , Ol , “You are student.”) can automatically return true because reapplying Or /Ol to the other copy leads to the result “You are a students.”, which is incorrect according to the SMR. Automatic detection of conflicts is the most desirable way. However, it is not easy to achieve. On the other extreme, the detection of conflicts could be completely manual if the SMR is unable to be formulated in any way. As a result, it is completely up to the user to determine what updates made by another user should be merged into her/his working copy, possibly with consultation with that user. Manual detection of conflicts is the most general way for all applications, although it is the lest desirable way. In most cases, the detection of conflicts is the combination of automatic and manual detections. In other words, some conflicts can be automatically detected while others have to be detected by human. The formulation of the SMR is non-trivial, particularly for those semantic rich applications. Therefore the formulation of the SMR itself deserves a lot of research on it. In the FORCE (Flexible Operation-based Revision Control Environment) prototype, some simple semantic merging policies based on which the SMR is easy to be formulated are used just for demonstrating the feasibility of separating semantic and syntactic merging. Our focus in this paper is to devise a generic and flexible syntactic merging mechanism, which is able to support a range of semantic merging policies provided the SMR has been formulated and the Semantic-Conflict function has been defined. 2.2

The Mechanism Component

The mechanism component executes syntactic merging according to the SMR specified in the policy component. A remote update will be applied to a local working copy only if it does not semantically conflict with any update performed by the local user according to the specified SMR. The mechanism component also includes some data structures: repository for storing versions, logs for storing updates, and documents of working copies. These data structures are needed for the mechanism component to execute syntactic merging. What semantic merging policies can be supported by the syntactic merging mechanism strictly relies on how the syntactic merging mechanism is designed. As explained before, if the syntactic merging mechanism is unable to merge concurrent updates made within the same line, the semantic merging policy cannot support integrating concurrent updates made within the same line. Most existing asynchronous collaborative systems, such as RCS [14], CVS [3], ClearCase [1], and Subversion [4] adopt the state-based merging approach as the syntactic merging mechanism. State-based merging [8], as the name implies, is performed by comparing different documents to generate deltas against the base document (one of those


309

documents), and then applying the deltas on the base document to generate a new document. State-based merging is mainly used to merge text documents. A representative is the diff3 [9] merging algorithm, which is used to merge updates made between the original copy and a developed working copy into another concurrently developed working copy. However, the merging algorithm may not be able to merge all updates made in one developed working copy into another although there is no semantic conflict at all. The main reason is derived deltas are coarse-grained. Deltas are coarse-grained in the sense that they are line-based, which means if there is a single update in a line, the entire line is regarded as updated. Consequently the deltas would be the deletion of the old line, and then the insertion of the new line. Deltas could even be block-based, which means if every line in a block (i.e., several consecutive lines) contains an update, the deltas would be the deletion of the old block, and then the insertion of the new block. We refer to the block as an edit block, which contains a single line or several consecutive lines in a document. As a result, the effect region of a delta must start at the beginning of a line and contain one or several consecutive lines. In state-based merging, if an edit block in a working copy is overlapping with an edit block in another copy, updates in the edit block made in that copy are not able to be merged into the other copy. An typical example is an update made within a line in one working copy cannot be merged into another working copy in which an concurrent update has been made within the same line. As an alternative to state-based merging, operation-based merging [8], saves operations (updates) on a document into a log as the deltas between the current state and the initial state of that document. Operation-based merging is done by re-executing operations performed in one working copy on another copy. For example, consider the configuration in Figure 2. If R0 = W01 = W02 contains “You are student”. Site 1 performs an operation O11 to insert a space and an article “a” between “are” and “student”, and then W11 = “You are a student”. Therefore the delta between W11 and W01 is O11 . Site 2 performs an operation O21 to append a full stop mark to the line, and then W12 = “You are student.”. Therefore the delta between W12 and W02 is O21 . To merge the update made at Site 1 into Site 2’s working copy whose current document state is W12 , simply re-executing O11 on W12 would do, and the new document state of Site 2 ’s working copy would become W22 = “You are a student.”, which has integrated the effects of both O11 and O21 . Operations are far more fine-grained than the deltas generated in state-based merging because the effect region of an operation could start at any position and contain arbitrary number of characters in a document. There are two types of primitive editing operations insert and delete. The basic notation of an editing operation is ins/del[P , L, S], representing inserting/deleting string S whose length is L at position P in a document. The fine-granularity of operations makes operation-based merging capable of syntactically merging all concurrent updates made to a shared document by different users. We therefore adopt operationbased merging as the syntactic merging mechanism for the proposed merging

310

H. Shen and C. Sun

framework. Moreover, operation-based merging could support more document types in addition to text documents. For example, operation-based merging was used in the GINA application framework [2] to merge command objects performed in an multi-user application at one site into another. The Bayou infrastructure [6] used operation-based merging to merge operations performed in one copy of a shared database into another. 2.3

Issues of Operation-Based Merging

Operation-based merging has its own challenging issues. One major issue is how to control the size of logs that store saved operations. In a distributed teamworking environment, each participant works independently on his own working copy and her/his independent work tends to be very long. With executed operations accumulated in a log, the log could grow very huge, resulting in a poor system response because the larger a log is, the longer transferring it over the network plus executing operations within it takes. This issue was raised in [6,10], but few solutions were proposed. We have devised the technique of operational merging [11] by which operations whose effect regions are overlapping or adjacent can be merged. As a result, a log can be compressed in the way that its size and the number of operations within it are reduced. With the proposed CALOM (Compress A Log by Operational Merging) control algorithm [11], a log can be fully compressed in the sense that given any two operations in the log, their effect regions are neither overlapping nor adjacent. Moreover, the compressed log is equivalent to its original log in the sense that the compressed log has the exact effect as the original one, which guarantees the correctness of the compression. Our focus in this paper is to tackle another challenging issue, that is how to devise operation-based merging protocols and algorithms for the syntactic merging mechanism, which is able to support a range of semantic merging policies. As explained in the previous section, merging at the committing stage is easy. The problem is merging at the updating stage. Firstly, some operations performed in one working copy may not be able to be re-executed in another due to semantic conflicts. Secondly, if an operation can be re-executed, it may not be able to be re-executed as is because the current document state of a working copy is different from the one on which it was defined. For example, suppose a document contains a line “You are student”. A user performs an operation O1 = ins[7, 2, “ a”] to insert a space and an article “a” at position 7 (i.e., between “are” and “student”) while another user performs an operation O2 = ins[15, 1, “.”] to insert a full stop mark “.” after “student”. O1 and O2 are not semantically conflicting, but re-executing O2 in the other copy would produce the result “You are a stude.nt”, which is obviously wrong. Finally, as shown in Figure 2, oper1 ations O11 , · · ·, O1m represent the deltas between Wm and its original copy R0 while operations O21 , · · ·, O2n represent the deltas between Wn2 and its original copy R0 . When Site 2 updates her/his working copy with version R1 to generate 2 2 a new document state Wn+1 , what are the deltas between Wn+1 and its original copy R1 ?


311

It is non-trivial to solve these issues. The IceCube approach [7] attempted to find an ordering of operations when doing merging to minimize conflicts in terms of application semantics and user intentions. The Bayou infrastructure [6] tried to achieve correct merging with a set of ordering and closure constraints on the propagation of the operations made to a shared and replicated database. These solutions are ad hoc and strictly glue semantic merging and syntactic merging together. Therefore they are unsuitable to be adopted as the syntactic merging mechanism to support a range of semantic merging policies. In the following section, we will systematically address these issues and present merging protocols that have been implemented in the FORCE (Flexible Operation-based Revision Control Environment) prototype.

3

Merging Protocols and Algorithms

At the very beginning, the full state of a document is imported into the repository and marked as the initial version V.0. To collaboratively edit the document, every participant needs to check out a separate copy of V.0 as her/his working copy, and the version number 0 as her/his base version number. The checkout protocol will be presented later in this section. Then s/he starts editing her/his own working copy with editing operations continuously accumulated in a log sequentially. 3.1

Commit a Working Copy

To commit a working copy into the repository, the OMCS (Operation-based Merging at the Committing Stage) protocol is devised as follows. Protocol 1. OMCS Suppose the user at Site k whose base version number is bv and log is LL, wants to commit her/his working copy into the repository: 1. Site k sends a committing request to the repository server, containing the base version number bv. 2. Upon receiving the request, if the repository has been locked because someone is updating it, the repository server puts the request into the waiting queue. When the lock is released and it is time to respond Site k’s committing request: – If the latest version V.lv in the repository is V.bv, the repository server locks the repository and sends a consentient reply to Site k. – Otherwise, the repository sends a rejective reply to Site k. 3. Upon receiving the reply from the repository server, if it is rejective, Site k warns the user to update her/his working copy before committing. If it is consentient: – Site k sends all operations in LL to the repository server, and then empties the log LL. – Site k updates the base version number to bv+1.

312

H. Shen and C. Sun

4. Upon receiving operations from Site k, the repository server executes these operations sequentially on V.bv to generate the full state of the latest version V.lt = V.bv+1, and V.bv is then replaced with those operations, which are the deltas between V.bv+1 and V.bv. After then, the lock on the repository is released. 3.2

Update a Working Copy

Suppose two users User 1 and User 2 have concurrently developed their own working copies from the same base version. To merge updates made in one working copy into the other, some essential issues need to be discussed. Firstly, some operations performed by User 1 may not be able to be reexecuted in User 2’s working copy because they are semantically conflicting with some operations performed by User 2. As explained in previous section, the Semantic-Conflict(SMR, Or , Ol , CT) function can determine whether Or and Ol are semantically conflicting according to the SMR. A precondition for the function to return the correct judgement is Or and Ol must be defined on the same context CT. In other words, Or and Ol must refer to the same context so that their parameters are directly comparable. The following concepts can help clarify this issue. Definition 1. Operation context Given an operation O, its context, denoted as CTO , is the document state on which O’s parameters are defined. Given an initial document state S0 and a list of operations in L = [O1 , · · ·, On ] performed on S0 . The current document state is denoted as S0 ◦ L = S0 ◦ [O1 , · · ·, On ]. So CTO1 = S0 , CTOi = CTOi−1 ◦ [Oi−1 ] (1 < i ≤ n). Apparently the deltas between the current document state and S0 is the list of operations [O1 , · · ·, On ]. Definition 2. Context equivalent relation “ ” Given two operation Oa and Ob , Oa is context equivalent Ob , denoted as Oa Ob , iff CTOb = CTOa . So the precondition of the Semantic-Conflict(SMR, Or , Ol , CT) function is Or Ol with CTOr = CTOl = CT. An example is given below to show how the function returns a wrong judgement when the precondition is violated. Consider a document contains one line “abcd”. User 1 performs an operation O11 to delete the character located at position 2 in line 1, that is character “c”. User 2 performs two operations O21 and O22 sequentially. O21 inserts a newline character “\n” at position 2 in line 1, that is between character “b” and “c”. After then the document contains two lines: “ab” and “cd”. Then O22 deletes the character at position 1 in line 2, that is character “d”. If the SMR specifies that operations performed within the same line are semantically conflicting, then Semantic-Conflict function would return false because it detects O11 and O22 were performed in different lines. This judgement is obviously wrong because O11 and O22 are actually targeting at the same line that contains characters “c”


313

and “d”. The root of the problem is it does not hold that O11 O22 . O11 was defined when the document contains only one line “abcd” while O22 was defined when the document contains two lines “ab” and “cd”. As a result, their line parameters are not directly comparable. However, given two lists of operations 1 L1 = [O11 , · · ·, Om ] and L2 = [O12 , · · ·, On2 ] performed on the same base version, O11 O12 , but it does not hold that Oi1 Oj2 (1 < i ≤ m, 1 ≤ j ≤ m). The next definition helps derive the second issue. Definition 3. Context preceding relation “ →” Given two operation Oa and Ob , Oa is context preceding Ob , denoted as Oa → Ob , if f CTOb = CTOa ◦ [Oa ]. Given a log L = [O1 , · · ·, On ], it must be Oi−1 → Oi (1 < i ≤ n). Suppose an operation Ok (1 < k < n) cannot be re-executed, Ok cannot be simply omitted by executing Ok+1 right after Ok−1 because Ok+1 was defined on the document state after the execution of Ok and cannot be executed as is on the document state before the execution of Ok . So the second issue is that in order to omit Ok , operations Ok+1 , · · ·, On must be changed to achieve Ok−1 → Ok+1 , and Oj → Oj+1 (k+1 ≤ j < n) so that operations Ok+1 , · · ·, On can be correctly re-executed. Thirdly, if an operation performed by User 1 does not semantically conflict with any operation performed by User 2, that operation should be re-executed in User 2’s working copy. But as explained in previous section, that operation may not be able to be executed as is because the current document state of User 2’s working copy is different from the one of User 1’s working copy on which that operation was originally defined. When we investigated this problem, we found it is the same in nature as the intention violation problem [13] in real-time distributed collaborative editing systems. Therefore operational transformation technique [12] proposed for solving the intention violation problem in real-time distributed collaborative editing systems can be used to tackle this problem. Some background knowledge about the operational transformation technology, which is related to our solution, is briefly introduced as follows. There are two types of primitive transformation functions [12]: one is the Inclusion Transformation function – IT (Oa , Ob ), which transforms operation Oa against operation Ob in such a way that the impact of Ob is effectively included in the parameters of the output operation Oa if Oa Ob ; and the other is the Exclusion Transformation function – ET (Oa , Ob ), which transforms Oa against Ob in such a way that the impact of Ob is effectively excluded from the parameters of the output operation Oa if Oa → Ob . For the example given in previous section, the document initially contains a line “You are student”. Operation O1 = ins[7, 2, “ a”] performed by User 1 is to insert a space and an article “a” between “are” and “student”, and operation O2 = ins[15, 1, “.”] performed by User 2 is to insert a full stop mark “.” after “student”. When O2 is to be re-executed on the current document state of User 1’s working copy, its execution form at Site 1, denoted as EO2 , should be achieved by inclusively transforming O2 against the concurrent operation O1 at this site to take into account O1 ’s effect. In other words, EO2 = IT (O2 ,

314

H. Shen and C. Sun

O1 ) = ins[17, 1, “.”]. After the execution of EO2 , O2 would have been correctly merged into Site 1’s working copy to produce a new document state “You are a student.”. The first issue re-emerges here, that is, to ensure IT (Or , Ol ) returns a correct result, the precondition is Or Ol so that the parameters of Or and Ol are directly comparable [12]. The operational transformation technology can also be used to help solve the second issue. A procedure Transpose (Oa , Ob ) is defined to transpose Oa and Ob where Oa → Ob to achieve Ob → Oa . Therefore, if Ok−1 → Ok → Ok+1 , after Transpose (Ok , Ok+1 ), it becomes Ok−1 → Ok+1 → Ok . As a result, Ok+1 can be executed right after Ok−1 . Procedure 1. Transpose(Oa , Ob ) { }

O := ET (Ob , Oa ); Ob := IT (Oa , O); Oa := O;

An important issue is the ET (Ob , Oa ) function used in Transpose may not succeed if the effects of Oa and Ob are overlapping [11]. For example, consider a document initially contains string “abc”. Two operations O1 and O2 have been executed sequentially. O1 = ins[3, 3, “123”] to insert string “123” after “c” and O2 = del[2, 3, “c12”] to delete string “c12”. Then ET (O2 , O1 ) would not succeed because it is impossible to exclude O1 ’s effect from O2 since the definition of O2 relies on the characters “12” inserted by O1 . The operational merging technique [11] can be used to make their effects disjointed [11], thus making ET transformation successful. In this example, O1 and O2 should be merged as two disjointed operations O1 = ins[3, 1, “3”] and O2 = del[2, 1, “c”]. So Transpose (O1 , O2 ) = (O2 , O1 ) where O2 = ET (O2 , O1 ) = del[2, 1, “c”] and O1 = IT (O1 , O2 ) = ins[2, 1, “3”]. Suppose User 1 and User 2 have concurrently developed their working copies from the same base version V.bv1 = V.bv2 = V.0. User 1’s log L1 = [O11 ] and s/he committed her/his working copy as V.1. User 2’s log L2 = [O21 ] and s/he wants to update her/his working copy with V.1. On one hand, if O11 can be reexecuted in User 2’s working copy, its execution form EO11 should be IT (O11 , O21 ). After the execution of EO11 , bv2 should be update to 1. The last issue is O21 is not able to represent the delta between the current document state after the execution of EO11 and the base version V.1 because O21 represents the delta between the document state before the execution of EO11 and base version V.0. The delta between the current document state after the execution of EO11 and = IT (O21 , O11 ). The reasoning is as follows. The current V.1 should be O21 document state CDS = V.0 ◦ [O21 , EO11 ] = V.0 ◦ [O11 , O21 ] (where O21 = IT (O21 , O11 ), derived from the Transformation Property 1 [12]) = V.0 ◦ O11 ◦ [O21 ] = V.1 ◦ [O21 ]. As a result, the delta between CDS and V.1 is O21 . Because O11 and O21 need to be inclusively transformed against each other, the SIT (Oa , Ob ) (Symmetric Inclusive Transformation) procedure is defined to inclusively transform operations Oa and Ob symmetrically.


315

Procedure 2. SIT(Oa , Ob ) { Oa := IT (Oa , Ob ); Ob := IT (Ob , Oa ); Oa := Oa ; Ob := Ob ; } On the other hand, if O11 cannot be re-executed in User 2’s working copy, the deltas between the current document state and V.1 should be the list of operations [O11 , O21 ] where O11 is the inverse of O11 . The inverse of an operation reverses the effect of that operation. For instance, if an operation deletes a string at some place in a document, then its inverse would be to insert that string at that place into the document. The makeInverse (O) function has been defined to make operation O’s inverse operation O, which has the same parameters as O’s except its operation type is opposite to O’s. The reasoning is as follows. The current document state CDS = V.0 ◦ [O21 ] = V.0 ◦ [I, O21 ] (where I is an identity or a null operation) = V.0 ◦ [O11 , O11 , O21 ] (O11 ◦ O11 = I) = V.0 ◦ O11 ◦ [O11 , O21 ] = V.1 ◦ [O11 , O21 ]. As a result, the deltas between the current document state and V.1 are the list of operations [O11 , O21 ]. In sum, the Merge-Control (RL, LL, V.bv) function is defined with inputs: RL - a list of operations that represent the deltas between the version V.bv and the latest version V.lt in the repository; and LL - a list of operations that represent the deltas between the base version V.bv and current document state CDS at a site. The output are two lists of operations ERL and TLL where the ERL stores a list of operations that should be executed sequentially on the current document state of the working copy to be updated, and TLL stores a list of operations that represent the deltas between the new base version and the new current document state of the working copy that has been updated by the execution of operations in the ERL. Function 1. Merge-Control(RL, LL, V.bv): (ERL, TLL) { RCT := V.bv; //initial remote context for (i = 1; i ≤ |RL|; i++) { //make copies of RO[i]’s and LL’s current states CRLi := makeCopy (RL[i]); CLL := makeCopy (LL); LCT := RCT; //initial local context for (j = 1; j ≤ |LL|; j++) { if Semantic-Conflict (SMR, RL[i], LL[j], LCT) { //Recover RO[i]’s and LL’s states RL[i] := CRLi ; LL := CLL; //Remove RL[i] from RL O := Remove-Operation (i, RL); i := i-1; //Append O to TLL Append-To (makeInverse(O), TLL);

316

H. Shen and C. Sun

//Exit the loop since a conflict occurs break; } else //update local context LCT := execute LL[j] on LCT; //Transform RL[i] and LL[j] against each other SIT (RL[i], LL[j]);

} //If RL[i] is not conflicting with any //operation in LL, append it to ERL if (j > |LL|) { Append-To (RL[i], ERL); //update remote context RCT := execute CRLi on RCT; }

} //All transformed operations in LL are appended to TLL Append-To (LL, TLL); }

return ERL, TLL;

The Remove-Operation (k, L) function is defined to remove the k th operation Ok in the list L, and return Ok on the current context. In other words, after the removal of Ok , L would become [O1 , · · ·, Ok−1 , Ok+1 , · · ·, O|L| ] where Oi → Oi+1 (1 ≤ i < k-1), Ok−1 → Ok+1 , Oj → Oj+1 (k+1 ≤ j < |L|), and O|L| → Ok .

Function 2. Remove-Operation(k, L): Ok { for (i = k; i < |L|; i++) Transpose (L[i], L[i+1]); Ok := L[i]; Remove (L[i]); return Ok ; } The OMUS (Operation-based Merging at the Updating Stage) protocol is devised as follows. Protocol 2. OMUS Suppose the user at Site k whose base version number is bv and log is LL, wants to update her/his working copy with concurrent changes made by other users, which have been committed into the repository: 1. Site k sends a updating request to the repository server, containing the base version number bv. 2. Upon receiving the request, if the repository is locked, the repository server puts the request into the waiting queue. When the lock is released and it is time to respond Site k’s updating request:


317

– If the latest version in the repository is V.bv, the repository server sends a reply to Site k, telling no update available. – Otherwise, the repository server sends a list of operations to Site k, which are the deltas between version V.bv and the latest version V.lt. 3. Upon receiving the operations from the repository, Site k puts them in a list RL with early committed operations on the left and later committed operation on the right, and executes Merge-Control (RL, LL, V.bv) = (ERL, TLL). After then, a) Site k executes operations in the ERL sequentially on the current document state. b) Site k updates its base version number from bv to lt. c) Site k substitutes TLL for its log LL. 3.3

Checkout a Working Copy

The checkout process does not involve document merging. However it is essential to merging because base versions, which are indispensable for merging, are retrieved from the repository only by means of the checkout process. The checkout process is closely related to how versions are stored in the repository. Statistics shows that checking out the latest version is far more frequent than checking out old versions [14]. So in the repository, the latest version of a document is stored as the full state while other versions are represented as deltas. In state-base merging, an old version is represented as reverse deltas [14] from the latest version to that version. As a result, an old version can be retrieved by executing all reverse deltas on the latest version in the repository. In operation-based merging, an old version is represented as deltas from that version to the latest version. As a result, an old version should be retrieved by undoing all operations in the deltas between that version and the latest version in the repository. There are three reasons why reverse deltas are not used. First, the Merge-Control function for the OMUS protocol requires operations in RL are deltas not reverse deltas. Second, operations used for representing deltas are able to represent reverse deltas as well by using their inverse operations. Last, updating a working copy is far more frequent than checking out a new working copy. Therefore it is more efficient to store deltas instead of reverse deltas in the repository. To checkout a new working copy from the repository, the COCO (Control Operation-based Check Out) protocol is devised as follows. Protocol 3. COCO Suppose the user at Site k wants to checkout version V.vn from the repository as her/his working copy: 1. Site k sends a checkout request to the repository server, containing the version number vn. If the version number is omitted, the latest version would be checked out.

318

H. Shen and C. Sun

2. Upon receiving the request, if the repository is locked, the repository server puts the request into the waiting queue. When the lock is released and it is time to respond Site k’s checkout request: – If vn is omitted, or vn = lt where lt is the latest version number in the repository, the repository server sends the full state of V.lt to Site k. – If vn > lt, the repository server sends a rejective reply to Site k. – If vn < lt, the repository server generates the full state of V.vn by executing the inverses of operations in V.lt-1 from the right to the left, then the inverses of operations in V.lt-2 from the right to the left, until the inverses of operations in V.vn from the right to the left on version V.lt, and then sends the full state of V.vn to Site k. For example, if V.lt-1 = lt−1 [O1lt−1 , · · ·, Om ] and V.vn = [O1vn , · · ·, Onvn ], the execution sequence lt−1 would be Om , · · ·, O1lt−1 , · · ·, Onvn , · · ·, O1vn . 3. Upon receiving the reply from the repository server, if it is rejective, Site k warns the user of the invalid version number. If it is a file, Site k makes it as the working copy and marks the base version number bv = vn.

4

An Example

In this section, we use a concrete example to illustrate the proposed merging protocols and algorithms presented in the previous section. As shown in Figure 4, a document initially contains a line “You are student”, which has been imported into the repository as the initial version V.0. Two users User 1 and User 2 want to collaboratively edit the document, so they checkout (at step 1) separate working copies of V.0 from the repository. Initially, their working copies have the same document state CDS1 = CDS2 = “You are student” and their base version numbers have the same value bv1 = bv2 = 0. User 1 performed two operations O11 and O12 sequentially where O11 = ins[15, 1, ”s”] to insert character “s” after “student” and O12 = ins[8, 5, “good ”] to insert “good” between “are” and “student”. After then its current document state CDS1 becomes “You are good students”, and its log LL1 = [O11 , O12 ]. At the same time, User 2 also performed two operations O21 and O22 sequentially where O21 = ins[15, 1, “.”] to insert a full stop mark “.” after “student” and O22 = ins[7, 2, “ a”] to insert a space and an article “a” between “are” and “student”. After then its current document state CDS2 becomes “You are a student.”, and its log LL2 = [O21 , O22 ]. At this moment, User 1 and User 2 both want to commit [3] their working copies into the repository. Suppose User 1 gets access to the repository first, then User 2’s commit request would be blocked. At step 2, because lt = bv1 = 0, operations in LL1 = [O11 , O12 ] would be sent to the repository server and re-executed on V.lt = V.0 = “You are student”. After then, a new version V.lt = V.1 = “You are good students” is generated, and V.0 is replaced with the list of operations O11 and O12 . At User 1’s site, bv1 is updated to 1, and LL1 is emptied. Then User 2 gets access to the repository, because (lt=1) > (bv2 =0), User 1 would be warned to update her/his working copy. At step 3, the deltas between V.bv2 = V.0 and the latest version V.lt = V.1, O11 and O12 , would be transferred to User 2’s site and stored in a list RL. Then


5. update

2. commit

V.lt = V.2 : You are a good student. V.1: O11 , O 21, O V.0 : O 11, O 12

319

4. commit

22

V.lt = V.1: You are good students V.0: O 11, O 12

3. update

V.lt = V.0 : You are student 1. checkout

1. checkout

bv 1 = 0 LL1 = [ ] CDS 1 = You are student O 11, O 12

bv 2 = 0 LL2 = [ ] CDS 2 = You are student O 21, O 22

bv 1 = 0 LL1 = [ O 11, O 12] CDS 1 = You are good students

bv 2 = 0 LL2 = [ O 21, O 22] CDS 2 = You are a student.

bv 1 = 1 LL1 = [ ] CDS 1 = You are good students

bv 2 = 1 LL2 = [ O11, O 21, O 22] CDS 2 = You are a good student.

bv 1 = 2 bv 2 = 2 LL2 = [ ] LL2 = [ ] CDS 1 = You are a good student. CDS 2 = You are a good student.

Fig. 4. An example of operation-based merging

Merge-Control(RL, LL2 , V.0) is executed as follows. Suppose the SMR specifies grammar checking rules. 1. CRL1 = makeCopy (O11 ) = ins[15, 1, “s”] and CLL2 = makeCopy (LL2 ) = [O21 , O22 ] where O21 = ins[15, 1, “.”] and O21 = ins[7, 2, “ a”]. a) Semantic-Conflict (SMR, O11 , O21 , “You are student”) = false, so after SIT (O11 , O21 ), O11 = ins[15, 1, “s”] and O21 = ins[16, 1, “.”]. b) Semantic-Conflict (SMR, O11 , O22 , “You are student.”) = true, so O11 is recovered with CRL1 = ins[15, 1, “s”] and LL2 is recovered with CLL2 = [O21 , O22 ] where O21 = ins[15, 1, “.”] and O21 = ins[7, 2, “ a”]. Then after Remove-Operation (1, RL), O11 = ins[20, 1, “s”], and RL = [O12 ] where O12 = ins[8, 5, “good ”]. Finally O11 = del[20, 1, “s”] is put into TLL. 2. CRL1 = makeCopy (O12 ) = ins[8, 5, “good ”] and CLL2 = makeCopy (LL2 ) = [O21 , O22 ] where O21 = ins[15, 1, “.”] and O22 = ins[7, 2, “ a”]. a) Semantic-Conflict (SMR, O12 , O21 , “You are students”) = false. Then after SIT (O12 , O21 ), O12 = ins[8, 5, “good ”] and O21 = ins[20, 1, “.”]. b) Semantic-Conflict (SMR, O12 , O22 , “You are students.”) = false. Then after SIT (O12 , O22 ), O12 = ins[10, 5, “good ”] and O22 = ins[7, 2, “ a”].

320

H. Shen and C. Sun

c) O12 = ins[10, 5, “good ”] is put into ERL. 3. Finally O21 and O22 are put into TLL. Then TLL = [O11 , O21 , O22 ] where O11 = ins[20, 1, “s”], O21 = ins[20, 1, “.”], and O22 = ins[7, 2, “ a”]. ERL = [O12 ] where ins[10, 5, “good ”]. By executing operation O12 = ins[10, 5, “good ”] in ERL on the current document state CDS2 = “You are a student.”, CDS2 becomes “You are a good student.”. Then bv2 is updated to lt = 1, and LL2 is replaced with TLL = [O11 , O21 , O22 ]. Now bv2 = lt = 1, User 2 is able to commit [3] her/his working copy into the repository. At step 4, after the executions of operations O11 , O21 , O22 on V.1 = “You are good students”, a new version V.lt = V.2 = “You are a good student.” is generated. Then V.1 is replaced with the list of operations O11 , O21 , O22 . At User 2’s site, bv2 is updated to 2, and LL2 is emptied. Finally in step 5, when User 1 wants to update [3] her/his working copy, because LL1 is empty, operations O11 , O21 , O22 from the repository would be executed as is on the current document state CDS1 = “You are good students” to generate a new document state CDS1 = “You are a good student.”. Then bv1 is updated to 2. At this time, if another user User 3 wants to checkout V.0 as her/his working copy, the repository server would execute O22 = ins[7, 2, “ a”], O21 = ins[20, 1, “.”], O11 = ins[20, 1, “s”], O12 = del[8, 5, “good ”], and O11 = del[15, 1, ”s”] sequentially on V.lt = V.2 = “You are a good student.” to recover V.0, and send the full state of V.0 = “You are student” to User 3. Then bv3 = 0 and CDS3 = “You are student”.

5

Conclusions and Future Work

In this paper, we contribute a flexible merging framework in which semantic merging policies and the syntactic merging mechanism are separated. The policy component makes various application-dependent semantic merging policies by formulating semantic merging rules (SMR). The mechanism component performs syntactic merging according to the SMR specified by the policy component. We have applied the proposed framework to the design of a flexible syntactic merging component for a version control system, which uses a single syntactic merging mechanism to support a range of semantic merging policies. We devised operation-based merging protocols: OMCS for operation-based merging at the committing stage and OMUS for operation-based merging at the updating stage. We also devised the operation-based checkout protocol COCO for checking out new working copies as base versions. In addition, we devised a merging algorithm Merge-Control and some auxiliary algorithms to do syntactic merging at the updating stage. These protocols and algorithms have been implemented in FORCE, a testbed for demonstrating the proposed merging framework. In the future, we will continue validating and evaluating the proposed merging framework. We will also investigate possible issues involved in adopting distributed repositories in the framework.


321

Acknowledgments. The work reported in this paper has been partially supported by an ARC (Australian Research Council) Large Grant (000-00711). The authors also wish to thank David Chen, Aguido Horatio Davis, and Jingzhi Guo for their valuable comments.

References 1. L. Allen, G. Fernandez, K. Kane, D. Leblang, D. Minard, and J. Posner. Clearcase multisite:supporting geographically -distributed software development. Software Configuration Management: Selected Papers of the ICSE SCM-4 and SCM-5 Workshops, (number 1005 in LNCS):194–214, 1995. 2. T. Berlage and A. Genau. A framework for shared applications with a replicated architecure. In Proc. of ACM Symposium on User Interface Software and Technology, pages 249–257, 1993. 3. B. Berliner. Cvs ii:parallelizing software development. In Proc. of 1990 Winter USENIX, pages 341–352, 1990. 4. B. Collins-Sussman. The subversion project: buiding a better cvs. Linux Journal, Volume 2002(Issue 94), 2002. 5. W. Courington. The network software environment. In Technical Report FE197-0, Sun Microsystems, 1989. 6. W. K. Edwards, E. D. Mynatt, K. Petersen, M. J. Spreitzer, D. B. Terry, and M. M. Theimer. Designing and implementing asynchronous collaborative applications with bayou. In Proc. of ACM Sympisium on User Interface Software anf Technology, pages 119–128, 1997. 7. A. M. Kermarrec and P. Druschel. The icecube approach to the reconciliation of divergent replicas. In Proc. of ACM Symposium on Principles of Distributed Computing, pages 210 – 218, 2001. 8. E. Lippe and N. van Oosterom. Operation-based merging. In Proc. of the Fifth ACM SIGSOFT Symposium on Software development environments, pages 78 – 87, 1992. 9. W. Miller and E. W. Myers. A file comparison program. Software - Practice and Experience, 15(1):1025–1040, 1990. 10. J. P. Munson and P. Dewan. A flexible object merging framework. In Proc. of ACM conference on Computer Supported Cooperative Work, pages 231 – 242, 1994. 11. H. Shen and C. Sun. A log compression algorithm for operation-based version control systems. In Proc. of the 26th IEEE Annual International Computer Software and Application Conference, to appear. 12. C. Sun and C. A. Ellis. Operational transformation in real-time group editors: issues, algorithms, and achievements. In Proc. of ACM conference on Computer Supported Cooperative Work, pages 59–68, 1998. 13. C. Sun, X. Jia, Y. Zhang, Y. Yang, and D. Chen. Achieving convergence, causalitypreservation, and intention-preservation in real-time cooperative editing systems. ACM Transaction on Computer Human Interaction, 5(1):63–108, 1998. 14. W. F. Ticky. Rcs – a system for version control. Software Practice and Experience, 15(7):637–654, 1985.