resource management in open tuple space systems

R ESOURCE M ANAGEMENT

IN

O PEN T UPLE

S PACE S YSTEMS Ronaldo Parente de Menezes

Submitted for the Degree of Doctor of Philosophy

Department of Computer Science

August 1999

Abstract

This thesis investigates the problem of resource management (memory and I/O) in open distributed implementations of the L INDA model. The principal focus of the research is to deal with the problem of memory exhaustion by implementing garbage-collection techniques at the kernel level of L INDA since explicit methods that rely on users or computational languages are not feasible. The problem of I/O is also explored as it side-effects the solution based on garbage collection. This dissertation shows that the main problem to overcome when implementing a garbage-collection scheme in L INDA is the lack of information about processes’ use of tuple spaces. In order to solve this problem a graph is maintained at the kernel level with all the necessary information. Two ways of maintaining the graph are explored: centralised and distributed. To construct the graph in question two new techniques have been proposed: Process Registration/Checkout and Tuple Monitoring. These techniques form the basis of the graph maintenance at the kernel level. A model, named L IGIA, is proposed incorporating such techniques and an implementation of this model is presented. When looking at a general solution for garbage-collection, problems related to I/O operations became apparent. After proving that I/O is a coordination problem, the solution presented was to bring I/O into the L IGIA model. This is achieved with the introduction of a new operation called tuple spaces.

ii

which enables I/O to be done via

Contents

Abstract

ii

Dedication

1

Acknowledgments

2

Declaration

4

1 Introduction

5

1.1 Thesis Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7

1.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

8

2 Review

10

2.1 Coordination Models/Implementations . . . . . . . . . . . . . . . . . .

11

2.1.1

The L INDA Coordination Model . . . . . . . . . . . . . . . . . . .

12

2.1.2

Multiple Tuple-Space L INDA . . . . . . . . . . . . . . . . . . . . .

17

2.1.3

Melinda . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

23

2.1.4

York Linda 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

26

2.1.5

York Linda 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

28

2.1.6

PLinda . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

30

2.1.7

Bonita . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

32

2.1.8

Bauhaus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

36

2.1.9

Laura . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

41

2.1.10 The PageSpace Effort . . . . . . . . . . . . . . . . . . . . . . . . .

43

iii

CONTENTS

iv

2.1.11 JavaSpaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

48

2.1.12 T Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

49

2.2 Garbage-Collection Schemes . . . . . . . . . . . . . . . . . . . . . . . .

52

2.2.1

Reference Counting . . . . . . . . . . . . . . . . . . . . . . . . . .

52

2.2.2

Reference Listing . . . . . . . . . . . . . . . . . . . . . . . . . . .

55

2.2.3

Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

56

2.3 Distributed I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

60

2.4 Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

62

3 The L IGIA Model

64

3.1 General Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

65

3.2 The Universal Tuple Space . . . . . . . . . . . . . . . . . . . . . . . . . .

66

3.3 First-Class Tuple-Space Handles

. . . . . . . . . . . . . . . . . . . . . .

70

3.4

Tuple Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

72

3.5

-Ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

74

3.6 Process identifiability . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

75

3.7 Tuple Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

78

3.8 Termination-Ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . .

79

3.9 Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

84

4 Incorporation of Garbage Collection

85

4.1 What Garbage-Collection Scheme? . . . . . . . . . . . . . . . . . . . . .

87

4.2 Centralised Graph Structure . . . . . . . . . . . . . . . . . . . . . . . . .

89

4.2.1

Graph General Description . . . . . . . . . . . . . . . . . . . . .

90

4.2.2

The Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

92

4.2.3

The Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

93

4.2.4

Spawning Main Processes . . . . . . . . . . . . . . . . . . . . . .

94

4.2.5

Process Registration . . . . . . . . . . . . . . . . . . . . . . . . .

95

4.2.6

Creation of Tuple Spaces . . . . . . . . . . . . . . . . . . . . . . .

96

4.2.7

Creating Bridges Between Tuple Spaces . . . . . . . . . . . . . .

97

4.2.8

Process Termination . . . . . . . . . . . . . . . . . . . . . . . . .

99

4.2.9

Retrieving Handles from Tuple Spaces . . . . . . . . . . . . . . . 101

CONTENTS

v

4.2.10 Execution of Primitives . . . . . . . . . . . . . . . . . . . . . . . . 102 4.3 Centralised Garbage Collection . . . . . . . . . . . . . . . . . . . . . . . 104 4.4 Distributed Graph Structure . . . . . . . . . . . . . . . . . . . . . . . . . 109 4.4.1

Building a Distributed Graph . . . . . . . . . . . . . . . . . . . . 115

4.5 Distributed Garbage Collection . . . . . . . . . . . . . . . . . . . . . . . 119 4.5.1

Reference-Counting Phase . . . . . . . . . . . . . . . . . . . . . . 120

4.5.2

Local Tracing Phase . . . . . . . . . . . . . . . . . . . . . . . . . . 122

4.5.3

Global Tracing Phase . . . . . . . . . . . . . . . . . . . . . . . . . 126

4.6 Algorithm’s General Description . . . . . . . . . . . . . . . . . . . . . . . 129 4.7 Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 5 Coordination of Input/Output Operations

136

5.1 What is Distributed I/O? . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 5.2 I/O and Garbage Collection . . . . . . . . . . . . . . . . . . . . . . . . . 139 5.3 Internal Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 5.4 The Semantics of I/O Operations in L IGIA . . . . . . . . . . . . . . . . . 144

5.5

5.4.1

The

5.4.2

Management of File Spaces . . . . . . . . . . . . . . . . . . . . . 149

Operation . . . . . . . . . . . . . . . . . . . . . . . . . . 145

L IGIA Generalised . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 5.5.1

Extending the Abstraction of Location . . . . . . . . . . . . . . . 151

5.5.2

Overlapping I/O and Computation . . . . . . . . . . . . . . . . . 153

5.5.3

Distributed Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

5.5.4

Fault-Tolerant File System . . . . . . . . . . . . . . . . . . . . . . 155

5.5.5

Locking Records and Creating Gaps . . . . . . . . . . . . . . . . 156

5.5.6

Associative Access to Files . . . . . . . . . . . . . . . . . . . . . . 158

5.5.7

Persistent Tuple Spaces . . . . . . . . . . . . . . . . . . . . . . . . 161

5.6 Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 6 Implementation of L IGIA 6.1

163

L IGIA Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

6.2 Server System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 6.3 Agent System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

CONTENTS

vi

6.4 Garbage Collector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 6.4.1

Registering Processes . . . . . . . . . . . . . . . . . . . . . . . . . 182

6.4.2

Scoping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

6.4.3

Process Checkout . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

6.4.4

Centralised Garbage Collection . . . . . . . . . . . . . . . . . . . 188

6.4.5

Distributed Garbage Collection . . . . . . . . . . . . . . . . . . . 189

6.5 I/O System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 6.6 Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 7 Experimental Results

195

7.1 Configuration Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 7.2 Garbage Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 7.2.1

Memory Exhaustion . . . . . . . . . . . . . . . . . . . . . . . . . 196

7.2.2

Process Registration and Checkout . . . . . . . . . . . . . . . . . 198

7.2.3

Avoiding Memory Exhaustion . . . . . . . . . . . . . . . . . . . . 199

7.2.4

Garbage-Collection Overhead . . . . . . . . . . . . . . . . . . . . 201

7.2.5

Hiding the Overhead . . . . . . . . . . . . . . . . . . . . . . . . . 204

7.3 Input/Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 7.4 General Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 7.4.1

Java Tuple Building . . . . . . . . . . . . . . . . . . . . . . . . . . 208

7.4.2

Via Separate Thread . . . . . . . . . . . . . . . . . . . . . . . 209

7.4.3

Storing and Retrieving Tuples . . . . . . . . . . . . . . . . . . . . 211

7.4.4

Bulk Operations with Tuples . . . . . . . . . . . . . . . . . . . . . 212

7.4.5

Multi-thread Servers . . . . . . . . . . . . . . . . . . . . . . . . . 213

7.5 Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 8 Conclusions and Future Research

215

8.1 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 8.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 8.2.1

Garbage Collection . . . . . . . . . . . . . . . . . . . . . . . . . . 218

8.2.2

Bringing I/O operations within the model . . . . . . . . . . . . . 219

8.2.3

A novel run-time system . . . . . . . . . . . . . . . . . . . . . . . 219

CONTENTS

vii

8.3 Closing Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 A L IGIA Classes Definition

220

A.1 Ligia’s Java Classes Description . . . . . . . . . . . . . . . . . . . . . . . 220 A.2 Agent System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 A.3 Server System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224 A.4 I/O System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 Bibliograph

228

List of Algorithms

4.1 Creation of processes (process registration). . . . . . . . . . . . . . . . . 130 4.2 Creation of tuple spaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 4.3 Storing of handles into other tuple spaces. . . . . . . . . . . . . . . . . . 132 4.4 Destructive retrieving of handles. . . . . . . . . . . . . . . . . . . . . . . 132 4.5 Non-destructive retrieving of handle. . . . . . . . . . . . . . . . . . . . . 133 4.6 Update of the graph based on bulk primitives . . . . . . . . . . . . . . . 134 4.7 Process termination. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

viii

List of Codes

2.1 Part of a M ELINDA program . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Bonita simulation of 2.3

. . . . . . . . . . . . . . . . . . . . . . . . . . . .

25 34

L INDA program which stores one hundred tuples and removes them afterwards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

35

2.4 Bonita implementation of L INDA example in Code 2.3. . . . . . . . . . .

35

2.5 A service in large specialised whisky shop offering several operations. .

42

2.6 A service in an off-licence shop offering one operation . . . . . . . . . .

42

4.1 Code of a L IGIA main process. . . . . . . . . . . . . . . . . . . . . . . . .

97

4.2 Code of executing several different primitives . . . . . . . . . . . . . 102 5.1 Sketch of the processes’ code taking part of a Job Scheduling Problem.

ix

161

List of Programs 3.1 Program where the process offers the service

and pro-

cess request for this service in the tuple space . . . . . . . . . . . .

68

3.2 Example where -ordering behaves differently in L IGIA and Bonita. .

75

3.3 Termination-ordering with . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Termination-ordering assuming the standard semantics of . . . . . 3.5 Termination-ordering with primitive in the L IGIA case. . . . . . . .

82 82

3.6 Termination-ordering with primitive

83

. . . . . . . . . . . . . . . .

3.7 Termination-ordering with primitive 6.1

. . . . . . . . . . . . A toy program composed of two processes. The keeps indef initely spawning agents which only create a tuple space and

83

84

terminate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 6.2 Example of a program where some tuple spaces are used only within a specific scope. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 6.3 Modification of Program 6.2 introducing scope in the coordination dimension. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 6.4 Introduction of method in the process shown in Program 6.3. . . 185 6.5 Java makes the method waits for the completion of the thread

6.6

(in charge of the s). . . . . . . . . . . . . . . . . . . . . . 187 The implementation waits for the completion of the thread and therefore Java should not have to wait at the end of . . . . . 187

x

List of Tables

7.1 Execution of the primitive

compared with the execution of prim-

itive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 7.2 Creation of

objects in L IGIA . . . . . . . . . . . . . . . . . . . . . . 209

7.3 Values expected and measured (real) showing the scalability of a L IGIA server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

xi

List of Figures

2.1 Pictorial representation of formals and actuals as part of tuples in the matching mechanism. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

15

2.2 Pictorial representation of the standard L INDA. Processes are spawned within a tuple space. . . . . . . . . . . . . . . . . . . . . . . . .

17

2.3 Pictorial representation of Gelernter’s original multiple tuple-space proposal. Tuple spaces are created inside tuple spaces and processes (active tuples) execute within a tuple space. . . . . . . . . . . . . . . . . . .

19

2.4 Representation of the flat tuple-space structure proposed for the York L INDA 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

27

2.5 Representation of the flat tuple-space structure proposed for the York L INDA 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

29

2.6 Formula of time costs of an original L INDA primitive. . . . . . . . . . . .

32

2.7 Representation of the coordination structure in . . . . . . . . .

39

2.8 An example illustrating the execution of primitives in . . . . . .

40

2.9 The PageSpace architecture (based on [CTV97]) . . . . . . . . . . . . . .

47

2.10 Cyclic garbage being created but not detected because the reference counting does not work properly with cyclic structures. . . . . . . . . .

53

2.11 Invariant of the weight reference-counting algorithm. . . . . . . . . . .

54

2.12 Situation using indirect references in the weight reference counting. . .

55

xii

LIST OF FIGURES

xiii

2.13 In the reference-listing method, each cell has a list of references as opposed to a counter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

56

2.14 Illustration of the problem when mutator and collector are running concurrently and one possible solution where they collaborate. . . . .

58

2.15 Example showing the Gelernter’s proposal for represent files as tuple

3.1

spaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

61

L IGIA structure of tuple space and processes. . . . . . . . . . . . . . . .

67

3.2 Without a universal tuple space some processes are unable to communicate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

69

3.3 Impossible cases of ordering between the termination message and the primitive 4.1

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

81

and create a boundary separating L IGIA from the outside world. 88

4.2 Graph situation after the creation of and nodes by the L IGIA system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

93

4.3 Two main processes, and , register themselves with the kernel just before they start running.

. . . . . . . . . . . . . . . . . . . . . . . . . .

96

4.4 Creation of tuple spaces. A tuple space has to be linked to its creator. .

96

4.5 Execution of the steps of Code 4.1 . . . . . . . . . . . . . . . . . . . . . .

98

4.6 Process terminates generating garbage. . . . . . . . . . . . . . . . . . 100 4.7 All possible cases of handle retrieval. . . . . . . . . . . . . . . . . . . . . 103 4.8 Pictorial execution of Code 4.2. . . . . . . . . . . . . . . . . . . . . . . . 105 4.9 Tuple spaces being garbage collected based on reference counting. . . 107 4.10 Tuple Spaces being garbage collected based on tracing. . . . . . . . . . 108 4.11 Local and global collectors cooperating on the collection of objects using a distributed graph structure. . . . . . . . . . . . . . . . . . . . . . . 111 4.12 A distributed representation of graph structure divided into two locations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 4.13 Pictorial view of the graph in a distributed execution of operations in L IGIA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

LIST OF FIGURES

xiv

4.14 Pictorial view of the graph in a distributed execution of operations in L IGIA (continuing from Figure 4.13) . . . . . . . . . . . . . . . . . . . . . 118 4.15 Local reference-counting phase at location

finds the tuple space

garbage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 4.16 Example of a situation after a local marking phase in a system with two local graphs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 4.17 Scenario just after the end of the local tracing phases. . . . . . . . . . . 128 4.18 Global graph. Result of the agglutination of the graph in Figure 4.17 and already marked by the global tracing phase. . . . . . . . . . . . . . 128 5.1 Update of the garbage-collection graph on I/O operations. . . . . . . . 141 5.2 Implementation of process migration in L IGIA would causes an I/O problem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 5.3 Representation of a file space and a tuple space in terms of and .

. 148

5.4 Structure of the L IGIA kernel in terms of TSMs and FSMs. . . . . . . . . 150 5.5 Scenario depicting the location-free abstraction of I/O objects. . . . . . 152 5.6 Execution of the primitive

in a file space. . . . . . . . . . . . . . . . . 157

5.7 Dependence graph of a Job Scheduling Problem. . . . . . . . . . . . . . 160 6.1 Pictorial view of the L IGIA architecture . . . . . . . . . . . . . . . . . . . 166 6.2 Number of messages generated by the routing of L IGIA operations.

. . 169

6.3 Flow of a message within the server system . . . . . . . . . . . . . . . . 172 6.4 Structure of tuple spaces and tuples

. . . . . . . . . . . . . . . . . . . . 175

6.5 Processes running at locations; in each one an I/O server is also running 192 7.1 Memory exhaustion in a L IGIA server due to tuple-space creation. . . . 197 7.2 Overhead added by process registration and checkout. . . . . . . . . . . 199 7.3 Memory exhaustion problem . . . . . . . . . . . . . . . . . . . . . . . . 200 7.4 Overhead added by the tuple monitoring as well as by the existence of an active process representing the garbage collector in the server. . . . 203 7.5 Raw Garbage-Collection Overhead . . . . . . . . . . . . . . . . . . . . . 205 7.6 Experiment with file operations . . . . . . . . . . . . . . . . . . . . . . . 206 7.7 Creation of

objects in L IGIA . . . . . . . . . . . . . . . . . . . . . . 208

LIST OF FIGURES

xv

7.8 Comparison between time taken and time observed by a user when a sequence of s is being performed. . . . . . . . . . . . . . . . . . . . . 210 7.9 Comparison of time taken to execute , and followed by 7.10 Tuple movement via

- and via

. . . . 211

. . . . . . . . . . . . . . . . 212

7.11 Scalability of a L IGIA server with relation to the number of processes. . 213 A.1 Relationship between L IGIA’s agent classes and Java classes and interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 A.2 Relationship between L IGIA’s server classes and Java classes and interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 A.3 Relationship between L IGIA’s I/O classes and Java classes and interfaces.223

Dedication

To Sarah and Kalil, for their support, help and love.

1

Acknowledgments

It is always difficult to remember everyone to thank, so if I forget anyone please forgive me. First I would like to thank Dr. Alan Wood for always being keen to help me with any work or personal problem; for always keeping the door of his office open and being receptive the many times I knocked. I would like to thank Dr. Antony Rowstron for all the help and support and for the innumerable hours we spent discussing our work over tea time. I have to thank my examiners Dr. Colin Runciman and Dr. Robert Tolksdorf for all their comments which surely helped me improve the quality of this thesis. I must thank my friends at the department of Computer Science, specially my great friends José Campos, Paulo Pedro, and Eraldo Ribeiro Jr., for providing me with such a friendly environment. Then there is Pedro Rafael Sampaio, for the innumerable hours we spent chatting on the telephone; my bill was surely high but that surely made me happy. There are also my friends in York, namely, Danielle and Dominic Richardson, Cesar Ortega-Sanchez and Eugenia Schettino and Paul Sparks and Tobey Burnett. Thank you all the time we spent together. I would like to thank Jorge Borba and Márcio Braga for their friendship; a friendship that is not bounded by land limits. I could not forget my family; all my brothers and sisters, Fl a´ vio Augusto, Flávima, Henilde, Robson and Henilton, for always being on my side in all the decisions I have made. My parents Flávio and Maria do Carmo for basically being who they are; I love 2

LIST OF FIGURES

3

you both. Finally, but certainly not least, I would like to thank my wife for helping me reviewing the English in this thesis despite of all the things she had to do, and for being not only my wife but, above all, my dearest friend. And I have to say thank you to Kalil, my son, who is the reason of my life.

Declaration

This thesis is the result of my own work. It is not substantially the same as any report I have submitted for any other qualification at any other university. Some of its contents have been published. Chapter 3 has been partially published at the 2nd Workshop on Distributed Systems (WOSID), Brazil, 1998 [Men98]. Chapter 4 has been in part published at the 10th International Conference on Parallel and Distributed Computing and Systems, USA, 1998 [MW98c] and at the 14th International Symposium on Computer and Information Sciences, Turkey, 1999 [MW99]; earlier work on this subject has also been published at the 15th Brazilian Computer Networks Symposium (SBRC), Brazil, 1997 [MW97]. Chapter 5 has been in part published at the 31st Hawaii International Conference on System Sciences, USA, 1998 [MW98c] Another paper on the use of L INDA as a Web framework [MMW99] and not directly associated with my work in this dissertation has been published. A technical report [MW98c] based on the work described in this thesis has been published at the Department of Computer Science, University of York.

4

Chapter 1

Introduction

L INDA was proposed by Gelernter [Gel85] and has since formed the basis of several research and application projects. L INDA introduced the concept of orthogonality between computation and coordination; Gelernter and Carriero [GC92] proposed this separation arguing that this is a good way to build a complete distributed language — take a computation language (like C, Pascal) and add coordination to it. The coordination part of a distributed language is responsible for dealing with tasks that involve more than one process in their completion, namely tasks that include synchronization and communication between processes. The L INDA coordination model is based on the concept of tuple spaces — shared associative memories. All communication is performed via these structures which unify the concept of communication with the concept of synchronization. Apart from the first version of L INDA, implementations allow processes to create tuple spaces, that is, create ‘local’ memories at will. As originally proposed, L INDA was a model for parallel computing and not distributed computing; this means that in general the implementations of L INDA were closed implementations. Only later it started to be noticed that the intrinsic characteristics of the model and the above mentioned orthogonality argument makes L INDA interesting for distributed computing as well. General purpose distributed systems may be restricted if implemented in closed systems so open implementations of L IN 5

6 DA

started to appear. So far the story seems to be flowing perfectly. However, if put together, some con-

cepts may cause some problems. In closed implementations of L INDA, the system is well controlled and optimisations of all sorts can be done since at compile time all the information in an application is available to the L INDA implementation which can then manage the resources in the best possible way. What if the concept of compile time does not exist for the whole system? or, what if the application is implemented on the top of an open implementation of L INDA? This is when this thesis comes in. Open systems in general might mean heterogeneity; processes running in different machines running different operating system, implemented on different languages. In L INDA open systems only L INDA itself provides the processes with some homogeneity. What about the tuple spaces being created? Memory is a finite resource; it may be large but it is surely limited to a certain degree. Once tuple spaces are created they are never deleted. In closed implementations of L INDA, at compile time the application can be optimised and the compiler can eliminate any tuple space from the system as soon as it becomes unnecessary and it is easy to know whether it is unnecessary because the compiler basically knows everything that may happen in the system. In open systems the story is completely different. A process cannot decide by itself whether a tuple space created is necessary in the system or not after its termination; of course in some cases it might be possible but in general this is clearly not the case. The individual computational languages also cannot decide on L INDA tuple spaces’ usefulness as tuple spaces are objects that exist beyond the boundaries of a computational language. The users should not be bothered with such tasks and even if they were they would not be able to securely decide if a tuple space is necessary or not. What is the solution then? Tuple spaces surely need to be removed from the system but nothing seems to be able to implement this securely. The solution is called garbage collection. It has been mentioned that L INDA provides homogeneity so if one could implement a data structure at the L INDA level that keeps track of processes’ use of tuple spaces, a garbage-collection system could be implemented to use this structure and remove any unnecessary tuple space from the

1.1. Thesis Overview

7

system. This thesis explores how such a structure can be maintained in a centralised and distributed fashion within L IGIA (the L INDA based open implementation used in this dissertation). But heterogeneity does bring other concerns that although not apparently linked to garbage collection will be shown to be very much attached to the garbage-collection problem. When carrying out research on garbage collection in L INDA, it became apparent that I/O is problem that has to be dealt with so as to have a stable (and correct) solution to the garbage-collection problem. The I/O problem noticed was that depending on whether a process can do I/O or not the garbage collection would have to behave differently. But why does this happen if I/O is computation and the garbage collection is dealing with coordination structures? The answer could not be more obvious: I/O operations side-effect the coordination environment because I/O management is a coordination problem. The second part of this thesis describes how to bring I/O into the L INDA model and at the same time make the garbage-collection system aware of whether processes can do I/O or not and consequently use this information when removing some structure from the L INDA system. The incorporation of I/O within L INDA is done based on the existence of a new primitive implemented in L IGIA, called

, that

abstracts files as tuple spaces so that processes can deal with files in a similar way as dealing with tuple spaces. This dissertation proves the need for a garbage-collection system and the need to have I/O abstracted in L INDA terms if L INDA is to become a general purpose distributed system. Again, this thesis focusses on open implementations of L INDA because although the same problems exist in closed implementations, they are under the control of the compiler and any solution can be implemented at the compiler level.

1.1

Thesis Overview

Chapter 2 reviews the areas relevant to this thesis. Several L INDA-like models and implementations are discussed highlighting the fact the none of them pays the

1.2. Contributions

8

necessary attention to the problem of garbage collection and I/O. A review of the relevant garbage-collection algorithms and some work done in the distributed I/O area is also presented. Chapter 3 introduces the L IGIA model. This model has been proposed for open systems and therefore assumes the existence of a garbage-collection system that would be attached to the L IGIA implementation. Chapter 4 explores the theory of incorporating garbage collection in L IGIA. Both centralised and distributed solutions are explored. Chapter 5 introduces the theory behind the introduction of distributed I/O via tuples spaces implemented in L IGIA. An exhaustive account of the possibilities supported by this proposal is presented. Chapter 6 describes the implementation of the L IGIA model as well as the implementation of a centralised and a distributed version of the garbage collection. The implementation of a restricted version of the distributed I/O system is also described. Chapter 7 presents a number of experimental results extracted from the L IGIA implementation. Experiments test the performance of the garbage collection, the I/O system as well as general performance results. Chapter 8 discusses the conclusions about the research described in this dissertation. Also a section on future work briefly explores some of the questions that appear as the result of the work described here.

1.2

Contributions

This dissertation has made the following contributions: the problem of maintaining enough information in L INDA that could be used by a garbage-collection algorithm has been fully described and explored;

1.2. Contributions

9

The definition of I/O as a coordination problem and a proposal for bringing I/O into L IGIA using a new operation (

) has been presented; and

a run-time system, called L IGIA, has been implemented, providing the following interesting/innovative ideas: – the introduction of process registration with the kernel which enables the implementation of optimisations at the kernel level; – the implementation of tuple monitoring by the kernel, which together with process registration gives more control to the kernel enabling optimisations; – implementation of a two-stage garbage-collection system; – abstraction of files as tuple spaces — the introduction of the concept of file spaces.

Chapter 2

Review

This chapter reviews the three important areas related to this thesis: distributed coordination in particular L INDA, garbage collection, and distributed input/output. Coordination models and implementations are reviewed in Section 2.1; the concept of coordination is defined and a number of L INDA-like coordination models/implementations are described. The descriptions try to emphasise the strong and weak points of each proposal highlighting at the same time how they cope with the problems related to resource management. At the end of each model/implementation description garbage collection and distributed I/O is put in context and their need is made clear. In Section 2.2 garbage collection is explained and some algorithms described; the emphasis is given to standard algorithms and to a few that have been considered in adapting to the L INDA case. In Section 2.3 the problem of distributed I/O is put in context and solutions to related systems are described. Although the distributed I/O case in L INDA is completely different from all the cases described, it is important to realise that the same problem is also being dealt with in other distributed environments.

10

2.1. Coordination Models/Implementations

2.1

11

Coordination Models/Implementations

One of the first questions that normally comes to one’s mind with regards to coordination models is “what sort of coordination is this?” Spoken as it stands, coordination is a vague term and perhaps meaningless. When one refers to coordination models, the idea is that the model is able to coordinate distributed/parallel tasks involving more than one process. Normally, coordination is needed in communication and synchronisation of processes, but it can also be necessary in other more specific aspects of distributed systems. Nonetheless, the term coordination itself needs definition. According to Carriero and Gelernter [GC92]: “Coordination is the process of building programs by gluing together active pieces (processes)” From this definition, one can observe that coordination is defined as being orthogonal to computation since according to the definition no changes in the processes are required during the process of gluing. In other words, coordination can be seen as the process of managing activities in a distributed system which involves two or more processes. Derived from the term coordination is the definition of coordination models. Once more, Carriero and Gelernter [GC92] gave the following definition: “A coordination model is the glue that binds separate activities (processes) into an ensemble.” A more formal definition of coordination model states that a coordination model is a triple ( , , ) [Cia97]:

are the entities that need to coordinate. These entities are active objects performing tasks which are dependent on other active objects. In the terms used in this thesis, the entities are processes.

is the set of media (media for short) used in the coordination. The media is

the way that entities interact among themselves.


12

are the laws that rule the actions of the coordination entities.

Coordination languages are implementations of the laws of coordination models which can be put together with a host (computational) language. A coordination and a computational languages combined form a complete distributed language which is able to construct distributed systems using the idea of orthogonality of computation and coordination. Several computational languages have already been used together with a coordination model, some described later in this section. The definition given by Ciancarini [Cia97] is general and one might consider several levels of abstraction of coordination models. In short anything where a triple ( , , ) can be defined can be seen as a coordination model.

For instance, the language C combined with the Unix operating system can easily be considered a coordination model since entities, media and laws can be defined. Processes in C (entities) can use Unix sockets (media) to coordinate. The laws of this model would clearly be laws that govern communication via sockets. Other coordination models include: GAMMA (General Abstract Model for Multiset Manipulation) [BL90], CHAM (Chemical Abstract Machine) [BB90], IAM (Interaction Abstract Machine) [ACP93], LO (Linear Objects) [AP92], Manifold [AR93] and Actors [Agh86]. Although all the problems of garbage collection and distributed I/O also exist in these models this thesis concentrates on the variations of the L INDA model [Gel85].

2.1.1 The L INDA Coordination Model L INDA [Gel85] is by far the most studied coordination model. When first proposed, L INDA was aimed at parallel computation but later its properties were studied further and its applicability to distributed systems became apparent. In a nutshell, the standard L INDA model consists of a shared associative memory which can be accessed by processes via primitives provided. L INDA also introduced a new model for concurrent programming. The novelty relied on the fact that L INDA used a different model for communication: generative communication. Generative communication was argued to be different enough from other models — namely shared variables, message-passing, remote calls — to constitute a different model.


13

As a coordination model, L INDA can be defined as a triple ( , , ). The entities, ,

in L INDA are processes and tuples. Processes are executable codes while tuples are ordered lists of typed elements each of which may be either an actual (a value) or a formal (a definition of a value). The media,

, is called tuple space and can be seen

as a multi-set (or a bag) of tuples. The tuple space is a shared resource and therefore every process can access it. The laws, , that allow processes to interact are based

on associative addressing where a matching mechanism compares two tuples to see if they have similar characteristics. L INDA provides three primitives 1 to access the tuple space:

: Where ,

,

is a list of elements (formals or actuals) forming

a tuple. is not different from the other fields, but it is specially used as an identifier — a tag — which allows tuples to be organized in “groups”. The execution of the primitive results in the insertion of the tuple

into the

tuple space.

: Where ,

a tuple and the tuple

is a list of elements (formals or actuals) forming

is executed, a tuple matching

is withdrawn from the tuple space. If no tuple is found,

is suspended (blocked) until a tuple is made available when

then the execution of

,

is an actual. When the primitive

the execution of

resumes.

: This primitive when executed behaves similarly to

except that

if a matching tuple is found within the tuple space the tuple is read instead of withdrawn. The idea of using

as a tag for tuples was soon dismissed only because this tag

does not guarantee that an application would not be side-effected by the execution of other. Yet, this can be seen as an early version of tuple-space handles introduced in the multiple tuple-space L INDA [Gel89] (see Section 2.1.2). A concept that derived from associative addressing is non-determinism. The matching mechanism finds a matching tuple but does not guarantee the retrieval of a par1

Although Gelernter refers to the name operations, the name primitives will be used throughout this thesis.


14

ticular tuple; any matching tuple is returned. Non-determinism is a central concept in L INDA and one that makes L INDA’s communication scheme different from other approaches, like message passing on shared variables for instance. Non-determinism is a well-known concept and can be formally expressed as in Property 2.1. Property 2.1 Non-determinism Given a relation , and .

is non-deterministic if there exist , and such that ,

The matching mechanism of L INDA uses what Gelernter called structured naming [Gel85] which consists of an ordered list of actual elements belonging to the tuple in the same order they appear in the tuple. For instance the tuple . A tuple matches a tuple if ( )= ( ) has a structured name and ( )= ( ), and if , "!

#

; where %$

( ), ( ) is a func-

tion that returns the number of fields in a tuple , (& ) is a function that returns the type of the -th field of a tuple and is the set of formals (types) available in a given language. As defined by Gelernter [Gel85], formals can be stored within the tuple space, or in his words, “can exist within a tuple”. In order to retrieve a tuple with a formal, the tuple has to have an actual to match the formal, since formals cannot match formals. The idea is that of a jigsaw puzzle of only two pieces, it does not matter who has the holes (formals), or the bumps and the straight lines (actuals), but that when they are put together they fit and form an object as a whole. Figure 2.1 gives a pictorial idea of the matching mechanism. Figure 2.1 shows all possible cases of matching: actual and formal ( matching , and matching ), formal and actual ( ' matching ' , and matching ), and actual and actual ( matching ). L INDA provides communication orthogonality which allows data to be sent with the sender having no knowledge of the receiver, as is the case with the receiver which does not need to know about the sender in order to receive the data. This orthogonality in communication has two consequences which defines the basic behaviour of the L INDA model: Space uncoupling: This concept will be also called location free. It refers to the fact


15

Figure 2.1 Pictorial representation of formals and actuals as part of tuples in the matching mechanism. Formals Actuals

-t

-a

a

-b

b

-c

c

-d

d

-e

e

+t

Actuals Formals

that L INDA processes are not aware of physical location of objects. A tuple stored within the tuple space can be retrieved (removed or read) by any process at any location as long as the process knows about the format of which will match . Time uncoupling: This means that the notion of time does not exist in L INDA and consequently two processes can communicate even when their execution times do not intersect. A tuple stored within the tuple space, if not removed by a process, remains there forever. Gelernter’s proposal introduced L INDA to the world and formed the basis for a number of other proposals based on L INDA to be published (e.g. [Gel89, Hup90, CGKW91, BZ91, AS92, NS93, Tol93]). Some of them implement the standard L INDA, others implement a modified model and there are also those which are theoretical and concern themselves with modifying the standard model without implementing it. The scope of this thesis is open L INDA-like implementations. The term L INDA-like is used to identify a family of implementations based on the general properties of L INDA described here. Later, Gelernter and his colleagues at the Yale University published a series of papers on L INDA refining the model regarded as the standard L INDA model. Nonblocking versions of some primitives were introduced and the primitives were de fined in a more informal way. A primitive was also proposed which is the L INDA


16

mechanism for spawning new processes. The standard primitives as they are known today are:

: Stores the given

within the tuple space.

: Removes a tuple matching the given

from the tuple space

or blocks if a matching tuple cannot be found.

: Non-blocking version of

. If a matching tuple is not found,

is returned.

: Reads a tuple from the tuple space matching the

or block-

s if one cannot be found.

: Non-blocking version of . If a matching tuple is not found,

is returned.

: Spawns an active

in the tuple space.

This original proposal assumes the existence of a single tuple space which is known by every process in the system, later this tuple space will be called tuple space in the multiple tuple-space proposal. Figure 2.2 gives a pictorial view of the standard L INDA model; processes (represented by squares) execute within a single tuple space (the tuple space). Originally proposed for parallel computation this model is not appropriate for large-scale applications nor for open distributed systems as conflicts may arise. In large-scale applications conflicts in the format (template) are likely to happen. A model where a single tuple space (a single shared memory) exists does not provide any separation of concerns, so important for large-scale applications and open distributed systems. With a single tuple space, tuples have to include some identification tag common to all tuples which are part of the same concern which is certainly not sufficient to avoid conflicts. Separation of concerns was one of the driving forces for the proposal for the multiple tuple-space version described in Section 2.1.2. This model does not require garbage collection as the tuple space is a persistent object that cannot be collected. One could argue that for this reason garbage


17

Figure 2.2 Pictorial representation of the standard L INDA. Processes are spawned within a tuple space.

default

collection should be done at the tuple level. Note that, as originally defined in L IN DA,

a tuple is not uniquely identifiable and unless the model is changed no garbage

collection can be employed on this level. Still, if one decides to change the model to have unique identification of tuples, this model is surely not L INDA-like as this identification of tuples implies the use of another sort of communication model — breaking the most important L INDA characteristic.

2.1.2 Multiple Tuple-Space L INDA The multiple tuple-space L INDA model was proposed by Gelernter [Gel89] as an extension of the standard model described in Section 2.1.1. At the time of its proposal, the multiple tuple spaces idea was already well accepted in the L INDA research community considering that its original description assumed this possibility and because the limitations of a single tuple-space model were evident. The main difference in the multiple tuple-space model is that processes are now able to create tuple spaces. Multiple tuple spaces make the L INDA model more modular and introduce separation of concerns. For open distributed implementations of L INDA the existence of multiple tuple spaces is crucial. In these systems the applications tend to run for a


18

longer period of time and tend to be large — multiple tuple spaces provide a better way to implement these sort of applications. A new operation, called , was introduced so as to execute the task of creating tuple spaces; a new type, , has been defined. Some might argue that the underlying model here is the same and that multiple tuple spaces are just an implementation feature. The view in this thesis is that multiple tuple spaces introduce a different model; it clearly has similarities but this is because L INDA is still the central concept. The model is different for the triple defining the model has been modified. The entities, , are the same as the standard L INDA model. The media,

, have clear-

ly been modified as multiple tuple spaces are now used — the characteristics of the media are the same but the triple is not defined on characteristics but on the media itself. The last element of the triple, the laws , have also been modified; the differ

ence here is subtle and can easily pass unnoticed. The way the associative matching is performed is different as it occurs only within some context (a given tuple space); this can also be noticed since the expression of the laws, the primitives, has significantly been changed. As said before, a new operation has been added to allow creation of tuple spaces; the semantics of is as follows:

: It is defined as an operation that has to be spawned (like a process) via in order to create a tuple space. The execution of would create a tuple space within the tuple space. From the point of cre

ation onwards a process can execute , , , and

.

The creation of tuple spaces introduces a hierarchy of tuple spaces, similar to the structure of directories in a file system. Thus, operations like become possible — the tuple space contains all other tuple spaces as depicted in Figure 2.3. Gelernter’s implementation of his model is called L INDA 3 and it is quite a general proposal as several new concepts are presented. Most of the semantics is missing and the weakness of the proposal is to try to be general and introduce too many new ideas


19

Figure 2.3 Pictorial representation of Gelernter’s original multiple tuple-space proposal. Tuple spaces are created inside tuple spaces and processes (active tuples) execute within a tuple space.

ts3 ts5

ts2 default

ts1

ts6 ts4

without explaining them in detail and disregarding most of the problems involved. Gelernter proposed several ideas that still today are not well understood. Notwithstanding the problem of trying to tackle many problems, Gelernter introduced one of the most important concepts in L INDA developing towards a distributed model: separation of concerns by permitting creation of tuple spaces. Multiple tuple spaces provide an interesting environment for distributed systems. With the introduction of this new concept the primitives had also to be extended for the multiple tuple-space case. Below is the description of the primitives as assumed by Gelernter in L INDA 3:

: Similar to the original primitive . It stores the

in the

tuple space now defined in the primitive. If a tuple space is not defined the execution occurs in the tuple space.

: Removes a tuple from matching the

or blocks if

a suitable tuple is not found. Tuples outside are not verified in the matching mechanism.


: Non blocking version of

. If a matching tuple is not found

is returned.

20

: Reads a tuple from matching the

or blocks if

such a tuple is not found.

: Non blocking version of . If a matching tuple is not found

is returned.

: Spawns an active

within the tuple space defined in the

primitive. One should notice that L INDA 3 does not implement the primitives as originally proposed. As defined at the model level the primitives are preceded by the tuplespace handle whereas above the handle is defined as an argument of the primitive. In L INDA 3 what distinguishes a tuple space from another is a handle. Surprisingly a handle is defined only as a string. Therefore the handle in the primitives above is not necessarily a unique name and the semantics of the primitives when a collision of names occurs is not clear. The uniqueness of a handle is another important feature in open systems — conflicts of names can lead to undesired inconsistencies. It should be taken into consideration that, although it can be regarded as an improvement, the multiple tuple-space L INDA when implemented using string handles is similar to the single tuple-space case. The creation of handles should be dealt with by the L INDA implementation itself and not be left for users to decide. In large applications, it is likely to have different tuple spaces with the same handle. Again, if the users define the handles, separation of concerns cannot be guaranteed in the same way that it cannot be guaranteed in a single tuple-space system by only using a special identification element in a tuple. In L INDA 3 tuple spaces are first-class objects. Still today it is rather obscure within the L INDA community how the primitives should behave — what are their semantics — when tuple spaces are indeed first-class objects. Throughout this chapter one will see that most L INDA implementations avoid this idea. As first-class objects, tuple spaces can even be elements of tuples.


21

The concept of active tuples (processes) is also present here. Even though processes are not first-class objects (cannot be used as tuple elements for instance) they can be affected by operations on tuple spaces. Gelernter argues that when an active tuple space2 is retrieved using an

, all processes in this tuple space are stopped. Still, they can resume if the same tuple space is ed again. This is certainly an interesting idea but before being implemented several issues have to be tackled: The question of how active tuples (processes) can be stopped and resumed is not trivial to answer. Some languages implement the continuation passing style paradigm which can be used to this end but clearly this is not an option in every language. In order to stop a process one should store at least a minimum of data to allow it to be resumed. What comprises the minimum data? How is it possible to avoid undesirable side effects? How can it be implemented? A tuple space to be retrieved in L INDA has to be associated with a variable, perhaps of a complex data type as it has to contain all tuples (active and passive). Retrieval of active tuple spaces means solving the question of how the processes executing within tuple spaces can be duplicated. If tuple spaces can be removed via

they can also be read via

implying that the processes executing within

can be duplicated. Security is another problem. If one spawns a process one expects the process to execute unless something is done to prevent it. However the model as it is, allows other processes to retrieve the tuple space in question and consequently stop the spawned processes. This also shows that the separation of concerns is not perfect as it is impossible to spawn a process which is inaccessible to any other process. One could say that by spawning a process in a local tuple space this would not happen, but what is a local tuple space if its handle is just a simple string? Still, one should be able to describe how the process will be resumed. Continuation passing style can be used to this end in some programming languages but not 2

A tuple space containing at least one active tuple.


22

in others. In order to resume a process its state has also to be stored and this is not a trivial task as it may involve storing the state of several other structures which may be side effected. Although Gelernter tries to envision all possible applications of multiple tuple space he does not mention that the introduction of multiple tuple spaces in open implementations requires garbage collection. When talking about persistent tuple spaces Gelernter says: “A persistent tuple space is a tuple space that, once created via , continues to exist until it is explicitly destroyed via

. The fact that all processes

inside such a tuple space have terminated does not mean, in other words, that the tuple space disappears.” In reality one can read from this statement that tuple spaces are assumed to be destroyed or to disappear — a sort of explicit garbage collection. However this is not feasible in open distributed implementations. In order to safely destroy a tuple space via the

primitive a process has to be aware of whether or not other processes are

accessing this tuple space. Except for the case when a tuple space is local this cannot be done safely in L INDA as processes do not have mechanisms for this purpose. Clearly this problem can also be used as an argument against tuple spaces being firstclass objects — what happens if another process is accessing a tuple space which is removed via the primitive

?

Apparently L INDA 3 assumes that a tuple space, if not local, is a persistent object. In open systems to consider all non-local tuple spaces persistent by default can lead the system to a memory exception; but as L INDA 3 is a closed implementation it is feasible to use some sort of compile-time analysis. Once again, compile-time analysis cannot be used in open systems as a compile-time concept for the entire system is not defined. The garbage collection presented in this thesis can be adapted to the case when tuple spaces are stored within others, but the extension to tuples spaces as first-class objects is beyond the scope of this thesis. Gelernter [Gel89] discusses some interesting considerations with regards to the


23

use of tuple space as an I/O mechanism. His view is that of a file system being replaced by persistent tuple spaces with directions given on how one should go about doing it; if is a persistent tuple space one can execute for instance where mimics the index in a file. It is argued that this abstraction eliminates some overhead in the L INDA system as the data is already in the L INDA format and conversion is unnecessary. One step further is taken when it is argued that with persistent tuple spaces a separate file system is unnecessary. Multiple tuple spaces with persistence can do the job of a file system that, without offering any advantage to users, provides at least a unified semantic framework. Although Gelernter’s ideas were not the original motivation of this thesis to tackle the distributed I/O problem in L INDA, his ideas have been investigated further and several issues that were not even mentioned before have been considered. Elimination of a file system is completely impractical as the world is not L INDA like. If L INDA is to deal with files, as this thesis proposes it must, the original file system has to be preserved and only an abstraction of it should be brought into L INDA. The introduction of I/O in the model has indeed a visible advantage to users who can become completely unaware of location of the resources therefore extending the abstraction of location already in the model. In open systems it may be interesting to have the property of being a persistent tuple space defined after the creation of the tuple space and not necessarily at the time of its creation — a tuple space might become persistent later in the execution of the program. This proposal makes this possible with the introduction of a new primitive which not only associates a tuple space to a file but makes this tuple space persistent.

2.1.3 Melinda M ELINDA [Hup90] is based on the multiple tuple-space L INDA model which goes a step further than Gelernter’s L INDA 3 [Gel89]. In terms of implementation, M ELINDA considers not only tuple spaces, but also processes, as first-class objects. It is possible to create, read, snapshot, suspend, destroy, archive or re-activate either processes or tuple spaces.


24

Although it might be subtle to distinguish whether a modification is occurring at the model or at the implementation level, it is assumed here that M ELINDA, like L INDA 3, defines the same triple ( , , ) and therefore is based on the same model. On the implementation level M ELINDA assumes the existence of a tuple

space which contains everything and is known to every process. Since L INDA 3 already assumes tuple spaces as first-class objects, the basic difference encountered in M ELINDA is that the latter also implements first-class processes. As argued in Section 2.1.2 it is difficult to deal with tuple spaces when they are active; dealing with active processes is at least as hard; in fact the problems are the same only more explicit in the latter case. To overcome the problem of dealing with active objects, M ELINDA introduces the type-modifier which can be used with any type . An object of type is defined as being of the type when passive. It should be noticed that the role of the type modifier is to allow operations

with active processes. For instance, if a process

of the type

is being executed

into a tuple space one is able to:

Suspend an execution: Snapshot a process:

Store an execution:

Resume an execution: .

.

.

, where

, where

is of the type

is of the type

.

The role of the type modifier

is clear but some questions, concerned with

the meaning of some operations, remain unanswered. Hupfer argues that when an active tuple space is suspended, all of its active tasks are suspended. Due to the L INDA asynchronism it is impossible to guarantee immediate termination of every task; the author’s answer for this is that the tasks will stop as soon as possible. But what is the meaning of this statement? Suppose two instructions in M ELINDA as shown in Code 2.1. The shown causes an image of the active tuple space to be assigned to

operation

. According to


25

Code 2.1 Part of a M ELINDA program

#

!"

Hupfer, these two lines in Code 2.1 should represent a duplication of a tuple space with all its contents, including active tuples. Now one can come back to the problem of stopping as soon as possible. What is really ed if some tasks have not been stopped yet?, or does the block waiting for all processes to stop? The of should be taken as the continuation [HF87] of the active tuple space

, but this

continuation is not well defined and the paper raises some doubts with regards to what is this continuation. Except for these interpretative problems of the model’s semantics, the assumption that processes are first-class objects should be implemented with modifications in the model in order to avoid users stopping others’ processes deliberately (as explained in Section 2.1.2). Maybe the inclusion of concepts like access rights to tuple spaces and tuples [Woo99] could help. The implementation of M ELINDA recognises different tuple spaces according to their classification: persistent or not. A tuple space has to be defined as persistent if it is required to live outside the scope of the creator. For the context of garbage collection such information is important as it is guaranteed that every non-persistent tuple is local and can be garbage collected when out of scope. Contrariwise the assumption of having users deleting persistent tuple spaces explicitly as argued by Hupfer [Hup90] is not an option. It is complex for users to decide about the usefulness of a tuple space as this would require sufficient knowledge of this tuple space’s situation in the entire system which is not always possible. The garbage-collection scheme proposed in this thesis would fit M ELINDA but being a closed implementation it will certainly do better with a compile-time type of optimisation. Distributed I/O is not considered here and M ELINDA does not elaborate on the


26

basic ideas of L INDA 3.

2.1.4 York Linda 1 The York L INDA kernel I [DWR95] is an open implementation of a variant of the multiple tuple-space L INDA model. The model implemented is significantly different from the standard multiple tuple-space model to be considered a new model. The entities, , are the same as in the standard multiple tuple-space model but the media,

, and

mainly the laws, , are different. The media, although still based on the tuple-space idea is affected by the non-existence of a tuple space. Remember that the

media is the way entities interact and surely in this case it is different — the flat structure of tuple spaces also contribute to this difference. The laws have been modified because bulk operations now exist; this introduces a new way in which processes interact with the media. The implementation includes a new primitive called

[BWA94] that realizes the concept of bulk movement of tuples. The primitive

can move a set of tuples matching a given template from one tuple space to another. Its informal semantics is as follows:

: It moves all tuples matching the

the tuple-space source, , to the tuple-space destination,

from

, and returns

the number of tuples moved. This semantics is somehow imprecise with active tuple spaces; the definition of “all tuples” in the context of an active tuple space is not clear. Despite being a multiple tuple-space implementation, the model supposes no Universal Tuple Space ( )3 which would be a tuple space known by every process similar to the tuple space proposed by Gelernter [Gel89]; every tuple space has to be previously defined. Hence, all primitives that deal with tuple spaces have to define explicitly the tuple space involved in the operation. The structure of this model is completely flat as depicted in Figure 2.4. Although multiple tuple spaces exist they are not first-class objects and they are not stored within others. 3

The paper refers to this tuple space as Global Tuple Space (

).


27

Figure 2.4 Representation of the flat tuple-space structure proposed for the York L IN DA 1.

ts3

ts5 ts6

ts2

ts1

ts4

The

implementation is such that processes are supposed to execute outside

the tuple spaces, but every process is supposed to turn into a tuple inside a previously defined tuple space; the tuple space is defined in the primitive which means that all processes are linked to some tuple space. At this point one question arises; if every process is linked to a tuple space, to which tuple space are the main processes (processes started outside the L INDA system) linked? According to Douglas et al. these processes are an exception to the rule and do not turn into a tuple when they finish their execution. This to some extent breaks the seamlessness characteristic of the abstraction since the concept is not uniform in the entire system. The lack of a makes this implementation not suitable for general open systems. Observe that without a global resource (a tuple space) two processes running independently cannot get to communicate with each other. The separation of environments here is too restrictive to implement some large, general purpose applications. Processes can only communicate among themselves if they have at least an ancestor in common (in the hierarchy of process spawning).


28

In terms of distributed I/O the York L INDA 1 does not allow spawned processes to do I/O operations; only the main processes of computation are able to do I/O. The reason for this restriction is that the authors envision migration of processes to different operating systems which may compose the open system in question. Migration of processes causes an I/O problem if the underlying environment is heterogeneous. Chapter 5 shows that this is one of the motivations for bringing distributed I/O into the L INDA model. The -ordering problem is highlighted in this proposal. This problem is quite relevant and has been forgotten by most implementations of L INDA. The -ordering guarantees the suitableness of the implementation in coordination tasks. The implementation guarantees that of a series of tuples being stored in a tuple space any pair of tuples will appear in this tuple space in the same order they were sent. In other words, the order of arrival of tuples into tuple spaces has to be the same order that they were sent via . Later on in this dissertation it is shown that a variation of this situation is required in the proposal described. The authors argue that tuple spaces are first-class objects in their proposal but in fact, by the description given in the paper, it seems that handles, and not tuple spaces, are first-class objects. There is an important difference between these two. Operations with handles are easily implemented, since handles are pointers to tuple spaces. On the other hand, operations on tuple spaces themselves are difficult and a good semantics for these operations is missing. Garbage collection is definitely an issue here. In an open implementation compiletime optimisations are not possible. On the other hand, the lack of a universal tuple space causes the system to be separated into disjoint groups of processes and tuple spaces. This means that the garbage collector has to account for this and basically run on each disjoint group independently. In Chapter 4 when the scheme for garbage collection is described the idea above will become clearer.

2.1.5 York Linda 2 The York L INDA 2 [RW96] is another open implementation of L INDA. Generally speaking, this implementation uses the same model as the York L INDA 1 with the d-


29

Figure 2.5 Representation of the flat tuple-space structure proposed for the York L IN DA 2 UTS

ts3

ts2

ts5 ts1

ts4

ts6

Non-local Tuple Spaces

ifference that has been introduced as shown in Figure 2.5. The implementation assumes that handles, but not tuple spaces, are first-class objects and preserves the -ordering feature. Also, the same concepts in terms of I/O of the York L INDA 1 are

implemented here. York L INDA 2 might be seen as an improved implementation of York L INDA 1. This implementation introduces some important new ideas; other than paying attention to the need for , the authors tackle two problems: the so-called multiple-

prob-

lem and performance by improving the organization of the tuple-space structure. After the proposal of

the idea of bulk primitives becomes very much alive.

During the pursuit for a good semantics for used to replace the problematic

and

primitives, it was observed that another

operation was necessary to fill a gap left by the multiple-

and how this primitive could be

primitive. The problem, called the

problem, occurs when a given process has to repeatedly perform a

of

a tuple from a tuple space. Due to the non-deterministic characteristics of L INDA’s matching mechanism, the process can keep reading the same tuple over and over. If removal of the tuples is not an option, another primitive is necessary to perform this operation;

was then proposed to this end. This primitive copies a


30

set of tuples matching a given template from one tuple space to another; it can solve the multiple-

problem by copying the tuples to another tuple space and on this

tuple space remove the tuples one by one via

. This would not side-effect the other

processes which keep accessing the original tuple space. The semantics of

is as follows:

: It copies all tuples matching the

from the tuple-space source, , to the tuple-space destination,

, and re-

turns the number of tuples copied. The second and most important modification is the differentiation between local and global tuple spaces where a local tuple space is known only by one process. The authors implement a simple idea where a tuple space remains local until its handle is passed to some other process via a tuple space. At this point the tuple space becomes global. By using this simple rule they managed to improve the performance of the implementation by storing the local tuple spaces locally to the processes using it. Figure 2.5 shows non-local tuple spaces marked. With respect to garbage collection, information of this kind is useful since better decisions can be made based on it — if a tuple space is local and the process accessing it terminates, this tuple space can automatically be garbage collected.

2.1.6 PLinda PLinda (Persistent L INDA) [JLSW96] is a fault-tolerant L INDA implementation. In the context of this dissertation the fault-tolerant properties of PLinda are not considered. In fact PLinda is only reviewed here because persistency is an issue for the distributed I/O scheme described in Chapter 5. PLinda is said to be persistent because the system is fault-tolerant. Persistency in this thesis is on the tuple-space level and occurs as a consequence of the I/O proposal. Fault-tolerance is completely related to implementation — the model level is not affected. PLinda implements the same multiple tuple-space model as implemented in L INDA 3. In fact Jeong et al. extend the standard single tuple-space L INDA but the result is basically the same. The multiple tuple-space idea is achieved through


31

the creation of tuple groups; each group has one unique id which is a first-class object and consequently can be stored into the global tuple space 4 or into other tuple groups. The pictorial view of this model is exactly the one in Figure 2.3, in fact tuple groups are tuple spaces in the L INDA 3 nomenclature — PLinda uses the name tuple space only to refer to the tuple space ( ) calling the others tuple groups. Some of the problems in the L INDA 3 implementation were addressed here. For instance, neither the tuple space nor tuple groups are first-class objects making stronger the argument that such an assumption is not realistic. They also overcome the separation of concerns by having the handle being generated by the server. This improves the idea of separation of concerns as it becomes more difficult to guess a handle — nothing is said about handle uniqueness. Fault-tolerance is achieved by using transactions; PLinda adds the operations and which define the start and end of a transaction; by using this sort of programming style fault-tolerance is achieved. In addition to the operations added to the model to provide fault-tolerance, a primitive to delete tuple groups is implemented. The

primitive takes as an argument a tuple group id to

kill the group. However, the view taken in this thesis is that the task of deleting tuple groups should not rely on users. Since the users are not always aware of the entire system situation the use of such a primitive could lead the system to an inconsistent situation. Still, PLinda is a closed implementation and compile-time analysis can be used to verify if a destroy-group is likely to cause side-effects. In PLinda processes can migrate but nothing is said about the I/O problem that this may cause. The possibility of process migration is in fact one of the motivations for including I/O within L INDA. Having described the problems, it is fair to say that PLinda does not suffer the consequences of them because PLinda is a closed implementation. This means that I/O redirection, tuple-space deletion and handle uniqueness, can be dealt with during compile-time solving any possible inconsistency. 4 The authors do not refer explicitly to the existence of such a tuple space but they seem to distinguish tuple groups from a unique tuple space (probably global).


32

Figure 2.6 Formula of time costs of an original L INDA primitive.

!"# $ !%& )*+,-./ ('

2.1.7 Bonita Bonita [RW97] is a variation on L INDA where the primitives provide asynchronous access to tuple spaces (from within a process) instead of synchronous as provided by the original L INDA primitives. This generally means that the model implemented in Bonita is different as the laws, , have been modified. Indeed the model implemented here is quite similar to the model implemented by the York L INDA 2 only with the laws modified. Rowstron and Wood are motivated by the current trend in the L INDA community to produce better semantics for primitives and alter them to be better applied to distributed coordination. L INDA was first proposed as a model for parallel computation, only later was it noticed that it also produces good results in distributed environments. L INDA primitives therefore are optimised for parallel computing rather than distributed computing. It has been observed that in open distributed implementation the costs associated with L INDA primitives are due to the time taken to send and receive a message over the network. Bonita tries to use the pipeline concept using the time a message spent travelling on the network. Take for instance the case of the

primitive. The semantics of

is that the pro-

cess executing it will block in the case where a tuple is not available in the tuple space. It should be noted that this is not true in the distributed environment as the processes will block for some time even when a matching tuple is available. In practice the time cost of a L INDA primitive is generally described as the formula in Figure 2.6. In Figure 2.6 each term of the formula identifies a specific cost regardless of how an open L INDA system is implemented.

: The time taken to construct the message. The local process has to gath-

er all the information necessary, create a tuple/template and arrange this in a message which will be used in the communication.


&

33

: The time taken by a message sent to the kernel in the communication channels.

: The time that a message has to wait before it is processed by the L INDA kernel.

: The processing time. The time taken for the kernel to process the message

received. This includes unpacking the message, finding a suitable tuple according to the template given and sending the return message to the user process.

"# $

: The time that the primitive remains blocked when a tuple is not available.

% (' : The time taken by a message sent from the kernel to the process in the communication channels.

*+,- /

: The time taken for the user process to extract and interpret the message

sent by the L INDA kernel. This cost is measured from the user point of view and not from the server point of view. This means that an costs only

and the other costs involved

( ) only affect the server (kernel) side. From the user process

point of view, the cost of is only

because it packs the message, sends it to

the kernel and carries on. The aim in Bonita is try to make all primitives take approx-

.

*+,-./

imately

and parallelise the other cost with local computation.

Before describing the improved set of primitives, the authors introduce a different time cost which becomes necessary to have a realistic account of the cost of this new set of primitives.

$ : The time taken to check locally if a message from the kernel has arrived. Given the cost above, a new set of primitives is proposed so as to achieve this goal. The primitives can preserve the functionality of L INDA primitives as they can be easily simulated.

: It or to request a tuple given a . If a

is used either to store a

is given, the primitive will store this tuple within the specified tuple space. If a


34

is given a tuple is being required and it will be either removed from the tuple space ( case) or read from the tuple space (

case). This primitive always takes

. Even when a tuple is required the

process is not blocked waiting for it.

'

: This

primitive manages bulk movement of tuples between tuple spaces. A request is to be either sent to the kernel requiring all tuples which match the moved from

to

( case) or copied from

case). This primitive also takes only

.

to

(

. The process is not

blocked and the result of how many tuples were moved or copied is retrieved later.

: This primitive tests to see if the result associated with the given

is

already locally available. This primitive is non-blocking and returns or $ depending on the result. The cost of this primitive is independent of the result. '

: This is the only blocking primitive of Bonita. It blocks while waiting

for the associated result to arrive locally. This primitive, when making use of this new method, that is, not trying to simulate the L INDA primitive, has the cost of

$ "# $ *+,-./

. The parallelism comes from the fact that the

& ! cost relative to

% (' can be parallelised with local

computation. Code 2.2 Bonita simulation of

.

! !"#$&%' ()" * ! + "

#

As said before the Bonita primitives can mimic allmost all the L INDA primitives, the exceptions being

and

. Rowstron and Wood argue that these primitives are


35

Code 2.3 L INDA program which stores one hundred tuples and removes them afterwards.

#

+ + !&) " + ! ! + !"# % # + ! ! !"# * &%!" #

Code 2.4 Bonita implementation of L INDA example in Code 2.3.

+

! ! !) ( !"# *%!"

# +

#

#

! ! % ) ! % &# "

+ + !&) " + ! ! + *% #

2.1. Coordination Models/Implementations unnecessary as the primitives

36

and

can be used instead. Code 2.2 shows how the new set of primitives can simulate an . The cost of the simulation shown in Code 2.2 is the same as an . But as argued,

there are better ways to use the Bonita primitives in order to parallelise operations. Suppose for instance a L INDA code example as in Code 2.3 which stores one hundred tuple within and them removes them one by one. Using the costs described by the authors the example in Code 2.3 has time cost

&

$ & "# $ *+,- / . This can be greatly reduced when using Bonita primitives. Code 2.4 shows how the L INDA code in Code 2.3 can be implemented more efficiently. Using the same account scheme the cost of the Bonita example in Code 2.4 is *+,-./

$ $ "# $ . Clearly the improvement cannot be guaranteed for all cases, but this is what is expected as shown by Rowstron [Row97]. Bonita is clearly a multiple tuple-space system and as argued by the authors themselves a model for open distributed system. Garbage collection is not addressed in their papers and any improvement in terms of speed is irrelevant if the system stops due to an exception. Some might place garbage collection in the fault-tolerance area. However garbage collection falls on a more basic area. Fault tolerance is when software tries to achieve something even when the hardware is faulty whereas garbage collection is a scheme that tries to make the software itself reliable to run for a long period of time. Distributed I/O is also not addressed. Bonita proposes a new way to access tuple spaces. Any flaw that a L INDA system presents in terms of handling I/O is also present in Bonita.

2.1.8 Bauhaus L INDA [CGZ95] was originally proposed so as to fill some gaps left by the

multiple tuple space L INDA model [Gel89]. Carriero et al. argue that the multiple tuple spaces lacked at least two important characteristics: multiple first-class tuple spaces and operators to unify the way objects stored with tuple spaces (processes, tuples and tuple spaces) are dealt with.


37

The model underlying is significantly different from the original multiple tuple-space model. First, the entities, , are still tuples and processes, both now called multisets (passive and active), but the distinction between them disappears from the point of view of the model since the laws apply for both of them; the media,

, is now also referred to as a multiset; in fact anything in is a multiset;

the laws, , reflect the changes described so far; as multiset is the sole structure the

matching mechanism is replaced by a multiset inclusion operation. The authors argue that is more powerful and at the same time simpler than L INDA. This is the case because can be seen as a generalization of the multiple tuple-space idea: Tuples and tuple spaces as multisets: replaces the ordered tuples by unordered multisets; at the same time, tuple spaces are also replaced by unordered multisets. This surely unifies the operations with tuple space and tuple; operations with tuple spaces (removal of objects, adding of objects) are now applicable to tuples and operations with tuples (using the primitives) are applicable to tuple spaces. Furthermore, the unification of concepts naturally implements a hierarchy of multiple tuple spaces. Templates as multisets: Templates are also superseded by multisets. The semantics of the primitives in is also different as described later in this section. Basically, the matching mechanism has been replaced by operations over multisets. Processes as multisets: Processes are active tuples and therefore are also represented as multisets (active multisets). This automatically makes processes firstclass objects. Yet, the same primitives are used with passive and active multisets. The semantics of the primitives should also be changed. An important concept in in order to understand the semantics of the primitives is the location of a process within the structure of tuple spaces. A process is always executing within some multiset which is the scope of the primitives executed in this process; a process


38

can hop from one multiset to another but it always operates only in the multiset it is currently within. The general semantics of the primitives are as follows:

: This primitive puts the object in the multiset containing the object

. As said before only the multisets which are in the same hierarchical level as the process executing the primitive are considered. Both and

can be empty

or composed of more than one element.

: The idea here is that a multiset will be removed and assigned

to . Once again, only the multisets on the same level of the process are considered by the primitive; from those multisets that contain , one chosen nondeterministically is removed and returned.

: This primitive behaves like an

but the multiset returned

is not removed, it is just read. Another important difference between

and

is that since processes can be stored within multisets, their behaviour with

relation to the active processes is different. If a primitive

is used the active

objects remain active outside the coordination structure; in the

case only

an image of each active object with an indication of its state at the time of the execution of the primitive is returned; the original active object keeps executing within the coordination structure.

: This primitive moves the process executing the primitive down a level

to a multiset containing ; a special symbol, , is used to represent the process moving up a level. This primitive had to be implemented because handles do not exist in ; in order to get access to a multiset a process had to move up and down in the hierarchy. Figure 2.7 shows a pictorial view of a coordination structure; this figure

could also be written as Now Figure 2.8 shows the situation of the coordination structure after three different executions of primitives. Figure 2.8(a) after execution of an by ; observe that this is not the only possible situation. Figure 2.8(b) after the execution of an by the process

. In Figure 2.8(c) after the execution of a

by .


39

Figure 2.7 Representation of the coordination structure in .

3.47 2

’a’ P1

4

P3 "Paul"

2 P2

The idea of unifying the treatment of tuples, tuple spaces and processes in is interesting and is certainly an advance for the distributed coordination area. Still, the problems of dealing with active multisets persist here. Observe for example what Carriero et al. say: “An

command is somewhat similar to the

command. However, the

returned should maintain its active processes as active processes, and it should be removed from the coordination structure” The obvious question here is “How?”. As discussed before, it is not trivial to solve this problem; side effects which are difficult to solve can happen; a multiset can contain other multiset with active objects which in turn contain multisets with active objects and so on. , although based on L INDA, implements quite a different model. There-

fore the garbage-collection scheme described in Chapter 4 does not apply to directly. In fact, the reason for this incompatibility is not the method introduced in this thesis but the way is defined. Carriero et al. do not consider garbage collection in as it is almost certain to be inconceivable. Multisets in


40

Figure 2.8 An example illustrating the execution of primitives in .

3.47

3.47 2

2

’a’ P1

4

P1

4

P3

P3 "Paul"

"Paul"

2 P4 P2

(a)

2 P4 P2

(b)

3.47 2 P1

4

P3 "Paul"

2 P4

P2

(c) "! # $%&(')

have the same “non-unique identifiability” characteristic as tuples in L INDA. This brings a concern with regards to the applicability of ideas in open systems; as any multiset is persistent it is almost as if one had a L INDA system composed of

and several tuples in it — these tuples can never be collected. Although some

internal links could be created the availability of a multiset to any process would still make garbage collection impossible. Carriero et al. do not address I/O in but the method described in Chapter 5 could be adapted to fit here; the lack of a true separation of concerns would diminish the validity of the proposal — a file when abstracted as a multiset would lose its access rights and become available to all processes.


41

2.1.9 Laura Laura [Tol93, Tol98] is a service-based coordination language for open distributed systems. Laura understands that processes in open systems need to coordinate in order to use and offer services. Due to the nature of open systems, coordination has to be uncoupled, that is, processes should not coordinate via direct communication. Direct communication assumes that the system has a static nature and processes perform repeated tasks during a long period of time. This is surely not true in open systems as their dynamic nature makes it impossible for a known process to still be executing at some time in the future. Establishing connections in open systems can lead to errors as the other process might leave the system at anytime. User uncoupling and offering of services is what has driven the proposal of Laura. Although being only a variation of L INDA, renaming some of the primitives, changing the matching mechanism to one specific to services, Laura offers a good insight on how to use coordination models in distributed service-based paradigms. Laura is composed of a service space, equivalent to in L INDA in which forms (special tuples specific to service-based communication) are stored. The processes are provided with variations of the L INDA primitives so that they can offer and request a service and get the result of a service.

vice

: Adds a request-form for the ser-

, the operation ( ) required to be performed by the service; the arguments ( ) which can either be formal or actuals as re-

quired by the operation. Generally at least one of the arguments is a formal and represents the result value a process is looking for. This is a blocking primitive; the primitive waits for an offer-form to appear in the service space and for the process to write the result-form in it.

: Adds a form in the service space offer-

. This is also a blocking primitive in case a request-form (added by the primitive ) does not exist in the ser-

ing a service with the given

vice space. When it exists the process answers the request and outputs the result.


42

: This is the primitive used to store result-

forms within the service space and consequently unblock a process executing the primitive

.

The matching mechanism differs from the original L INDA mechanism in order to work better with forms. Each form is composed of a set of operation signatures (an interface) defined as the name of the operation, their argument types and result type. The matching mechanism is such that if a request-form is added, the kernel will try to find a service-form (a service offer) so that their interfaces match. Therefore Laura does not use names when trying to identify a service using instead only their interface signature. Given two forms, a service-form matches a request-form if the type of the interface in the service-form is a subtype of the interface in the request-form. Generally speaking, this means that services can serve less sophisticated requests. Code 2.5 A service in large specialised whisky shop offering several operations.

) !

! )! ) ) ! ) )

) + &

*

( )

Code 2.6 A service in an off-licence shop offering one operation

)!

! !

) *

Suppose the definitions of services in Code 2.5 and 2.6. Code 2.5 describe a service provided by a large whisky shop; several operations are provided as users of this


43

service might require extra information about a given malt. However, an off-licence shop, as in Code 2.6, would offer only one operation — the only one required by its users. If a service-form for the large whisky shop is put in the service space offering a service it can match a request form of an off-licence shop as the

operation

is available in both. Although proposed for open systems, the use of only one service space makes the implementation of large realistic application hard to achieve. In large systems the complexity of managing this sole service space is huge; distribution of concerns is impossible. In short, Laura falls into the single tuple-space trap which has motivated several L INDA-like models to provide a multiple tuple-space environment. In terms of garbage collection the use of a single service space means that garbage collection cannot be done. Forms are not identifiable uniquely and in open systems it is impossible to know if a given form stored within the service space will be required in the future. In the real world, this service space might tend to grow in size and lead the system to memory exception. However, should a multiple service space version of Laura appear, garbage collection has to be implemented. The scheme proposed in this thesis would be applied to Laura with little modification as long as the model implemented does not significantly differ from the multiple tuple-space model [Gel89]. As for distributed I/O, Laura can easily be used to implement a distributed I/O system — services can be offered to do distributed I/O. Such an implementation, though, would not be in accordance with the fact that distributed I/O is a coordination problem and also with the fact that location should be transparent; I/O implemented as services breaks one of the most important characteristics of Laura: location transparency.

2.1.10 The PageSpace Effort The PageSpace [CTV97] is an effort to integrate WWW and coordination models in order to provide real distributed capabilities to applications running on the web. Basically the Web does not provide support for distributed, cooperative applications where active entities are necessary. One of the main questions here is “Why would L INDA improve the existent envi-


44

ronment?” or in other words “ What makes L INDA attractive for distributed application over the Web?”. Ciancarini et al. have identified points showing the attractiveness of coordination models like L INDA to the WWW environment: Use of Tuple Spaces: The concept of tuple spaces in a distributed environment provided the notion of asynchrony and concurrency — important points for a better use of the WWW — for free. Concurrency is achieved by having processes using the tuple space as a communication channel and synchrony is achieved due to the shared memory flavour of tuple spaces. Message passing (the opposite of shared memory) is extremely synchronised as any communication involves a synchronization. Uncoupled structure: Agents5 are unaware of physical location bringing the WWW environment up a level in terms of abstraction (space uncoupling). The concept of time does not exist in L INDA-like coordination models — processes can communicate even if they do not exist at the same time (time uncoupling). Associative Addressing: Data in a tuple space is retrieved by contents and not by “name”. The WWW can greatly take advantage of this characteristic by implementing this scheme in the way pages are retrieved from URLs. Orthogonality: Coordination is a concept orthogonal to computation, therefore using a coordination model in the WWW will have no influence on the way computational programs behave on the Web. The availability of the web is causing users to demand more sophisticated applications (including distributed applications) to be integrated between themselves. This is where coordination models play a central role. Such applications would became a bunch of useless programs if their integration is not coordinated properly. PageSpace is proposed to provide a midware platform that together with the Web can integrate distributed applications. Mechanisms for transforming the Web into a platform suitable for distributed computing are widely available. The Common Gateway Interface (CGI) for instance 5

The name agents here is used to represent both client and server processes.


45

is a mechanism for interfacing external applications with information servers, such as HTTP or Web servers. A plain HTML document that the Web daemon retrieves is static, which means it exists in a constant state: a text file that does not change. A CGI program, on the other hand, is executed in real-time, so that it can output dynamic information. It should be noticed though, that CGI provides no means for creating distributed application; as a matter of fact, in an application interfaced via CGI only the server part is doing some processing. Each time a client requests the URL corresponding to a CGI program, the server will execute it at real-time. The clients limit themselves to mere query formulation using HTML. The other side of the story occurs when most of the activity is done by clients; this is the case with applets and plug-ins for instance. Despite the impression of being called distributed, they are generally stand-alone applications and perform no distributed cooperative work. In the cases where they do perform some cooperative distributed work, problems like security or a complete abandonment of the Web as a central unifying environment happen. PageSpace poses as a midware environment that can be used to coordinate distributed agents (clients and servers) but integrated with the functionalities of the WWW (like hypermedia and the unified view it offers). PageSpace is proposed as a L INDA-like environment. However, Ciancarini et al. claim that the original L INDA model (as defined in Section 2.1.1) is not suitable for open computing since it becomes hard to support multiple applications sharing a single tuple space. They propose to expand the basic coordination behaviour of L INDA by allowing declarative rules on the coordination as implemented in ShaDE [CCR95] and to use the concept of the Laura coordination language [Tol96] where tuple spaces are replaced by service spaces containing forms describing service offers, requests and results and where matching is performed based on an interface included in each form (see Section 2.1.9). The PageSpace architecture is composed of five classes of agents. These agents run concurrently in cooperation to provide the functionalities required by the WWW and communicate between themselves using a shared coordination media. The classes are as follow:


46

User Interface Agents ( Agents) : These agents represent the front-end of applications. Depending on how sophisticated is the agent, different activities are supported. The Web is composed of a plethora of activities taking place at the client end, server end and even somewhere in the middle — in the midware. Interface agents support all of these activities depending on the interface. Avatars ( Agents): These are the persistent representation of user agents within the PageSpace. They are to provide the asynchrony as well as time uncoupling. From the point of view of the PageSpace, avatars are like agents but they stop users from being required to be logged into the system during all of the execution. Avatars, while not connected to users have to take the role of the user agents and return to them based on the communication behaviour set by the user: on-demand, periodically or continuously. Application Agents ( Agents): They are the ones that constitute the distributed applications interacting with other agents as well as providing and using services. Application agents are the ones that do the distributed processing; they can be specific user-interface agents, agents providing applications’ specific functionalities or server agents providing generic services. Kernel Agents ( Agents): They provide all the coordination mechanisms to applica

tion agents. They are implemented in Java and exist at each location executing PageSpace using possibly different Java virtual machines (characterising PageSpace as an open systems). Kernel agents are the central part of the coordination environment on each PageSpace machine providing access to the PageSpace via users own avatar; managing home agent operations like spawning a new application agent; managing application agents as they are all executed locally in the kernel; implementing the coordination primitives, matching rules and the shared space; and implementing the distributed architecture by having each kernel with an interface to other kernel agents. Gateway Agents ( Agents): Similar to application agents, gateways differ only be

cause they provide services that interact with external entities. They also provide communication to other coordination systems. Gateway agents serve as


47

Figure 2.9 The PageSpace architecture (based on [CTV97]) USERS’ PROCESSES ACTORSPACE α

β

δ

α

β

β δ

δ

ε

α

ζ

ζ

PAGESPACE LINDA

wrapper agents for external applications wanting to access services in the PageSpace system. Figure 2.9 depicts the PageSpace system ( environment). In this example three user processes (perhaps Web Browsers) are using PageSpace via an - communication.

agents (representation of users within PageSpace) communicate with agents

using the services offered by them. Also, there are communications among

agents

since they can both use and offer services. PageSpace is a distributed open system and as such should provide means of collecting unused data; repositories (like tuple spaces) are created and should be deleted when they become unnecessary in the system. Additionally,

agents might need to

be collected as services can cease to exist at anytime. Garbage collection of active objects like

agents can also be achieved in the garbage-collection scheme described

in Chapter 4 given that some properties are guaranteed. Distributed I/O is also not addressed by PageSpace but it is clearly important. Quoting the authors in [CKTV96] they say: “The user and the programmer is provided with an abstract conception in which distribution and heterogeneity are invisible...”


48

This means to say that the users and programmers always have a false impression that all operations are local; when I/O is considered this impression is broken. I/O is location-based and breaks any abstraction of distribution. Menezes and Wood [MW98a] have shown that I/O is a coordination problem; Chapter 5 explores this proposal in more detail showing how can I/O be embedded within L INDA-like coordination implementations. Although the project has been terminated now, PageSpace has greatly contributed to the understanding of how coordination mechanism can be used to enhance the quality of services offered on the Web.

2.1.11 JavaSpaces Implemented by Sun Microsystems, JavaSpaces [Sun97b] represents the first endorsement of a company to tuple-based communication abstractions. JavaSpaces is proof that coordination technology has an impact in the way distributed systems are designed. JavaSpaces cannot be considered a model but only a commercial open embedding of L INDA within Java. Yet, JavaSpaces tries to present itself as a new model that differs significantly from L INDA. However when its specification is analysed it becomes clear that the differences are in reality minimal. JavaSpaces does not introduce any new concept at the model level but its implementation uses several aspects of the Java language (Distributed Events [Sun98a], Transactions [Sun98c], Lease [Sun98b]) making JavaSpaces an interesting embedding. The model implemented by JavaSpaces is very similar to the model implemented by the York L INDA 2 (see Section 2.1.5); a multiple tuple-space model, not necessarily, but ideally flat. In theory JavaSpaces objects can be stored within JavaSpaces but as they are not active objects this would not bring many problems to the system in terms of semantics. Yet, it is not clear in the specification if the class defines a space or just a handle. Garbage collection is surely an issue here. JavaSpace is an open system although


49

restricted to the use of Java processes only. Handles exist 6 and can be passed to other processes and despite the use of Remote Method Invocation (RMI), the system has no control on the passing of handles. RMI is used to access methods of a remote object but cannot be used to gain information about the usefulness of a distributed object. Apparently a distributed garbage-collection system is being developed which would tackle garbage collection of JavaSpaces [Gro98]; this system would collect objects across different Java Virtual Machines. It is important that the garbage-collection scheme proposed in this report is not misunderstood. Garbage collection of JavaSpaces would be provided by the Java language which is not ideal; general purpose open systems include different host languages and a garbage-collection scheme provided by a particular host language would not be able to work in heterogeneous systems. Garbage collection of coordination objects should be provided within the coordination dimension and not by any computational host language. Distributed I/O is not considered by JavaSpaces but can surely be brought inside the system by adapting the scheme described in Chapter 5.

2.1.12 T Spaces Recently proposed as the IBM [WMLF98] version of L INDA, T Spaces, learning from other L INDA implementations and analysing the advantages among variations of the multiple tuple-space model, is a good open L INDA implementation. Proposed as a middleware for ubiquitous computing, T Spaces tries to fuse the ease of use of L INDA systems with the power of relational database systems to provide a framework that can be used to seamlessly provide high-quality communication among devices ranging from Tier-0 (anything smaller than laptop computer) to Tier-3 (Mainframes). T Spaces implement a different distributed tuple-space model. Not surprisingly the entities, , are the tuples and the processes; the media, , is composed of multiple

T Spaces (similar to to tuple spaces); the main difference lies on the definition of the laws, . The first interesting change in the laws is the inclusion of access controls;

in T Spaces users can establish access rights at a user or group level. Although not fully explored in T Spaces its inclusion in the model shows at least a concern with 6

The way the primitives are defined indicates that handles are used.


50

this important aspect of open systems; recently access control to open distributed tuple spaces system has been extensively explored where a suitable solution using capabilities is proposed [Woo99]. The effect of access control in the laws are clear; the way processes interact with tuple spaces is now different as a process might not have access to a given tuple space even when its handle is available to this process. Another interesting modification in the laws is that in T Space they gain a dynamic aspect. In addition to the default set of primitives (the representation of the laws) new primitives can be defined dynamically. The default primitives implemented by T Spaces are the basic L INDA primitives plus some additional ones related to query and synchronization. Below is the description of the primitives implemented and their semantics:

: This primitives writes a

object within the tuple space .

: This primitive removes a

tuple space matching the

object given. This

object from the object given is a

template because it can contain formals — non-valued typed objects. This is a blocking primitive, so that the process blocks until a matching

object is

stored within .

: This primitive removes a

object from the Space matching the object given. As opposed to in this primitive does not block. If a matching object is not found is returned.

: The same as but the

object is

not removed. A copy of it is returned.

: The same as but the

object is not removed. A copy of it is returned; if a matching object is not found is returned.

: T Space’s version of

, this primitive re-

turns a tuple containing all the tuples matching the template; the original tuples remain within the tuple space. The difference between this primitive and

is that in

the tuples are copied from one tuple s-


51

pace to another whereas here they are copied to an internal data structure in the form of a

object.

: T Spaces’ version of

, this prim-

itive returns a tuple containing all the tuples matching the template removing the original tuples from the tuple space. Again, this primitive and

differ as the latter moves the matching tuples from one tuple space to another instead of being removed and assigned to a local

: This primitive returns a count of the number of tu ples matching the given .

object.

: This primitive is used to perform atomic synchronisation.

It takes two

objects where the second is a template and atomically swaps them with a matching tuple and template from another .

The set of primitives implemented in T Spaces is quite complete where even a primitive that allows a simple form of rendezvous to be expressed ( ) has been proposed. T Spaces also paid attention to the need for bulk primitives [Row97] with . Yet, T Spaces’ implementation forces the introduction of and tuples to be first-class objects unnecessarily; also, the primitive

is meaning less in a dynamic environment and since a process cannot find out if a

object is being accessed by other people the result given by the primitive cannot be reliably used in any way. Garbage collection in T Space is provided by the Java language. Due to the use of RMI in its implementation and due to Java being the sole language used in T Space, the use of Java’s garbage collector is possible. Still, it is very na¨ıve to think that any language will become the universal standard. In general open systems, the L IN DA

midware would be the only common infrastructure and therefore it is necessary

to have garbage collection of L INDA objects within this infrastructure. The T Spaces specification forecasts that in future implementations a CORBA interface will be implemented to allow T Spaces to communicate with non-Java platforms. If this happens IBM will have to implement garbage collection on the L INDA level as proposed in this thesis.

2.2. Garbage-Collection Schemes

52

Distributed I/O has not been explored in T Spaces but I/O would also have to be implemented in the model; not only because it affects the garbage collection but because it is a coordination problem. T Spaces also assume that in the future implementations clients could be mobile. As explained in Chapter 5 this reinforces the idea that I/O also has to be implemented in the only common infrastructure so that I/O can be redirected independently of process migration.

2.2

Garbage-Collection Schemes

Garbage collection, the process of searching and reclaiming unused memory space, is a hard task and should not be handled by users. Every modern distributed environment should provide transparent garbage collection of objects in a similar way that access to local and remote locations is provided. A garbage-collection algorithm is generally composed of a mutator and a collector [DLM 78], where the collector is generally divided into two phases: the searching and the reclaiming phase. Mutators are those processes which make modifications in the structures of objects, and the collector uses the structures to decide on the situation of the objects. This section describes some garbage-collection algorithms for distributed systems. The number of papers proposing solutions for distributed garbage collection is enormous, but they are based on three different approaches: Reference Counting, Reference Listing and Tracing. The review in this section is based on the three concepts and not on particular papers. The description is general and only common characteristics are highlighted rather than a single-paper characteristics. The aim is to give a good idea of the three alternative approaches with the pros and cons of each one.

2.2.1 Reference Counting The basic idea of reference counting is that each cell has a counter of the number of references to itself and a cell is garbage whenever its counter falls to zero. The naive extension of this concept for a distributed environment consists in the


53

Figure 2.10 Cyclic garbage being created but not detected because the reference counting does not work properly with cyclic structures. a

1

ROOT

b

c

1

1

same idea with the difference that the pointer from a cell to another can be local or remote since the environment is divided into spaces. The reference-counting technique is considered an acyclic solution as it works only when the graph is acyclic. In cyclic structures the method is incomplete and cannot collect cyclic garbage as exemplified in Figure 2.10. In the Figure 2.10 the reference of to is deleted. The counter field of is reduced to and a cyclic garbage structure composed of , ' and is created. Since each one of these nodes has counter field equal to they are not garbage in the eyes of the reference-counting algorithm, but they are truly garbage since no root has access to them. Another drawback of this approach is that there is a potential race condition in the maintenance of the counters. The implementation of increment and decrement messages in an asynchronous distributed environment is difficult to achieve without a race condition; if one is careless a cell can have its counter set to zero without being garbage. Due to the asynchronism of most distributed systems, such race conditions have to be avoided using solutions which increase the overhead in the system such as using acknowledgement messages. The first idea aiming to eliminate the use of increment and decrement messages and use only one type of message was proposed by Bevan [Bev87] and Watson and Watson [WW87] independently. It is known as Weighted Reference Counting. The proposal consists in having each node with two weights: a partial weight and


54

Figure 2.11 Invariant of the weight reference-counting algorithm. TOTAL WEIGHT

PARTIAL WEIGHT

PARTIAL WEIGHTS TO

a total weight. Additionally each cell has a list of partial weights of the cells to which it refers (referred cells). The total weight does not change on the creation of new references to the cell but is decreased when a reference is deleted. Upon the first reference creation, the cell referred to has its total weight initialised with any value greater than zero, and its partial weight is initialised with half of the total weight, being the other half sent to the cell which refers to it (referring cell) which stores the value in its list of partial weights. Then, each time a reference is created the referred cell has its partial weight halved and the other half is sent along with the message to initialise the partial weight of the referring cell. The duplication of a reference is an operation that does not involve the referred cell. The operation consists in having a referring cell sending a copy of its reference to another cell. Upon the duplication the partial weight of the referring cell is halved and the remaining half is used to initialise the partial weight of the new referring cell. Finally, when a pointer is deleted its weight is used to decrement the total weight of the referred cell. Assuming no loss of messages and no duplication Figure 2.11 shows the invariant that has to be guaranteed. According to Plainfossé and Shapiro [PS94], the main problem with this approach is that the initialisation of the total weight is a limitation on the number of pointers

$

to a cell because if the initialisation is made with

only

duplications are possible.

Once a weight becomes one a trivial solution of adding the same value to this cell’s partial weight and to the total weight can be taken. Observe that the invariant in Figure 2.11 is guaranteed but this sum is not always possible because the length of a cell is limited. A better way to overcome this limitation is the use of indirect references [WW87]. When a cell drops its partial weight to one and needs to duplicate, a reference to this object is created with total and partial weight. Now every duplication creates an


55

Figure 2.12 Situation using indirect references in the weight reference counting. a 32 16

RT1

1

RT2

16

b 64 1

indirect pointer as shown in Figure 2.12 which describes a case where

needs to

duplicate its reference to ' but it is unable to do so because its counter is one and cannot be divided anymore. A new cell is therefore created and the duplication is done by making point to which points to ' . The dashed objects represent what is being added in the figure. As showed in Figure 2.12,

points to ' through . If one assumes that

are in different spaces, any failure in the space

would poison

and

, since it

would be unable to reach . This kind of solution can exhibit a domino effect where one failure causes the failure of several spaces [Rud90]. Another drawback is that a chain of indirections can be created looping back to the same space several times adding an enormous overhead to the system. Despite the problems, weighted reference counting is better for distributed systems than the basic approach. The overhead in general tends to be smaller because duplication does not involve the referred cell.

2.2.2 Reference Listing The basic idea of reference listing is to replace the counter with a list of references as shown in Figure 2.13. The concept of what is garbage remains the same. Every node which has an empty list is garbage. The same race-condition problems of reference counting apply to the reference listing because the increment and decrement messages are replaced by create and delete messages. The advantage in this method comes in the reliability of message duplication and loss, but the details of such characteristics are not considered since


56

Figure 2.13 In the reference-listing method, each cell has a list of references as opposed to a counter. RT1

RT2

a

RT3

RT1

RT2

a

3

Reference Counting

RT3

Reference Listing

it is out of the scope of this report. Shapiro et al. [SDP92] propose the SSP Chains (Stub-Scion Pairs) implementation of reference listing. The proposal is for what they call a classical distributed system 7 and avoids race conditions using time-stamped messages. In principle this makes the proposal unsuitable for L INDA as the concept of time-stamps breaks the asynchronous property of L INDA. Another interesting proposal using reference listing was made by Birrell et al. [BEN 93] in the context of network objects – small non-mobile objects which can be referred to from a remote space. In the context of this thesis, the most important point of the Birrell et al. proposal is the way race conditions are avoided. Birrell et al. suppose an ordering of operations in the graph implemented using acknowledgement messages. The L INDA implementation proposed in this thesis would need a similar solution in order to guarantee the absence of race conditions. Like the reference-counting techniques, reference listing is also an acyclic method of garbage collection. Reference-listing techniques are unable to collect cyclic garbage.

2.2.3 Tracing Is spite of being considered better than tracing for distributed systems, reference counting and reference listing still suffer from incompleteness. As explained before, 7

Without shared memory, partial failures and unreliable costly messages.


57

these methods are unable to collect cyclic structures. Tracing-based techniques search the graph marking all reachable nodes from the roots. At the end of the search, all unmarked nodes in the graph are garbage. The main advantage of tracing-based techniques is their suitability for cyclic structures. A tracing algorithm is usually described based on colours; if a cell is black it is marked and white otherwise. An implementation of tracing in distributed environments consists typically in combining local collectors working into local spaces and a global collector looking at the situation of the entire system. Local collectors are responsible for deciding upon the usefulness of local objects whilst the global collector does the same for public objects. The main problem of distributed tracing methods is to maintain the consistency of the graphs when the mutators operate in the graph concurrently with the collector. According to Plainfossé and Shapiro [PS94] in some cases this consistency is not achievable, since the collectors have to rely on snapshots of the structure which can be inconsistent. The first step towards a solution of distributed tracing was taken by Dijkstra et al. [DLM 78] in the proposal of an on-the-fly marking. Despite not being a distributed solution they introduce the idea of having the mutator cooperating with the collector in a concurrent way. They set up a new color to indicate that a cell was just created. Figure 2.14(a) illustrates the problem which occurs when mutator and collector are running concurrently, and Figure 2.14(b) shows the solution proposed by Dijkstra et al. At the end of the algorithm execution each cell could be either black, grey or white. White cells are those unreachable from the roots. Grey cells will be those created since the last collection but missed during the marking (searching phase). Black cells are those useful to the system at the time of the marking. During the reclaiming phase white cells are reclaimed and other colours are whitened. As said before, the proposal of Dijkstra et al. is not distributed and therefore does not solve the consistency problems related to distribution of data. Most consistency problems are caused by race-condition situations. Two common situations are de-


58

Figure 2.14 Illustration of the problem when mutator and collector are running concurrently and one possible solution where they collaborate. RT1

RT2

Mutator and Collector DO NOT Collaborate

RT1

RT2

Mutator and Collector DO Collaborate

scribed by Plainfossé and Shapiro [PS94] as the inconsistent snapshot problem and the message in transit problem. The inconsistent snapshot problem consists in having an erroneous collection because the snapshot, which is taken by the global collector, is done concurrently with the mutator and consequently does not reflect the true (stable) situation of the system. Similarly, the message in transit problem has to do with the snapshot. In this case the snapshot does not take into account messages in transit in the system. It should be noticed that the latency of a system could be variable and the causal order8 of the messages could be lost. The standard solution to end these inconsistencies is to use synchronisation. The use of a barrier of synchronization before taking a snapshot could end the problem but at a high cost. Another solution is the use of time-stamps replacing the marking field, as proposed by Hughes [Hug85]. The basic idea is to compute a threshold based on a global clock and propagate the time-stamps of the roots (which are initialised with the global clock) to all reachable nodes. After this, all nodes with a time-stamp less than the threshold are garbage. Again the details are not considered because time-stamps need the notion of time. This notion of time relies on an implementation of a global clock which is tricky in an asynchronous environment. 8

As defined by Lamport [Lam78].


59

Lang et al. [LQP92] recently introduced a new idea based on doing tracing within groups. The basic idea is nesting spaces 9 into independent groups. A space belongs to a group and cooperates for the graph construction of its group only. The algorithm consists basically of four phases: Group Negotiation: Each space chooses a group to take part or creates a new one. This phase can be based on a predefined configuration of spaces to groups. Initialisation: This phase distinguishes internal references from external references. Every node referred to outside of the group (external reference) is marked black. Propagation: A marking phase is executed within each group propagating black marks down into the local structure and marking the nodes from the roots. Reclaiming: Every white node can be reclaimed. As a matter of fact this algorithm is not pure marking in the strict sense. The proposal relies on a variant of reference counting to do the initialisation phase. Also, the proposal is not a cyclic garbage-collection algorithm in the strict sense since the collection of garbage cycles depends on the configuration of the groups: a cyclic garbage structure is only collected if fully included in a group. To get round this problem the authors assume that a space is not necessarily a member of only one group. Actually, a hierarchy of groups can be built in order to guarantee that every cycle will be contained in at least one group. The implementation of this hierarchy of groups will require a node to have a marking node for each group it belongs to, and unfortunately this could be expensive in terms of space. Although Lang et al. argue that their proposal is efficient, they have not implemented it yet. Hence, some accurate results of performance are still missing. For the L INDA context performance is not the main concern since such performance results could not be fully valid in L INDA. As it is described Chapter 5 the idea of nesting processes into groups appears interesting for L INDA, but unfortunately its application suffers from problems of race conditions which have yet to be solved. 9

A space is an abstract name representing a processor, a process, a location, etc.

2.3. Distributed I/O

2.3

60

Distributed I/O

This section describes two works incorporating I/O capability for message-passing environments, respectively MPI (Message-Passing Interface) [CFF 95] and PVM (Parallel Virtual Machine) [MS94]. The reason for describing these message-passing proposals is to make clear that the problem dealt with in this thesis for L INDA has been identified and solved in other platforms. Once these proposals for message-passing are described an outline of a more direct L INDA work is discussed [Gel89] . The basic idea of Corbett et al. [CFF 95] when describing the extension for MPI is that all I/O can be described in terms of message-passing. Operations such as writing on a screen, printing a job or writing to a file are analogous to sending a message and waiting for some acknowledgement when necessary. Despite being related to message-passing, Corbett et al. justify the inclusion of an I/O system for MPI using arguments that could equally be applied to L INDA-like systems: the I/O proposal of Corbett et al. is intended to be an extension to the standard MPI system since MPI lacks the ability to deal with file I/O, and there are no standard distributed I/O systems on which MPI can rely. The MPI-IO is mainly concerned with performance and therefore functionality is left as a second priority. This thesis takes the opposite view; it is concerned with functionality and also with unifying concepts. The PIOUS (Parallel Input/OUtput System) [MS94] is based on the inclusion of an I/O capability within PVM. As with MPI-IO, Moyer and Sunderam have identified the lack of a standard I/O system for distributed environments like PVM. PIOUS is actually a parallel file system, taking the view that I/O occurs via operations with files (like the Unix abstraction). PIOUS generalises PVM by proposing several new primitives to deal with files. This increases the complexity of the system significantly, since users have to understand how all these new primitives work and interact. The main advantage of the L INDA case over the PIOUS case is that not many modifications are necessary. In the solution proposed in Chapter 5, only one new primitive to link I/O devices to the L INDA tuple spaces is necessary, and the other L INDA primitives are used to access files.

2.3. Distributed I/O

61

Figure 2.15 Example showing the Gelernter’s proposal for represent files as tuple spaces.

[18,] STEVE, 25, MALE MARY, 17, FEMALE

[15,]

JOHN, 46, MALE JOANNA, 37, FEMALE

[16,]

CRIS, 20, FEMALE .....

[0,]

..... .....

[17,]

..... PAUL, 0, MALE

[19,]

Gelernter [Gel89] briefly described an abstraction of file systems in terms of persistent tuple spaces. He suggested that files can be represented as tuple spaces having tuples of the form

representing file elements, where

repre-

sents the sequential position of the fields in the file as shown in Figure 2.15. Gelernter also described a generalisation of his suggestion proposing the elimination of the file system in favour of one composed of persistent tuple spaces. Every file then could be visualised as a Unix-like byte-stream by storing tuples of the form

or as binary files by storing tuples of the form

.

The approach of this thesis differs from Galernter’s since the conventional idea of files is maintained and an abstraction layer over the existing file system is created. This thesis does not propose the replacement of a concept that is well accepted, since the world does not (and will not) consist only of L INDA processes — if one assumed the replacement of the file system by a L INDA based one, one would have to modify all other non-L INDA processes to incorporate the concept of tuple space, or create a layer of abstraction over the L INDA-based file system in order to regenerate the conventional view! This proposal also differs from Gelernter’s as it provides persistent tuple spaces by having files underlying such tuple spaces and not the op-

2.4. Final Remarks

62

posite. However, this thesis is not proposing a novel way of creating persistent tuple spaces, rather persistent tuple spaces are achieved “for free” as a consequence of the proposal for distributed I/O in L INDA. The question of how the inclusion of I/O in L INDA differs from the proposals for MPI and PVM, respectively MPI-IO and PIOUS, can very easily be answered and is related to how L INDA itself differs from both MPI and PVM. L INDA provides a more powerful programming environment and yet simpler. L INDA abstracts from many low level issues addressed in systems like MPI and PVM. Still, there are implementation of L INDA where the underlying model is based in MPI [CSS94] and PVM [DWR95, Gal97]. In these implementations, shared-memory commands are translated into messagepassing. The idea when introducing I/O in L INDA is to provide the model with high level operations to deal with I/O as well. These primitives could even be translated into their equivalent in MPI-IO and PIOUS. For instance, MPI-IO provides several ways of accessing a file: blocking, nonblocking, using pointers and explicit offset. These operations require a series of other operations to be executed telling for instance the way the file is partitioned. The primitive introduced in this thesis allows files the same file access functionalities as MPI-IO while removing from the users the burden of some low level operations. File partitioning, access rights, among others, is provided automatically by L INDA — as a file is abstracted as tuple space, given that tuple spaces are partitioned file will also have this feature (see Chapter 5 for more details). In Chapter 5 it is shown that the problems related to I/O in L INDA are far more intricate than Gelernter’s proposal would suggest and that most of them need deeper analysis in order to have a L INDA-based I/O coordination system working properly.

2.4

Final Remarks

This chapter presented an overview of the area explored in this dissertation. A comprehensive description of L INDA-like models and implementations allows one to form a background in the area and realise the need for a garbage-collection system in open implementations of L INDA. Whenever possible, in each model/implementation

2.4. Final Remarks

63

description, considerations with regards to garbage collection and I/O were presented. With regards to I/O, this chapter showed that other models, like PVM and MPI, considered adding I/O to the basic model.

Chapter 3

The L IGIA Model

In Chapter 2 several implementations of different variations of the L INDA model were described; unfortunately most of them fail to address resource management issues. L IGIA is a variation of the L INDA model that tries to consider issues of memory management and distributed I/O in its concept; at the same time, L IGIA is proposed as a fair where only well understood concepts are implemented. The goal in this chapter is to give a full account of the choices made for L IGIA. Based on the existing gaps left by other models, L IGIA tries to have a good trade-off between needed and desired features. This chapter is divided as it follows; in Section 3.1 the general issues of the model are described, the aim of this section is to present an overview of the model; all the more important issues are explained in separate sections. In Section 3.2 the Universal Tuple Space ( ) case is made; the need for a global resource in open implementations is emphasised as justification for the definition of a in the L IGIA model. Section 3.3 explains why handles in L IGIA are first-class objects but not tuple spaces nor processes; also the problem of handles as worldwide unique identifiers is explored. In Section 3.4 the I/O coordination problem and is explained as the motivation for having the tuple space defined in this model. Section 3.5 formally describes the -ordering problem as defined by Douglas et al. [DWR95] and motivates its inclusion in L IGIA. Similar to the problem of universal identifiability of tuple spaces, Section 3.6 explores the need for processes to be identifiable within 64

3.1. General Description

65

the model along with the need to have this identifiability in a universal (worldwide) sense. Section 3.7 describes the concept of tuple monitoring introduced in L IGIA; its assumed that the model can monitor the content of the tuples being stored within the tuple spaces. Section 3.8 introduces the concept of termination-ordering which is essential for the correct implementation of garbage collection.

3.1

General Description

L IGIA is based on the multiple tuple-space model [Gel89] and on implementations previously done at the University of York [DWR95, RW96]. As in any L INDA model, communication is done solely via tuple spaces which are created by processes, except for the Universal Tuple Space ( ) which is a persistent distributed tuple space provided by the model itself. Additionally to the model assumes the existence of an tuple space which represents the abstraction of I/O within the model. Both tuple spaces are known to every process executing within L IGIA; this corresponds to saying that every process in L IGIA has the ability to do I/O and has access to the global resource provided. The concepts implemented here make L IGIA different from other models. The model includes a new primitive and assumes the existence of an tuple space. In organizational terms, the structure of tuple space is flat and similar to the York L INDA 2; however the kernel1 sees this flat structure with links between processes and tuple spaces. The use of a flat structure means that no tuple space is stored within others, which eliminates the need for first-class tuple spaces. Handles on the other hand can exist within tuple spaces; a handle is a universal unique name; a process in possession of a handle can get direct access to this structure independently of the tuple space’s physical location. Processes are also the subject of some controversy. As first defined in the original L INDA model, processes had no identity, this means to say that not only were the messages anonymous to the process’s themselves but also to the kernel. In L IGIA, 1

To avoid confusion with the kernel of an implementation, in this chapter the work “kernel” refers to the equivalent of a kernel of an implementation but on the model level.

3.2. The Universal Tuple Space

66

processes, like tuple spaces, can be uniquely identified by the kernel but not by other processes — the kernel can, if necessary, monitor a process activity. Following the flat structure concept, processes also do not exist inside tuple spaces and do not turn into passive tuples when they finish their execution. A process is spawned linked to a tuple space not only for historic reasons (L INDA users expect this to be the case) but also because it is necessary for the kernel to understand the scope of that process within the system. The tuple space defined in the primitive is known by the process and thus can start to operate with this tuple space straightaway. In terms of primitives, L IGIA implements the standard primitives (

,

and ),

and ) and ), a primitive to create tuple spaces ( ), one to spawn processes ( finally a new primitive which deals with I/O ( ).

their non-preemptive version (

,

), two bulk-primitives (

Figure 3.1 shows the L IGIA flat structure of tuple spaces and processes. Processes execute outside tuple spaces but maintain a link to the tuple space where they were spawned. It is assumed that processes do not turn into passive tuples inside a tuple space when they terminate (although they may with no difference to the model) and the links between processes and tuple space are pure organizational — a kind of hi erarchy is created. It is assumed that the main processes started via non- means (outside the L INDA model) are linked to . Although some links are assumed in the model, the implementation will later expand this structure so as to attend to the need of a garbage collector. The structure in Figure 3.1 is used as the start point to build the structure used by the garbage collector.

3.2

The Universal Tuple Space

L IGIA is a model aimed at open distributed applications and presupposes the existence of a universal persistent resource called Universal Tuple Space ( ). As the tuple-space model assume no direct communication between processes and as all communication is done via a tuple space, the existence of a universal resource is the only way that processes separated from each other on different contexts can set up


67

Figure 3.1 L IGIA structure of tuple space and processes.

UTS

I/O

ts1

ts4

ts3

ts2 ts5

communication. L IGIA being a model from the L INDA family inherits properties of time- and space-decoupling and the generalization of these characteristics demand the existence of . In order to the make the need of in open systems clearer consider an example as in Program 3.1. It shows a sketch of a process that wants to offer the service

to the world; in L IGIA (and L INDA) the way to do this is to put a token in a tuple space with the offer.

is offered in the tuple space ; the process stores The service a tuple with the service name and a handle where the requests should be made; the process then blocks (

) on the tuple space waiting for requests. Should another


68

Program 3.1 Program where the process offers the service cess request for this service in the tuple space .

and pro-

"" ) ! + ! ! # ! )%!" & # %!" + * ! " + # * ) % *

process request the use of the service, the process in Program 3.1 unblocks, computes the result and stores this result back in the tuple space making the result available to the requesting process. Although very simplified, the idea of the situation above is reasonably common. The problem with the example is that the process offering the service is restricted to the scope of ; unless a tuple space known by every process is provided, there will always be some process without access to

. Figure 3.2 depicts an open environment showing the situation described above where the system is divided into two halves unable to communicate with each other for there is no resource to allow this communication. Figure 3.2 shows that the processes , and

cannot get access to tuple space

and consequently cannot see the service being offered within this tuple space — is separated from the other half by a virtual wall (a barrier) dividing the system

into two halves. It is necessary to have a tuple space to overcome this barrier; for instance, and

can communicate because

is a global resource in their context.

Obviously for the specific case in Figure 3.2 a solution can be formulated without the introduction of but when the problem is generalized it can only be solved with the introduction of a global resource such as . In Chapter 4 it is explained that the concept of a garbage object is based on its visibility from the outside world. should be used in the garbage-collection scheme because in L IGIA, is the closest to the outside world a process can get. As such,


69

Figure 3.2 Without a universal tuple space some processes are unable to communicate.

ts1

["OFFER","SpecialService",MyTS]

P1 MyTS

P2

ts2

P4

P3

3.3. First-Class Tuple-Space Handles

70

acts as a start point for a garbage-collection scheme; any useful object should be

visible from the outside world or from . The is a special case as it is not created by any process. As tuple spaces are created by processes and processes are spawned linked to a tuple space, is provided by the model as an exception to the rule so as to avoid the chicken-and-the-egg problem — if a process has created to which tuple space is this process linked?

3.3

First-Class Tuple-Space Handles

An object is said to be a first-class object in a language if it can be assigned to variables, compared, used as an object of other structures in the language, in short manipulated within the language. In L INDA the objects are tuples, tuple spaces and processes. The L INDA community has endlessly discussed the topic of first-class objects; still several implementations assume tuple spaces and/or process as first-class objects. Although first-class tuple spaces and processes is a very interesting idea and despite posing as a generalization to the model, in practice, its implementation leads to several questions that remain unanswered. The idea of tuple spaces as first-class objects was one first proposed by Gelernter [Gel89] in his description of the multiple tuple-space model. The thought is that tuple spaces as first-class objects could exist as tuple elements and be stored within other tuple spaces. The idea in general is very interesting but there are semantics that need to be defined before one decides to implement it. By now the idea of the L INDA primitives should be well understood. The model provides primitives to store, read and remove a tuple from a tuple space. The semantics of these primitives are well defined. By removing a tuple a process can side-effect another process with the same template but the effect is predictable in the model — this other process will be blocked until a suitable tuple is placed in the tuple space. If tuple spaces are now assumed to exist as tuple elements, a process could potentially remove this tuple or read the tuple. What are the semantics in this case? So far, no implementation has given a full account of the implications that this assumption may have. For instance, if a process removes (via

) a tuple containing a tuple space,

3.3. First-Class Tuple-Space Handles

71

what should happen to the processes accessing this tuple space? Can they still access the tuple space even though the tuple space no longer exists within the L INDA environment?, or should an exception occur in these process? What if the tuple is read instead of removed? Is the tuple space duplicated? If yes, to which version of this tuple space do the processes have access? Perhaps these questions are not difficult to answer but surely these situations have to be examined at the model level before an implementation of the model is done. Unfortunately so far no implementation has presented a reasonable semantics for this problem. The idea of processes as first-class objects also exists. This leads to even more intricate questions since processes are active objects. What does a

of a tuple con-

taining a process do? Is the process duplicated? what are the side-effects of this duplication? If a tuple is removed via

the same sort of problem occurs; If a tuple

containing a process is removed from the tuple space does the process stop? Is the process just blocked? Can this process be resumed? Is son, then how should the environment this process requires be kept for future reference? In general, tuple spaces and processes as first-class objects brings many questions to the model. To answer them all means complicating the model too much. L IGIA chooses not to implement tuple spaces and processes as first-class objects in order to have a simpler model and also because the implementation of these features is not essential. L IGIA instead implements tuple-space handles (references to tuple spaces) as first-class objects. These names can be manipulated without any problem and without any complication since the structure itself (the tuple space) is not being considered. Duplication of names can be done with no side-effect to the model. In L IGIA these tuple-space handles are generated automatically by the kernel. This is the case because the uniqueness of these names is essential to avoid conflicts in large-scale applications. In fact the universal naming problem has become a common problem in distributed systems due to the growing number of application (e.g. web applications) where name conflicts should be avoided. CORBA [Gro95] faces the same problem; CORBA’s object model relies to a great extent on the semantics of object references which uniquely identifies a local or remote object instance [Hen98]. In L IGIA tuple-space handles are guaranteed to be unique within the model.

3.4. Tuple Space

3.4

72

Tuple Space

One of the main ideas that L INDA has introduced is the abstraction of physical location. Processes in L INDA are not aware of the location where the objects (tuples, tuple space and other processes) are. This location-free characteristic is broken when I/O is considered. Ideally a process in L INDA should not need to pay attention to location but when I/O is considered a process has to deal with it in a location-based fashion — I/O is a concept intrinsically attached to location. Yet, no model so far has considered the problem that distributed I/O can pose to this abstraction of location. The problem above alone could be used as a justification to have L INDA dealing with I/O within the model. However the distributed I/O problem has other facets. In open distributed systems heterogeneity might make distributed I/O hard to perform. A common framework would ease things, a common way to deal with I/O regardless of differences in the underlying system; this to some extent is an extension of the space-decoupling idea introduced in L INDA. Therefore L INDA can pose as this common framework so the model should provide ways to allow L INDA abstract I/O. The L IGIA model assumes the existence of an tuple space which is the abstraction of Input/Output within the model and is a way of hiding I/O physical location from users. In a nutshell L IGIA makes I/O like a capability [Tan95] that can be passed via tuple spaces; this gives the kernel knowledge of which process is able to perform I/O operations. At the model level, the tuple space represents the capability of doing I/O; from the kernel point of view if a process has an handle this process can do I/O. Later in Chapter 5 it will be shown that at the implementation level a single I/O tuple space does not exist, the L IGIA implementation uses a new primitive to bring I/O into the system. Still the concept of a single tuple space is useful in the garbage-collection structure. At this point it should be clear that having L IGIA dealing with I/O makes the model more smooth. Users have to deal with fewer concepts as even I/O now is abstracted in terms of tuple-space operations. Nevertheless filling a gap in the model, the explanation below should make the case of I/O in L IGIA irrefutable. Start with the premise that L INDA as a general coordination language should be able to deal with any coordination problem (problems involving more than one pro-

3.4. Tuple Space

73

cess) including I/O. Instead, existing L INDA-like models leave this problem to be dealt by the computational host language in which their implementations are embedded. According to Gelernter and Carriero [GC92] a language can be divided into two orthogonal parts: a coordination and a computation part. L INDA acts as a coordination add-in for existing computational languages. Using this orthogonality one can also say that a given problem is said to be a coordination problem if it is not a computational one. It is argued that based on the above fact distributed I/O is a coordination problem, as opposed to Kielmann [Kie96] who classes I/O as computational. In order to demonstrate that this is the case, take the orthogonality argument of Gelernter and Carriero and ask: Is I/O a computation or a coordination problem? Since the areas are orthogonal a problem has to be classified as either of them. Consequently L IN DA,

being a coordination model, should not be affected by computation problems.

Therefore one can re-phrase the question: Does I/O interfere with the L INDA model? The answer is that it does — L INDA is a location-free model for distributed systems, and I/O, when done by computation languages, is not; therefore I/O is a coordination problem. It is also possible to state that I/O is a coordination problem by using the view of Corbett et al. [CFF 95] who see I/O as a message-passing operation. If I/O is a message-passing operation it involves more than one process and consequently needs coordination. Therefore, as a coordination problem, I/O must be suitable to be dealt with by L IGIA if L IGIA intends to be seen as a general coordination model for open systems. As said before, the tuple space is an abstract view of I/O within the model. When implementing this model a single tuple space will not necessarily exist to represent I/O. In L IGIA the ability of doing I/O is represented as a process having knowledge of the tuple space. For the sake of the implementation a new primitive

is proposed. The specific details of this primitive are described in Chapter 5. In the model point of view this primitive links a process to the tuple space. It is important to notice here that I/O is not being brought inside the model just for the sake of garbage collection. In fact, by trying to solve garbage collection the

3.5. -Ordering

74

problem of I/O appeared, but L IGIA incorporates this idea also because distributed I/O is a coordination problem.

3.5

-Ordering

Introduced by Douglas et al. [DWR95] (see Section 2.1.4), -ordering is a property that is expected by the users but has been forgotten by the majority of L INDA-like implementations. The L IGIA model assumes the existence of -ordering as defined by Douglas et al. as it represents an expected property in open systems. The basic semantics of -ordering has been explained before; L IGIA assumes what is formally described in Property 3.1. Property 3.1 -ordering

Let space as they be an ordered sequence of s to a tuple be an ordered are executed in the process (causal order) and let

sequence of those s as they arrive within . The -ordering is guaranteed for if and only if:

.

It is important to notice that -ordering is not a property that can be achieved using ad-hoc programming techniques. If the model does not guarantee the ordering of the messages it is impossible for a process to find out if the message just read is the last of a series even though the message says this. How in a non-deterministic model can a process find out if another message is still to arrive? The model (the kernel of the implementation) has to guarantee the ordering of execution as the primitives of the model cannot achieve the same result. Bonita [RW97] assumes -ordering from multiple tuple-space perspective. On the other hand all other models which consider -ordering in their descriptions consider it from a single process perspective, thus not introducing a global state which would not be viable in the open distributed case. Consequently, L IGIA assumes -ordering from a single tuple space perspective.

In the example shown in Program 3.2 -ordering will behave differently in L IGIA and Bonita. Assuming that the process executing Program 3.2 is the only one accessing

and

Bonita guarantees that the result of the primitive

is

$ as

3.6. Process identifiability

75

Program 3.2 Example where -ordering behaves differently in L IGIA and Bonita.

#& " #& " + ! + % + + % + ! + % ! + % " + * ! + *

%

the ordering is relative to multiple tuple spaces. In L IGIA the result of the primitive might be as the ordering is not guaranteed among multiple tuple spaces. The only guarantee is that when the tuple gets to the tuple has been stored already. No guarantee is given with relation to the tuple being or not being in . The -ordering problem relates to questions of what is expected from the model and what is up to the programmer to guarantee. L IGIA does not implement a crosstuple-space ordering; although ordering is expected when using the same object, this report takes the view that a user does not expect an ordering when dealing with different tuple spaces. In general tuple spaces represent different contexts and are stored at different locations so that there is no reason why the overheads of one should affect another. If a cross-tuple-space ordering is required it is up to the programmer to guarantee this by using explicit methods.

3.6

Process identifiability

It should be clear by now that L IGIA provides means for spawning processes; in L IGIA the primitive spawns a process (not an active tuple) and this process does not turn into a tuple. Given this description one could ask why the primitive needs a tuple-space handle if the process is completely unlinked to this tuple space? There is an explanation; first for historic reasons L IGIA defines a handle so that the semantics of the primitive becomes closer to what users are used to in other L INDA-like models; second the handle defined in the comprises the scope of that process, it means


76

that the process will start executing having knowledge of at least the tuple space defined in the primitive; finally, since L IGIA is concerned with memory recycling in its implementations, it is important to have a handle defined so that a structure of tuple space usage can be built accordingly and later used by a garbage-collection scheme. This leads to the question of process identifiability. Following the view of the standard L INDA model processes are not uniquely identifiable and anonymous communication is one of the important characteristic of the L INDA model. But how can a structure of tuple space usage be built if a process cannot be identified? The answer is simple, L IGIA introduces the concept of process identifiability from the kernel’s and not from the process’s point of view. This concept has been introduced by Menezes and Wood [MW97] later being implemented by the authors [MW98b] and also in WCL[Row98]. Process identifiability introduces awareness of processes in the kernel level of the model. By knowing what processes are accessing what tuple spaces, the kernel can build structures to allow optimisations in the implementation. There is currently a lot of debate on optimisation in L INDA implementations: migration of processes, migration of tuple spaces, fair distribution of tuples across the network and resource management are just a few examples. Process identifiability enables the kernel to maintain all the necessary structures so that optimisations of this sort can be implemented. Take for instance tuple-space migration. It is desirable in any open L INDA-like implementation that a tuple space be maintained closer to the processes that are accessing it; moving a tuple space to a location where the sole process accessing it is executing would reduce the unnecessary overhead in the communication between this process and the tuple spaces. How can one move a tuple space closer to a process if the kernel cannot identify the process in question? Take also the case of local tuple spaces. It would be interesting to make any new tuple space created in L INDA local to the creator until its name is made public. How can the kernel attach a local tuple space to a process and move this tuple space along with the process where this process migrates? In short, process identifiability can solve some of the practical problems related to optimisations in open L INDA-like systems without breaking


77

the concept of anonymous communication — user processes still do not know about each other and cannot uniquely identify each other. The idea is that a process willing to coordinate via L IGIA has to tell the kernel in advance about its will. From the kernel point of view how this registration is done in the implementation is irrelevant as it can be done in various ways ranging from explicit registration to a registration hidden from users. The choice of how to implement this operation depends on several factors among which the computation language chosen is the most important. Once a process is registered in the kernel, L IGIA will be able to uniquely identify this process, the idea is that a process has a unique name within the model and therefore is uniquely identifiable. The name of a process is given by the kernel. It should be generated in such a way that no other process attached to the kernel has the same name. In a worldwide context the problem of unique universal naming appears. How this feature is implemented however, is irrelevant at the model level since it can clearly be done; for instance, URI2 could be used as part of a process name. The dual of process registration is process check-out where a process tells L IGIA that from that point in time onwards it does not intend to use the model facilities — does not require coordination operations. From the point of view of coordination, a process does not exist after its check-out. This operation happens to be very important for garbage collection as it is the only one that can generate garbage as explained in more detail in Chapter 4. Needless to say that from the model’s point of view how the check-out scheme is implemented is not important; it could be done via a termination detection scheme or explicitly via a check-out operation, for instance. Later, in Chapter 6 the choice for an explicit check-out mechanism is made as it allows users to leave the L IGIA “world” before the process termination. Moreover, termination detection schemes see processes as a whole consisting of coordination and computation whereas for L IGIA the computational part of a process is irrelevant; a check-out mechanism would allow the user to inform the kernel that the process has finished its coordination part whether or not it has finished its computation. 2 The URI (Universal Resource Identifier) is a naming scheme that provides unique global identification useful for large-scale, Internet applications.

3.7. Tuple Monitoring

3.7

78

Tuple Monitoring

Tuple monitoring is a concept that together with the concept of process registration helps the kernel to be prepared for optimisations that may be necessary to be included in the implementation. The general ideal is that the kernel should be able to monitor the tuples being stored or retrieved from tuple spaces; this would allow the kernel to be aware of any vital information that is being passed, as tuple spaces are the sole form of communication among processes. In Section 3.6 an example was used where a process might want to create local tuple spaces. It was said that this tuple space would remain local until its name is passed to other processes. How can the kernel know that a tuple-space name is being passed? Clearly, as tuple-space handles are first-class objects and can be stored within tuples, the only way that processes have to tell other processes the names of their local tuple spaces is by putting the names within some tuple and subsequently storing this tuple within a tuple space. Given that this is the case, the kernel must be able to monitor the contents of tuples so that it can find out if handles are being passed. Tuple monitoring has been introduced by Rowstron [Row96] when he shows that this can be used to improve the performance of open L INDA implementations by building a hierarchical structure of tuple spaces. The L IGIA model assumes that tuple monitoring occurs as this is essential for the maintenance of any structure related to tuple-space usage. In Chapter 6 the way tuple monitoring is implemented in L IGIA is explained. Considerations with regards to performance degradation due to the overhead of doing this monitoring are discussed showing how can one avoid a big overhead by using tuple flagging — client processes flag the tuples that are supposed to be analysed by the kernel.

3.8. Termination-Ordering

3.8

79

Termination-Ordering

Similar to the -ordering problem, the termination-ordering of L IGIA assures that the termination message3 is the last message of a given process to arrive in the kernel. In fact, this can be loosened to consider ordering only between the termination message and messages representing primitives that store information in the tuple s pace ( , , and ); although in L IGIA does not store information in a tuple space, it is considered in the ordering because passing of information can occur during the spawning (via a parameter). L IGIA is a model for open systems where garbage collection is essential; in the context of garbage collection, the termination of processes plays an important role, since it is after receiving a termination message that a garbage collection might decide on collecting an object. In other words, if one is considering garbage collection of tuple spaces, after receiving a process termination message, some tuple space may be considered garbage and then collected. If termination-ordering is not guaranteed a tuple with reference to a tuple space might arrive in the kernel and cause an exception should a process retrieve this tuple and try to access the tuple space — this tuple space may no longer exist. Property 3.2 formally describes the termination-ordering property. The L IGIA model assumes this property and therefore any implementation of L IGIA has to guarantee Property 3.2. The termination-ordering property aims to guarantee that garbage collection can be done free of race-conditions. This can be done in a similar way to Douglas et al. [DWR95] for the -ordering problem. Douglas introduces acknowledgement messages in the execution of so that an is only executed when the previous has been acknowledged. Property 3.2 Termination-ordering

Let be an ordered set of messages in their causal order sent be an ordered set of the same messages by a process and let but in the order they were received by the server process. The termination-ordering is guaranteed if and only if: .

The problem of race-conditions between operations updating the structure used 3

Because the term termination can mean different things it should be made clear that the termination occurs after the last instruction of a process — external intervention is not considered.


80

by the garbage collector is well known. Usually, the way to solve this problem is to implement variations of distributed garbage-collection algorithms that take into account this problem. The case with L IGIA however is not the same; as L IGIA already assumes -ordering the option of extending this characteristic to take into account termination seems more attractive than implementing complicated garbage-collection algorithms that take this problem into account. At a glance the termination-ordering may seem somehow too restrictive, but looking carefully one can see that it is not as restrictive as it seems. The terminationordering imposes ordering only between the termination message and the last message of a sequence of storing primitives. By analysing Property 3.2 one can see that no ordering is imposed on the subset of messages

.

and these can be processed in any suitable order; it turns out that some of the messages in

are ordered already due to -ordering. Additionally the termination

ordering does not pay attention to the primitives

, ,

,

and ; the se-

mantics of these primitives are that they have to wait for a result at that point and as such the next message would never be sent before the result is received. So the

termination-ordering is a property that imposes an ordering between a set and where contains only messages with the primitives , ,

and

. Still, if

and

.

are implemented as atomic oper-

ations (as they should be) they need not to be considered since when the result is returned the operation has been carried out. In order to clarify the case with the primitives

, ,

,

Figure 3.3. The figure depicts the possible cases if a primitive

and consider

was to arrive af-

ter the termination message on the receiver side. In the first case, in Figure 3.3(a), a primitive

is followed by a sequence of primitives and by the termination

message; the receiver order however is such that the primitive

arrives after the ter-

mination message. This case is clearly impossible as the sequence of primitives cannot be sent to the receiver until the primitive to the

is executed by the receiver due

characteristics. In the second case, in Figure3.3(b), the primitive

follows

a sequence of primitives and is followed by the termination message. Again this

arrives at the receiver after termination but this cannot happen as the termination


81

Figure 3.3 Impossible cases of ordering between the termination message and the primitive . Sender

Receiver out

Sender

Receiver

Sender

out

Receiver

out

out

out

out

out

out

ternination

termination

termination

in

in

in

(a) before a sequence of s

(b) after a sequence of s

message is only sent to the receiver after the

(c) between two sequences of s

is processed by the same receiver.

The third case, in Figure3.3(c), is similar to the first case as the order on the receiver side cannot happen as the second sequence of primitives is only sent after the processing of the

. In Figure 3.3 the primitive

and (and

and

can be replaced by either ,

,

if they are atomic) and the sequence of

primitives can be replaced by a sequence composed of any one of the prim and if they are not atomic). In any case itives and (and

the ordering is maintained automatically and does not have to be guaranteed by the termination-ordering property. In order to show why the termination-ordering has to be guaranteed for the other primitives one should consider each of these primitives ( , , and

) individually.

and

are considered for the case where

they are not atomic operations. The first and more common case is with . In Program 3.3 the is being used to store a handle of

within

. As

has been created just before the execu-

tion of the only the process executing Program 3.3 was aware of the existence of

at that point; however, after the execution of

, the handle will be available to

any process accessing . Observe that if the termination is executed before the


82

Program 3.3 Termination-ordering with .

# & " + !% *

a garbage-collection scheme could decide that

is garbage and remove it as the

garbage collector will not have the information related to the storage of the tuplespace handle. This anticipated execution can occur because is a non-blocking primitive, it only sends a message to the server with the tuple to be stored. When the is then executed and the handle becomes available within , processes would have access to a tuple space which no longer exists within the system. One can then conclude that termination-ordering is therefore necessary when considering the primitive . Program 3.4 Termination-ordering assuming the standard semantics of

.

# & " + +

&) %!" *

Program 3.4 shows the case for the primitive . This primitive is very similar to an but in the case of the field of the tuples are evaluated in parallel. Although this is not the case in L IGIA it is interesting to show that when behaves as it was

originally defined the problem can be considered and the side-effects avoided. Simi lar to the case, the termination message should not be processed before the primitive. It should be noted that this does not impose the restriction that a parent must wait for the completion of every child process. Once the active tuple is stored in the tuple space in the form

the termination message can be processed.

Therefore the parent process can terminate even in the case the process executing is still executing. The existence of the active tuple within should be sufficient for the garbage-collection scheme implemented. In the L IGIA model has to be considered because the process can receive a handle as a parameter.


83

In this case the program would look more like Program 3.5. The execution of the

(not the completion of the child process) has to be carried out so as to avoid a pre mature collection of before the kernel can realise that the process has knowledge of

(via monitoring).

Program 3.5 Termination-ordering with primitive

in the L IGIA case.

# & " + + !&& ! *

Program 3.6 Termination-ordering with primitive

.

#& " #& " + ! )% ! ! * % + * ! * )% *

When considering the primitive

,

the problem is bit subtler. Like

returns a value but its execution is different. Although

and

should

be an atomic operation, some implementations might choose to lose this atomicity to gain performance or due to another implementation choice. The L IGIA model should be prepared to deal with ordering whether or not the implementation guarantees atomicity. Program 3.6 shows an example with the primitive spaces are created, then the handle of

is stored in

; two tuple

and immediately read so

as to guarantee that the took place. A is then executed moving all tuples matching from to . If we see as divided in two parts, one that removes all the tuple matching from and the other that stores the tuples in , the termination-ordering property has to guarantee that the second part does not occur after the termination message. As the first part of

re-

moves tuples containing handles the reference to another tuple space might be lost if the termination message is processed at the wrong time. The second part of

3.9. Final Remarks

84

which stores the handles in another tuple space has to be executed first so as to guarantee that references to the tuple spaces are restored. Program 3.7 Termination-ordering with primitive

.

#& " #& " + ! )% ! ! * % + ! %!" *

Program 3.7 shows the case for the primitive to the case for the primitive

which is very similar

shown in Program 3.6. The basic difference here

is that tuples are copied and not moved from

to

. For this reason one could

think that no ordering is required as the original links are not lost. Observe that this is only true if the tuple space in question is not being accessed by other processes. The original links can be removed between the first and the second part of causing the a situation similar to the primitive

. Therefore, the termination

message has to be executed after the second part of a

3.9

.

Final Remarks

This chapter has presented the L IGIA model, highlighting its differences to the other models described in the previous chapter. Among the new concepts introduced by L IGIA the three most important in this dissertation are: process identifiability which introduces the concept that processes are uniquely identifiable in L IGIA; tuple monitoring which states that the model has to analyse the contents of tuples being stored and retrieved from tuple spaces; and termination-ordering which says that after the message informing the kernel of a process termination arrives in a server, no other message from the same process will arrive in any server.

Chapter 4

Incorporation of Garbage Collection

Why should garbage collection be done in L IGIA? More importantly, can it be done?. By now the answer to the first question should be clear as L IGIA is aimed at open systems. In open environments compile-time optimisation is not an option as there is no concept of compile-time for the entire system. Indeed garbage collection is necessary as objects are created in L IGIA and are not removed by the system. In general open environments, it is impossible for a process to decide by itself if a given object is still required in the system — a process in L IGIA does not have a view of the global situation. The question of whether or not it can be done depends on the way one looks at the problem. One could argue that the primitive

could be used as a way to do

garbage collection where a process removes its tuples when they become unnecessary. This could indeed be the solution in a very controlled environment but not the solution for the general case; once a tuple is stored in a tuple space a process cannot be guaranteed to retrieve that tuple again as other tuples with the same template can exist in the tuple space. Still, by carefully analysing the characteristics of L IGIA (and all other L INDA-like models) one can easily see that the concept of ownership of a tuple does not exist; if a tuple was uniquely identifiable one would use its identification 85

86 to retrieve the tuple instead of the matching mechanism. The answer for whether garbage collection can be done is therefore no in this case. It should be noted that the statement above is not the answer to the problem; the statement is that direct garbage collection of tuples is impossible in a standard L INDA-like model. Garbage collection however has to be implemented and if the model does not provide a suitable way the model has to be changed to provide means for doing memory recycling, essential in any open system. It turns out that the model does not need to be modified as the model provides a structure which can be used in the garbage collection. Since the proposal of multiple tuple spaces [Gel89], a process can create tuple spaces which, as opposed to tuples, are uniquely identifiable. Tuple spaces have handles and a process can only access a tuple space if its handle is known to this process. Garbage collection of tuple spaces therefore is feasible and tuples can be collected as a consequence since they are stored in tuple spaces; the difference is that the granularity of the collection is coarser. Similarly to the case of using

to remove unnecessary tuples from tuple spaces,

one could argue that a delete-tuple-space type of operation could be implemented to remove unnecessary tuple spaces. However, the case here is also similar to the tuple case and an operation of this nature cannot be securely implemented since a process cannot declare a tuple space as garbage — a global view of the situation of this tuple space is hard to obtain from a process’ point of view. Even though tuple spaces have unique handles, once a handle has been stored within other tuple spaces a process does not know whether or not other process has retrieved this handle; the kernel on the other hand might know this information if L IGIA is the underlying model. It should be clear then that one of the main problems in doing garbage collection of tuple spaces in L INDA is that the information about whether processes require a given tuple space is insufficient. L IGIA has introduced the concepts of processes identifiability, unique universal tuple-space naming and tuple monitoring that consequently allow L IGIA to maintain a structure in the kernel that can be used by a garbage collector. The construction of this structure is race-condition free owing to the existence of the termination-ordering property (see Property 3.2). The argument above indicates that one of the main problems to overcome is to

4.1. What Garbage-Collection Scheme?

87

correctly maintain all the information necessary for doing garbage collection. In order to do this, this chapter will use the concepts introduced at the kernel level and propose a way to maintain a graph with the information about the accessibility of tuple spaces by processes. This chapter is divided as follows: in Section 4.1 the garbagecollection scheme used is briefly discussed aiming to form a background so that the graph structure can be well understood. In Section 4.2 the general structure of the graph is explained introducing all the concepts necessary for building it; the structure is described as being centralised. In Section 4.4 the solution is extended for the distributed case, the structure of this distributed graph is explained together with the concepts involved in doing this. In Section 4.6 the main procedures involved in the graph maintenance are described in high-level terms regardless of whether the graph is centralised or distributed. In Section 4.5 the execution of the garbage-collection scheme briefly described in Section 4.1 is explored further; several examples showing the graph situation during the execution of the garbage collector are depicted.

4.1

What Garbage-Collection Scheme?

In Section 2.2 three standard garbage-collection schemes were described. In order to decide which scheme is appropriate for L IGIA one should first look at the concept of garbage in L IGIA. As said before, an object becomes garbage based on its visibility from the outside world. If the structure implemented can represent the outside world a tracing method can easily be used. It turns out that L IGIA already has this representation in its specification, namely

and tuple spaces. These two tuple spaces create a boundary between L IGIA

and the outside world (see figure 4.1). represent the outside world because it is the channel that processes can use to setup communication when they have no previous knowledge of each other. The tuple space also represents the outside world because it represents the capability that a process has to do I/O; in a nutshell,

and can be seen as the representation of the outside world within L IGIA and

therefore processes can only access the outside world via them. However tracing methods are more difficult and expensive to be implemented

4.1. What Garbage-Collection Scheme?

88

Figure 4.1 and create a boundary separating L IGIA from the outside world. Comm Channel

UTS

I/O Devices

I/O

File System

in the distributed case than reference counting or reference listing. Yet, tracing is a complete method and can identify all garbage nodes in a given graph. In order to use the good points of tracing (completeness) and reference counting (suitability to distributed graphs) the idea is to build a graph that can be used by a hybrid method where reference counting executes more often but where tracing can be performed whenever required. The tracing phase is necessary because cycles are likely to happen in the graph and as explained in Section 2.2 neither reference listing nor reference counting can collect garbage cycles of nodes. Some methods based on reference counting and reference listing where garbage cycles can be collected can be found [Lin92, JL93]. In the L IGIA case, the benefit of implementing a more complicated algorithm does not repay its costs since one could expect that the reference listing would collect most of the garbage making the tracing phase a plus that can be used whenever necessary. Additionally, the standard algorithms suffice even for the distributed case because the model guarantees that their implementation would not have race-conditions — L IGIA resolves the raceconditions at the model level. The garbage-collection scheme implemented in L IGIA therefore is a two-phase

4.2. Centralised Graph Structure

89

garbage collector composed of a reference-counting phase and a tracing phase where the tracing phase is based on reachability from and to and — everything that cannot be seen by and and cannot see and is garbage. The graph structure follows the common tracing graph concept used in most tracing methods but also containing information to be used by a reference-counting algorithm — a reference counter. Using specific L IGIA terms an object is considered garbage if: 1. its node in the graph has counter field equal to zero; this concept is valid only for tuple-space objects as processes are not referenced in L IGIA — processes do not know about each other; or 2. its node in the graph is unreachable from and . As said before since the graph contains representations of active objects the tracing has to resume from these active nodes; to see if they can reach or . This idea will become clearer later in this chapter. If an object is considered garbage due to case 1 it would also be considered garbage in case 2, but the opposite does not hold and therefore the cases are shown separately. It is expected that in most of the cases the objects would be identified as garbage owing to the first case — identified by the reference-counting phase. The structure has an invariant that once an object is considered garbage it cannot become useful again. This invariant guarantees that no erroneous collection occurs by collecting a node which would eventually resurrect.

4.2

Centralised Graph Structure

Generally, L INDA-like implementations provide almost no information that can be directly used to identify whether a given tuple space is still required in the system. Thus, this section describes a way to build a data structure with all information about the accessibility of tuple spaces by process. All processes in L IGIA, although not physically attached to any tuple spaces, have a tuple space which is defined in the primitive that defines an implicit link.


90

Additionally, in the multiple tuple-space model all tuple spaces but have to be created by a process via the primitive. 1 These two characteristics are central to constructing a graph representing processes and tuple spaces where, in principle, processes are linked only to tuple spaces (via indirected arcs) but with tuple spaces being able to be linked to each other via special directed arcs. The idea is that the tuple spaces linked to a process

represent those to which

has access; and pro-

cesses linked to a tuple space represent those with knowledge of . The graph should represent the view that an object becomes garbage if it cannot be seen from the outside world and cannot see the outside world; in fact the reason for having the second part of this clause is that the graph represents active objects (processes) as well as passive objects (tuple spaces); the active objects might change the graph in such a way that objects not visible from the outside world become visible in the future. The graph constructed in the kernel should represent the idea of a garbage object.

4.2.1 Graph General Description The L IGIA kernel maintains a graph containing direct and indirect arcs which acts as the main source of information for the garbage collector. Below are the assumptions used in the maintenance of this structure; later these points will be addressed in more detail. At this point, the description of the graph maintenance is done regardless of distribution issues; in Section 4.4, having understood the simpler case described here, the distribution issues are explored. The nodes representing and are the roots of the graph. These nodes are the starting points of the tracing phase which decides if a given node is garbage based on its reachability from these starting nodes. These two nodes have counter fields set to infinity — they are never garbage collected. There is a god process which is assumed to spawn every main process 2 linked to . The god process is an abstraction of the users. It is not represented in 1

is created by a ‘god’ process in a bootstrap operation. The concept of main process is L INDA is not well defined. For our purposes a process is considered a main process if it is a process spawned by means other than $# . 2


91

the graph but its existence is made clear in order to unify the concept that every process represented in the graph was linked to some tuple space. The links between processes and tuple spaces are always via indirect arcs while the links between tuple spaces are done using weighted directed arcs called bridges. All processes and tuple spaces are represented in the graph. The nodes representing tuple spaces have an additional field: the reference counter. A node representing a process is linked to its initial tuple space, 3 the one to which this process is linked. This link represents the fact that this process has knowledge of this tuple space and can operate with it. A tuple-space node must be linked to the process node representing its creator. Tuple spaces are created by the primitive and due to tuple monitoring and process registration L IGIA knows which process is executing the primitive. A node representing a process has to be linked to all the tuple spaces of its knowledge. Other than the knowledge of its initial tuple space and the tuple spaces it creates, a process can acquire information about other tuple spaces by receiving their handles in two ways: – On the spawning of a process, some parameters can be passed and among them a handle might be present. Upon the process registration with the kernel, links between the node representing the process and the nodes representing tuple spaces whose handles were received as parameters are created. – Via tuples stored in other tuple spaces. As soon as a process has read or withdrawn a tuple containing a handle, a link in the graph is created to represent this situation; this can be achieved thanks to tuple monitoring. Nodes representing two tuple spaces must be linked by a bridge if the handle of 3

Initial tuple space = Tuple space defined in the $# primitive.


92

one is stored in the other. The bridge would be outgoing from the containing tuple space and incoming to the contained tuple space. Some points on the construction of this graph might be still unclear as the concept involved might not be well understood. In the next sections each one of these point are explored in more detail and whenever necessary a pictorial view presented.

4.2.2 The

Node

In Section 3.2 the case for in L IGIA has been made; it should be clear that open systems require the existence of a global, persistent resource. As this report moves towards the implementation of this model some of the concepts introduced at the model level will be restricted so as to achieve a more stable implementation. has been defined as a resource known to all processes; this concept can be modified without problems to a concept where is known to all main processes. In order for spawned processes (processes descendent from the main processes) to know about

, its handle has to be passed to them explicitly. Note that this concept does not

interfere with the argument that a global resource has to exist. In fact, the global resource still exists, only the access to it is being more controlled. The choice of making the access to automatic only to main processes happened because operations with are side-effecting; if a process can ever get access to , this process, and consequently all nodes reachable from it, cannot be garbage collected. By controlling the access to the occurrence of side-effecting operations is minimised and the number of tuple spaces that can be garbage collected is maximised. With regard to the creation of the graph, a node representing is added to the graph when the L IGIA system starts. This node is never garbage collected as it is created with counter field set to infinity and also because is the starting point of the tracing phase. only ceases to exist when L IGIA terminates its execution. In the context of this report this case will not be considered as it is only worth talking about the problem of a system if the system is running.


93

Figure 4.2 Graph situation after the creation of and nodes by the L IGIA system.

UTS

I/O

4.2.3 The Node In Section 3.4 the concept of I/O as a coordination problem was introduced. For the context of garbage collection the ability to do I/O is given by a handle. A process can do I/O if it knows about the tuple space; in the implementation it will be clear that the tuple space does not exist in reality and its existence is only in the context of garbage collection. In practice the implementation introduces a primitive called

that abstracts a file as a tuple space (explained in Chapter 5).

The tuple-space node in the graph should be seen as a representation of whether or not a process can perform I/O — like a capability that can be passed on between processes. In practical terms, if a process is linked to the node it can execute the primitive

. The link is created in the structure when the L IGIA system

starts to execute. Upon the initialisation of the L IGIA system the graph should be as depicted in Figure 4.2. Throughout this report anything in the graph structure shown in dashed lines is being added to the graph whereas dotted lines identify the removal of an object from the graph. Like in the case with , the main processes are assumed to be able to do I/O and therefore are linked to the node when they start. This again maximises the power of the garbage collector as even processes can be garbage collected; if a process is executing but cannot communicate with the outside world this process is garbage since whatever it is doing no other process can get to know. This can only be achieved because L IGIA provides a boundary between the system and the outside world via and . In fact this might need further explanation; the idea is that a process will be garbage only if what the process is doing is untraceable from the outside world. This means to say that a process might not have a direct link to the outside world (represented by

and ) and still be considered alive (not garbage) as long as it can communicate


94

with another process which has links to the outside world. So a process is garbage if it cannot communicate with the outside world directly or indirectly.

4.2.4 Spawning Main Processes L IGIA assumes the existence of a primitive (normally called

) to allow processes

to spawn others in parallel. The semantics of this primitive in L IGIA is such that a tuple space must be specified to be the process’s initial tuple space. If a L IGIA process is implicitly linked to a tuple space, to what tuple space the are main processes supposed to be linked? The main processes are started in L IGIA by an external entity and can from there onwards spawn other processes via ; even though these main processes are also L IGIA processes they are not linked to any tuple space for they do not require to start executing. In order to change this and say that all L IGIA processes are linked to some tuple space one should define the tuple space to be used. appears as a natural candidate as it is created by the kernel and also because is known to every main process already. In abstract terms, suppose that there is one god process which creates and which represents the union of the users in this sense: if a user wants to start a process he tells the god process which then spawns a process (main process) linked to . The god process would only do the user’s job and it exists to make the concept of process creation unified in the model — now all L IGIA processes are linked to some tuple space. The god process is not represented in the graph structure; as there is only one

and there is only one god process (a one-to-one relation), concerning the tracing

phase of the garbage collector everything reachable from would also be reachable from the god process and vice-versa; therefore the existence of a representation would be redundant. Additionally the god process does not exist in reality and it is just an abstraction of users; moreover, the graph structure represents only L IGIA objects. One can now take the view that all L IGIA processes, without exception, are created explicitly using the primitive. Upon the execution of the primitive a node in the graph is created linking the process to the tuple space defined in the primitive. In


95

order to assign a unique name to the node representing a process in the graph, the process registration (explained next section) has to immediately follow its spawning.

4.2.5 Process Registration As described before, processes in L IGIA have to register if they require the use of L IGIA’s coordination facilities. This has to be done so that L IGIA is able to uniquely identify any process. In L IGIA, processes are the mutators of the garbage-collection algorithm. That is, the primitives executed by them might require some modification in the graph structure. Since they are the mutators, the kernel must be able to identify them whenever they execute a primitive that could alter the graph. If processes are unidentifiable, we cannot keep track of tuple-space usability. The process registration occurs when the processes are spawned; this registration consists of two messages: one sent by the process to the kernel, assigns a unique identifier to the process; the other, sent back by the kernel, informs the process of its identifier. From there on, the process has to use this identifier when executing the L IGIA primitives. The use of this unique identifier is transparent to the processes themselves. The unique identifier is assembled in the messages sent to the kernel. Even though this can be seen as done by the processes the assembling can be hidden so that from the programmer’s point of view this does not exist. Upon the registration of a process the links between this process and the tuple spaces whose handles were passed to it as parameters are updated; also the links between this process and both and are created; from the L IGIA point of view this is when this process starts to exist. Figure 4.3 shows the graph situation when two main processes start to run in a L INDA environment and register themselves with the kernel. The figure depicts two processes and the necessary links being added to the graph. In this particular case both processes are main processes and by default they know about and . The names of the processes appearing in the figures are simplified for better clarity. In fact since L IGIA implements unique process naming the real identifier has to be such that this uniqueness can be achieved; this is explained later in Chapter 6.

4.2. Centralised Graph Structure Figure 4.3 Two main processes, before they start running.

and

96 , register themselves with the kernel just

UTS

I/O

P1

P2

Figure 4.4 Creation of tuple spaces. A tuple space has to be linked to its creator.

ts1

UTS

I/O

P1

P2

ts2

ts3

4.2.6 Creation of Tuple Spaces The L INDA model provides a primitive to create tuple spaces. The primitive creates a local tuple space and returns a unique handle. is a blocking operation as the unique handle has to be returned before the handle can be used in other primitives. The unique handle identifies the tuple space within the kernel and should be used as the label of the tuple-space node in the graph structure. The creation of tuple spaces is a kernel operation; the process requests the creation and the operation is performed at the kernel level. Upon the receiving of the handle by the process the graph will have been updated accordingly. The kernel can identify the creator of a tuple space, since the message requesting the creation will contain the process identifier. By using tuple monitoring the kernel can extract this identifier, create a node to represent the new tuple space in the graph using the unique handle and create an arc linking the process to the tuple space just created — a tuple space is linked to its creator process. Figure 4.4 shows the graph situation just after the creation of two tuple spaces by

4.2. Centralised Graph Structure process

and one tuple space by process

97 (starting from Figure 4.3).

4.2.7 Creating Bridges Between Tuple Spaces L IGIA assumes that handles are first-class objects and as such can be manipulated like other first-class objects. As they are first class, handles can be placed in tuples and consequently be stored within tuple spaces. The storing of handles in other tuple spaces is an important operation to the garbage-collection algorithm. Whenever a handle is stored a dependency is created between two tuple spaces; if a handle of is stored within dle is kept stored into

, would be garbage only if

this happens because any process with access to

, as long as the han-

becomes garbage too;

can retrieve the handle of

and gain access to the tuple space — processes accessing

have potential access

to . In order to make the graph structure represent this dependency the concept of weighted directed arcs (called here bridges) linking the two tuple spaces is used. The bridge will be outgoing from the containing tuple space and incoming to the tuple space which had its handle stored. The weight attached to the bridge indicates the number of tuple-space handles which are stored in a particular tuple space. Figure 4.5 depicts how the graph is modified in each of the steps shown in the Code 4.1. Code 4.1 Code of a L IGIA main process.

"!#$%

+ +2 43 +C +D

& '(*) -,./01 ,./01 5687 9:; 0?

A@@A >*B1 5687 9:;