Development of Rich Domain Models with Atomic ... - ALGOS Group

UNIVERSIDADE TÉCNICA DE LISBOA INSTITUTO SUPERIOR TÉCNICO

Development of Rich Domain Models with Atomic Actions João Manuel Pinheiro Cachopo (Mestre)

Dissertação para obtenção do Grau de Doutor em Engenharia Informática e de Computadores

Orientador: Co-Orientador:

Doutor António Manuel Ferreira Rito da Silva Doutor João Emílio Segurado Pavão Martins

Júri: Presidente: Vogais:

Reitor da Universidade Técnica de Lisboa Doutor José Luiz Fiadeiro Doutor Maurice Herlihy Doutor Amílcar dos Santos Costa Sernadas Doutor João Emílio Segurado Pavão Martins Doutor António Manuel Ferreira Rito da Silva Doutor António Paulo Teles de Menezes Correia Leitão Doutor Rodrigo Seromenho Miragaia Rodrigues

Julho de 2007

Resumo O modelo de domínio é o elemento central de uma aplicação com objectos moderna. É lá que reside o conhecimento que a aplicação tem sobre o domínio do problema. Logo, à medida que se desenvolvem aplicações para problemas com domínios maiores e mais complexos, os modelos de domínio tornam-se mais ricos. Infelizmente, a implementação de modelos de domínio ricos utilizando as linguagens de programação com objectos actuais é uma tarefa difícil. Para simplificar esta tarefa, eu proponho estender as linguagens de programação com novos construtores que permitam a utilização de acções atómicas, a especificação da estrutura do modelo de domínio, e a implementação de regras de consistência. Estes novos construtores são introduzidos na linguagem Java de uma forma amigável para o programador, de modo a que os programadores possam usá-los sem alterações significativas no processo de desenvolvimento. Para implementar acções atómicas, proponho um novo modelo de Memória Transaccional em Software, e descrevo uma implementação deste modelo em Java—a JVSTM. Depois, proponho uma nova linguagem—a Domain Modeling Language—que permite a especificação das entidades e das relações existentes num domínio. Finalmente, proponho a utilização de Predicados de Consistência que tiram partido da existência de acções atómicas para permitir a implementação de regras de consistência de forma ortogonal à implementação do restante comportamento.

Abstract The domain model is the central element of a modern object-oriented application. It embodies the application’s knowledge about the problem domain. Therefore, as applications encompass problems with larger and more complex domains, domain models become both larger and richer than ever. Yet, implementing a rich domain model with current object-oriented programming languages is a difficult task. To simplify this task, I propose to extend object-oriented programming languages with new constructs that allow the use of atomic actions, the specification of a domain model’s structure, and the implementation of domain consistency rules. I introduce these new constructs in the Java programming language in a programmer-friendly way, so that programmers may use them without major changes in the development process. To support atomic actions, I propose a new model of Software Transactional Memory and describe an implementation of this model as a pure Java library—the JVSTM. Then, I propose a new language—the Domain Modeling Language—to allow the specification of the entities and the relationships between entities of a domain. Finally, I propose the use of Consistency Predicates that build on the support for atomic actions to allow the implementation of consistency rules orthogonally to the implementation of the remaining behavior.

Palavras-chave Programação Concorrente com Objectos Memória Transaccional em Software Programas de Domínios Complexos Modelação de Domínio Predicados de Consistência Arquitecturas de Software

Keywords Concurrent Object-Oriented Programming Software Transactional Memory Domain-Intensive Applications Domain Modeling Consistency Predicates Software Architectures

Acknowledgments “No man is an island. . . ” is a famous quotation from John Donne, and it is so for a good reason. We all have others with whom we work, with whom we talk, with whom we despair, with whom we laugh, and with whom we dream. All of these people, one way or another, influence what we do and who we are. The work leading to this dissertation was a long journey that would not have been possible without the help of many. To thank them all, I have written the most inspired acknowledgments of all time, which, unfortunately, the margins of these pages are too small to contain. So, instead, I shall give a much paler alternative that does not do justice to all the help that I got. First and foremost, I would like to thank Professor António Rito da Silva, my adviser and my mentor, who went much beyond the call of duty in his advising work. The completion of this work owes much to his enthusiastic encouragement, continuous guidance, and permanent availability. Moreover, any coherence that you may find in this dissertation is a direct consequence of his unique ability to rise up above the trees and see the forest. I would like to thank, also, Professor João Pavão Martins, my co-adviser, both for having initiated me into the arts of scientific research, and for his support throughout my work. I am grateful to INESC-ID, and specially to its Software Engineering Group, for providing me not only the means to do my work, but also an healthy and enjoyable environment to work in. I am deeply indebted, also, to all the ESW’s members, and, in particular, to my colleague and friend António Leitão, for all the numerous discussions that helped me to shed light into the more obscure parts of this work. I owe to António, also, much of my knowledge on the art of programming and on programming languages. This work would not have been possible without the support of CIIST, and, specially, of the Fénix team. I am grateful to them for their enthusiasm in applying my work to the Fénix system. I would never had started this work in the first place if Fénix did not exist. I am specially grateful to Eng. Luís Cruz for all his support and insightful discussions during my work. It was a pleasure to work with him.

To my parents Marcos and Olinda, to my sister Anabela, to my parents-in-law Aníbal and Isabel, and to Susana and Paulo I thank all the encouragement and support. Finally, regardless of all the remaining help, none of this work would have happened without the enduring support, encouragement, and sacrifices of my lovingly wife Ana and my dearest son Rui. I heart-feelingly thank them for all their patience and ability to lead me through the most difficult phases of my work. I thank Ana, also, for her analytical reasoning that helped me countless times during my work. Discussing the most difficult problems with her allowed me, time and again, to make things clearer in my head.

Julho de 2007 João Manuel Pinheiro Cachopo

Contents

1 Introduction

1

1.1 Domain-Intensive Applications . . . . . . . . . . . . . . . . . . . . . . . . .

1

1.2 Evolution Rather than Revolution: an Engineering Approach

. . . . . . . .

3

. . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

1.2.2 Guiding Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4

1.2.1 General Approach

1.3 Thesis Statement

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

8

1.5 Outline of the Dissertation . . . . . . . . . . . . . . . . . . . . . . . . . . .

9

1.4 Notation

2 Motivation, Problem Statement, and Approach 2.1 Domain Models in the Software Development Process

11 . . . . . . . . . . . .

12

2.1.1 Software Development as the Transformation of Artifacts . . . . . . .

12

2.1.2 Domain Model: A Central Artifact in the Development Process . . . .

13

2.1.3 Domain Model at Various Levels: Analysis, Design, Implementation .

14

2.1.4 Standard Architecture for a Domain-Intensive Application . . . . . .

15

2.1.5 The Domain Layer Dependence on the Infrastructural Layer . . . . .

17

2.2 Example of an Application in the Banking Domain . . . . . . . . . . . . . .

18

2.2.1 Example’s Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . .

18

2.2.2 Application’s Functionality . . . . . . . . . . . . . . . . . . . . . . . .

19

2.2.3 Basic Domain Modeling Terminology . . . . . . . . . . . . . . . . . .

20

ii

CONTENTS

2.2.4 Initial Design of the Banking Domain Model . . . . . . . . . . . . . . 2.3 Problem Statement

22

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

23

2.4 Two Approaches to Solve the Problem . . . . . . . . . . . . . . . . . . . . .

26

2.4.1 The MDA approach . . . . . . . . . . . . . . . . . . . . . . . . . . . .

26

2.4.2 This Dissertation’s Approach: Reduce the Gap between Languages

.

27

2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

28

3 The Difficulties of Implementing a Domain Model

31

3.1 Implementation of a Concurrent Domain Model . . . . . . . . . . . . . . . .

31

3.1.1 Basic Thread-Safety . . . . . . . . . . . . . . . . . . . . . . . . . . .

32

3.1.2 Thread-Safety with More than One Object . . . . . . . . . . . . . . .

35

3.2 Failure Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

39

3.3 Implementation of the Banking Domain Model’s Structure . . . . . . . . . .

42

3.3.1 Implementation of Classes . . . . . . . . . . . . . . . . . . . . . . . .

43

3.3.2 Implementation of Associations . . . . . . . . . . . . . . . . . . . . .

46

3.4 Implementation of the Banking Domain Model’s Behavior . . . . . . . . . .

50

3.4.1 Implementation of the Basic Deposit and Withdraw Operations

. . .

51

. . . . . . . . . .

52

3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

54

3.4.2 Implementation of the Client’s Total Balance Limit

4 Versioned Software Transactional Memory

57

4.1 Introduction to Software Transactional Memory . . . . . . . . . . . . . . . .

58

4.1.1 Atomic Actions and the Property of Atomicity . . . . . . . . . . . . . .

58

4.1.2 Read Sets, Write Sets, Commits, and Aborts . . . . . . . . . . . . . .

59

4.1.3 Transaction Linearizability . . . . . . . . . . . . . . . . . . . . . . . .

60

4.2 The Rationale for the Versioned STM . . . . . . . . . . . . . . . . . . . . . .

61

4.3 The Versioned STM Model . . . . . . . . . . . . . . . . . . . . . . . . . . . .

62

4.3.1 Model Elements and Terminology . . . . . . . . . . . . . . . . . . . .

62

CONTENTS

iii

4.3.2 Operations on Versioned Boxes . . . . . . . . . . . . . . . . . . . . .

64

4.3.3 The Transactions’ Life-Cycle . . . . . . . . . . . . . . . . . . . . . . .

65

4.3.4 The Linearizability of Top-Level Transactions . . . . . . . . . . . . . .

67

4.3.5 Garbage Collection of Old Values . . . . . . . . . . . . . . . . . . . .

70

4.3.6 Examples: The Bank Revisited

71

. . . . . . . . . . . . . . . . . . . . .

4.4 Implementation of the Versioned STM Model

. . . . . . . . . . . . . . . . .

73

4.4.1 The JVSTM’s API . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

74

4.4.2 Interaction with the Java Memory Model . . . . . . . . . . . . . . . .

79

4.4.3 Implementation of Versioned Boxes . . . . . . . . . . . . . . . . . . .

80

4.4.4 Implementation of Transactions . . . . . . . . . . . . . . . . . . . . .

82

4.4.5 Atomic Commits . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

85

4.4.6 Speculative Read-Only Transactions . . . . . . . . . . . . . . . . . .

87

4.4.7 Implementation of Garbage Collection

. . . . . . . . . . . . . . . . .

88

4.5 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

95

4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

97

5 Domain Modeling Language

99

5.1 DML’s Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 5.2 Grammar Notation and Lexical Structure . . . . . . . . . . . . . . . . . . . 102 5.3 Domain Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 5.4 Value Types

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

5.5 Entity Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 5.5.1 Syntax of Entity Type Declarations . . . . . . . . . . . . . . . . . . . 106 5.5.2 Semantics of Entity Type Declarations . . . . . . . . . . . . . . . . . 108 5.6 Associations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 5.6.1 Syntax of Association Declarations . . . . . . . . . . . . . . . . . . . 112 5.6.2 Semantics of Association Declarations . . . . . . . . . . . . . . . . . 114

iv

CONTENTS

5.6.2.1 Roles with a multiplicity upper-bound of one . . . . . . . . . 116 5.6.2.2 Roles with a multiplicity upper-bound greater than one . . . 117 5.6.2.3 Bidirectional associations

. . . . . . . . . . . . . . . . . . . 119

5.6.2.4 Sets returned by the method getRoleSet . . . . . . . . . . 120 5.6.2.5 Association objects and their listeners . . . . . . . . . . . . . 120 5.7 Implementation of a Domain Specification . . . . . . . . . . . . . . . . . . . 122 5.7.1 Using the JVSTM to Make a Transactional Domain . . . . . . . . . . 123 5.7.2 Implementing Entity Types

. . . . . . . . . . . . . . . . . . . . . . . 124

5.7.3 Implementing Associations

. . . . . . . . . . . . . . . . . . . . . . . 125

5.7.3.1 Storing the Association’s Links . . . . . . . . . . . . . . . . . 126 5.7.3.2 Implementing the Role Methods . . . . . . . . . . . . . . . . 127 5.7.3.3 Implementing the Association-Aware Set . . . . . . . . . . . 128 5.7.3.4 Implementing the Association Class . . . . . . . . . . . . . . 130 5.7.3.5 Implementing Different Role Types . . . . . . . . . . . . . . . 130 5.7.3.6 Enforcing Multiplicity Constraints . . . . . . . . . . . . . . . 134 5.8 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 5.8.1 Associations as First-Class Language Constructs . . . . . . . . . . . 135 5.8.2 Patterns for Implementing Associations . . . . . . . . . . . . . . . . . 136 5.8.3 Generating the Code for Associations . . . . . . . . . . . . . . . . . . 137 5.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

6 Consistency Predicates

141

6.1 Domain Consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 6.1.1 Consistency of Single Objects . . . . . . . . . . . . . . . . . . . . . . 142 6.1.2 Consistency of Rich Domain Models 6.2 Examples of Constraints

. . . . . . . . . . . . . . . . . . 144

. . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

6.3 Consistency Predicates for Atomic Actions . . . . . . . . . . . . . . . . . . . 149

CONTENTS

v

6.4 Consistency Predicates in Java . . . . . . . . . . . . . . . . . . . . . . . . . 152 6.5 Enforcement of Multiplicities with Consistency Predicates . . . . . . . . . . 154 6.6 Implementation of Consistency Predicates in the JVSTM . . . . . . . . . . . 155 6.7 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 6.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

7 Validation

163

7.1 The Fénix Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 7.1.1 Fénix History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 7.1.2 The Original Fénix Software Architecture . . . . . . . . . . . . . . . . 165 7.1.3 The Use of this Dissertation’s Work in the Development of the Fénix System

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

7.1.3.1 Implementation of the Fénix Domain Model with the DML . . 166 7.1.3.2 Other Benefits of Using the DML . . . . . . . . . . . . . . . . 168 7.1.3.3 The JVSTM in the Fénix System . . . . . . . . . . . . . . . . 169 7.1.4 The Fénix Transactional Workload

. . . . . . . . . . . . . . . . . . . 171

7.1.4.1 Total Number of Transactions Over Time . . . . . . . . . . . 172 7.1.4.2 The Read/Write and Write/Conflicts Ratios . . . . . . . . . . 175 7.1.4.3 The Dimension of the Transactions . . . . . . . . . . . . . . 177 7.2 JVSTM Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 7.2.1 Benchmark Running Environment . . . . . . . . . . . . . . . . . . . 179 7.2.2 Results for the DSTM2 Benchmarks

. . . . . . . . . . . . . . . . . . 180

7.2.2.1 Results for the List Benchmark

. . . . . . . . . . . . . . . . 181

7.2.2.2 Results for the Red-Black Tree Benchmark . . . . . . . . . . 185 7.2.2.3 Results for the Skip List Benchmark 7.2.3 Results for the STMBench7 Benchmark

. . . . . . . . . . . . . 188

. . . . . . . . . . . . . . . . 192

7.2.3.1 The STMBench7 Data Structure . . . . . . . . . . . . . . . . 192

vi

CONTENTS

7.2.3.2 The STMBench7 Operations . . . . . . . . . . . . . . . . . . 193 7.2.3.3 The STMBench7 Execution Parameters and Results . . . . . 193 7.2.3.4 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . 194 7.2.3.5 Throughput Results . . . . . . . . . . . . . . . . . . . . . . . 194 7.2.3.6 Latency Results . . . . . . . . . . . . . . . . . . . . . . . . . 200 7.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

8 Conclusions

203

8.1 Main Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 8.2 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

Bibliography

207

List of Figures

1.1 The continuous space from data-intensive to domain-intensive applications.

2

1.2 Graphical notation used to illustrate the concurrent execution of methods.

8

2.1 The layered architecture from [Fowler, 2002]. . . . . . . . . . . . . . . . . .

16

2.2 The layered architecture from [Evans, 2003]. . . . . . . . . . . . . . . . . .

17

2.3 First design for the banking domain model. . . . . . . . . . . . . . . . . . .

22

2.4 Second design for the banking domain model. . . . . . . . . . . . . . . . . .

24

3.1 The possible execution of two concurrent calls to the method deposit for the same Account instance. . . . . . . . . . . . . . . . . . . . . . . . . . .

33

3.2 The possible execution of two concurrent calls to the method deposit for the same Account instance. . . . . . . . . . . . . . . . . . . . . . . . . . .

34

3.3 The UML class diagram with the relationship between the class Bank and the class Account: A bank can have many accounts, but an account belongs to exactly one bank. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

35

3.4 Execution of the method transfer during the execution of the method

totalBalance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

36

3.5 Execution of the method transfer during the execution of the method

totalBalance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

36

3.6 Deadlock caused by the concurrent execution of two calls to the method

transfer, which synchronizes on the source and target accounts. 3.7 Failure during the deposit of a transfer operation.

. . . .

38

. . . . . . . . . . . . . .

39

3.8 Implementation-level class diagram for part of the banking application’s domain model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

44

viii

LIST OF FIGURES

4.1 Graphical representation of a versioned box.

. . . . . . . . . . . . . . . . .

63

4.2 The graphical notation used to represent transactions. . . . . . . . . . . . .

68

4.3 Example of 8 successfully committed top-level transactions. . . . . . . . . .

69

4.4 Example of unreachable values.

70

. . . . . . . . . . . . . . . . . . . . . . . .

4.5 Parallel deposits on the same account using the STM model based on versioned boxes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

71

4.6 Execution of the method transfer during the execution of the method

totalBalance using the STM model based on versioned boxes. . . . . . .

72

4.7 Structure that represents a versioned box with three values in its history. .

81

4.8 Values stored in the Transaction’s fields readMap and writeMap. . . .

84

4.9 Final result after transaction T2 commits. . . . . . . . . . . . . . . . . . . .

85

4.10 Several write transactions committing at the same time. . . . . . . . . . . .

86

4.11 The hierarchy of transactions implemented by the JVSTM. . . . . . . . . . .

88

5.1 The effect that the DML has on the tool chain typically used for Java application development. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 5.2 The result of compiling an entity type in DML.

. . . . . . . . . . . . . . . . 109

5.3 The result of compiling an hierarchy of entity types in DML. . . . . . . . . . 110 5.4 Methods that a DML compiler must generate for a role declaration with a multiplicity upper-bound of one. . . . . . . . . . . . . . . . . . . . . . . . . 117 5.5 Methods that a DML compiler must generate for a role declaration with a multiplicity upper-bound greater than one. . . . . . . . . . . . . . . . . . . 118 5.6 Static fields that a DML compiler must generate to hold the Association objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 5.7 JVSTM-based implementation of the class ClientState. . . . . . . . . . . 125 5.8 Fields used to store the links of an association. . . . . . . . . . . . . . . . . 126 5.9 Implementation of the methods for a role with a multiplicity upper-bound of one. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 5.10 Implementation of the methods for a role with a multiplicity upper-bound greater than one. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

LIST OF FIGURES

ix

6.1 UML class diagram for two classes with a bidirectional association between them. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 6.2 UML class diagram showing the central elements for the implementation of consistency predicates in the JVSTM. . . . . . . . . . . . . . . . . . . . . . 157

7.1 Evolution of the number of classes and associations in the Fénix domain model.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

7.2 Total transactions successfully processed by the Fénix web application from October 2006 to June 2007.

. . . . . . . . . . . . . . . . . . . . . . . . . . 172

7.3 Total daily transactions successfully processed by the Fénix web application on February 2007. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 7.4 Total hourly transactions successfully processed by the Fénix web application on the first day of enrollments, the 17th of February 2007. . . . . . . . 174 7.5 Average daily number of transactions successfully processed by the Fénix web application for each day of the week from October 2006 to June 2007.

174

7.6 Average hourly number of transactions successfully processed by the Fénix web application from October 2006 to June 2007.

. . . . . . . . . . . . . . 175

7.7 Monthly total of read transactions, write transactions, and conflicts in the Fénix web application from October 2006 to June 2007. . . . . . . . . . . . 176 7.8 Hourly average of read transactions, write transactions, and conflicts in the Fénix web application from October 2006 to June 2007. . . . . . . . . . . . 176 7.9 Hourly total of read transactions, write transactions, and conflicts in the Fénix web application on the first day of enrollments, the 17th of February 2007. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 7.10 Transactions per second processed by each method for the List Benchmark with 100% of updates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 7.11 Transactions per second processed by each method for the List Benchmark with 50% of updates.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

7.12 Transactions per second processed by each method for the List Benchmark with 10% of updates.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

7.13 Transactions per second processed by each method for the List Benchmark with 0% of updates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

x

LIST OF FIGURES

7.14 Transactions per second processed by each method for the Red-Black Tree Benchmark with 100% of updates. . . . . . . . . . . . . . . . . . . . . . . . 186 7.15 Transactions per second processed by each method for the Red-Black Tree Benchmark with 50% of updates.

. . . . . . . . . . . . . . . . . . . . . . . 187

7.16 Transactions per second processed by each method for the Red-Black Tree Benchmark with 10% of updates.

. . . . . . . . . . . . . . . . . . . . . . . 187

7.17 Transactions per second processed by each method for the Red-Black Tree Benchmark with 0% of updates. . . . . . . . . . . . . . . . . . . . . . . . . 188 7.18 Transactions per second processed by each method for the Skip List Benchmark with 100% of updates.

. . . . . . . . . . . . . . . . . . . . . . . . . . 190

7.19 Transactions per second processed by each method for the Skip List Benchmark with 50% of updates. . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 7.20 Transactions per second processed by each method for the Skip List Benchmark with 10% of updates. . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 7.21 Transactions per second processed by each method for the Skip List Benchmark with 0% of updates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 7.22 Operations per second processed by each synchronization strategy for the STMBench7 benchmark with all the long traversals disabled and a readdominated workload.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

7.23 Operations per second processed by each synchronization strategy for the STMBench7 benchmark with all the long traversals disabled and a read-write workload. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 7.24 Operations per second processed by each synchronization strategy for the STMBench7 benchmark with all the long traversals disabled and a writedominated workload.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

7.25 Operations per second processed by each synchronization strategy for the STMBench7 benchmark with all the read-write long traversals disabled and a read-dominated workload.

. . . . . . . . . . . . . . . . . . . . . . . . . . 198

7.26 Operations per second processed by each synchronization strategy for the STMBench7 benchmark with all the read-write long traversals disabled and a read-write workload. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 7.27 Operations per second processed by each synchronization strategy for the STMBench7 benchmark with all the read-write long traversals disabled and a write-dominated workload. . . . . . . . . . . . . . . . . . . . . . . . . . . 199

List of Tables

7.1 Composition of the Fénix project’s programmers team at IST. . . . . . . . . 165 7.2 Lines of code for different parts of the Fénix system. . . . . . . . . . . . . . 168 7.3 Evolution of the number of classes and associations in the Fénix domain model.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

7.4 Number of boxes accessed by each type of transaction by the Fénix web application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 7.5 Number of large transactions, for each type of transaction, in the Fénix web application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 7.6 The results for the List Benchmark with 100% of updates. . . . . . . . . . . 182 7.7 The results for the List Benchmark with 50% of updates.

. . . . . . . . . . 182

7.8 The results for the List Benchmark with 10% of updates.

. . . . . . . . . . 182

7.9 The results for the List Benchmark with 0% of updates. . . . . . . . . . . . 182 7.10 The results for the Red-Black Tree Benchmark with 100% of updates. . . . 185 7.11 The results for the Red-Black Tree Benchmark with 50% of updates. . . . . 186 7.12 The results for the Red-Black Tree Benchmark with 10% of updates. . . . . 186 7.13 The results for the Red-Black Tree Benchmark with 0% of updates. . . . . . 186 7.14 The results for the Skip List Benchmark with 100% of updates. . . . . . . . 189 7.15 The results for the Skip List Benchmark with 50% of updates. . . . . . . . . 189 7.16 The results for the Skip List Benchmark with 10% of updates. . . . . . . . . 189 7.17 The results for the Skip List Benchmark with 0% of updates. . . . . . . . . 189

xii

LIST OF TABLES

7.18 The results of the STMBench7 benchmark with all the long traversals disabled and a read-dominated workload. . . . . . . . . . . . . . . . . . . . . . 195 7.19 The results of the STMBench7 benchmark with all the long traversals disabled and a read-write workload. . . . . . . . . . . . . . . . . . . . . . . . . 195 7.20 The results of the STMBench7 benchmark with all the long traversals disabled and a write-dominated workload. . . . . . . . . . . . . . . . . . . . . 195 7.21 The results of the STMBench7 benchmark with all the read-write long traversals disabled and a read-dominated workload.

. . . . . . . . . . . . . . . . 197

7.22 The results of the STMBench7 benchmark with all the read-write long traversals disabled and a read-write workload. . . . . . . . . . . . . . . . . . . . . 197 7.23 The results of the STMBench7 benchmark with all the read-write long traversals disabled and a write-dominated workload. . . . . . . . . . . . . . . . . 198 7.24 Maximum latency results for read-only short traversals and short operations with all the operations enabled and a read-dominated workload.

. . . . . . 200

7.25 Maximum latency results for read-only short traversals and short operations with all the operations enabled and a read-write workload. . . . . . . . . . . 200 7.26 Maximum latency results for read-only short traversals and short operations with all the operations enabled and a write-dominated workload. . . . . . . 201

List of Listings 3.1 A non-thread-safe class Account in Java with the basic getBalance,

deposit, and withdraw operations. . . . . . . . . . . . . . . . . . . . . .

32

3.2 The thread-safe version of the class Account. . . . . . . . . . . . . . . . .

33

3.3 The implementation in Java, without any concerns for thread-safety, of the class Bank with the two methods transfer and totalBalance. . . . . .

35

3.4 An implementation of the class Bank that uses fine-grained locks on the accounts accessed by each method. . . . . . . . . . . . . . . . . . . . . . .

38

3.5 Reimplementation of the methods withdraw and deposit, for the class

Account, to limit the withdrawal of an amount to the balance of the account, and to refuse deposits in a closed account. . . . . . . . . . . . . . . . . . .

40

3.6 Reimplementation of the method transfer to verify, before making any change, that the whole operation will succeed.

. . . . . . . . . . . . . . . .

41

3.7 Reimplementation of the method transfer to undo the withdrawal when the deposit operation fails. . . . . . . . . . . . . . . . . . . . . . . . . . . .

42

3.8 Partial implementation of the classes Account and ClientAccount. . . .

45

3.9 Implementation of a unidirectional one-to-one association between Bank and BankAssets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

47

3.10 Implementation of a bidirectional one-to-one association between Bank and

BankAssets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

49

3.11 Implementation of the unidirectional one-to-many association between the classes Client and ClientAccount. . . . . . . . . . . . . . . . . . . . .

50

3.12 Generic implementation of the methods deposit and withdraw for the class Account. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

51

3.13 Implementation of the method convertTo. . . . . . . . . . . . . . . . . . .

52

3.14 Checking the client’s total balance before the withdrawal is performed. . . .

53

3.15 Checking the client’s total balance after the withdrawal is performed. . . . .

53

4.1 Skeleton of the generic class VBox. . . . . . . . . . . . . . . . . . . . . . . .

74

4.2 Skeleton of the class Transaction. . . . . . . . . . . . . . . . . . . . . . .

75

4.3 Complete implementation of JVSTM’s version of the HelloWorld program.

.

76

4.4 Output produced by the execution of the HelloWorld program. . . . . . . . .

76

4.5 Changes needed in the class Account to use a VBox to hold the Account’s balance.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

77

xiv

LIST OF LISTINGS

4.6 Implementation of an atomic version of the method deposit. . . . . . . . .

78

4.7 Use of the annotation Atomic to make the method deposit atomic.

. . .

79

4.8 Structure of the classes VBox and VBoxBody. . . . . . . . . . . . . . . . .

81

4.9 Some of the fields of the class Transaction.

. . . . . . . . . . . . . . . .

82

4.10 The fields of the class ActiveTxRecord. . . . . . . . . . . . . . . . . . . .

91

4.11 The operation used during the start of a new transaction to find the transaction’s number. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

93

4.12 The operation used when a transaction finishes to clean up unreachable values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

94

5.1 Syntactic rules for a domain specification. . . . . . . . . . . . . . . . . . . . 104 5.2 Grammar rules for the syntax of a value type declaration. . . . . . . . . . . 105 5.3 Default value types in the DML language. . . . . . . . . . . . . . . . . . . . 106 5.4 Grammar rules for the syntax of an entity type declaration. . . . . . . . . . 107 5.5 Examples of entity type declarations in DML. . . . . . . . . . . . . . . . . . 108 5.6 Grammar rules for the syntax of an association declaration. . . . . . . . . . 113 5.7 Examples of association declarations in DML. . . . . . . . . . . . . . . . . . 114 5.8 The Java generic interfaces Association and AssocListener. . . . . . 121 5.9 Specialization of an association using an AssocListener. . . . . . . . . . 123 5.10 Implementation of the association-aware class AssocSet.

. . . . . . . . . 129

5.11 Implementation of the generic class DirectAssociation. . . . . . . . . . 131 5.12 Implementation of the class InverseAssociation. . . . . . . . . . . . . 132 5.13 The generic interface Role. . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 5.14 The implementation of the class RoleOne. . . . . . . . . . . . . . . . . . . 133 5.15 The implementation of the class RoleMany.

. . . . . . . . . . . . . . . . . 134

6.1 Ensuring that a client has always at least an active checking account. . . . 148 6.2 Implementation of the constraints for closed accounts. . . . . . . . . . . . . 149 6.3 Methods implementing the consistency predicates for the classes A and B. . 151 6.4 Consistency predicate for checking that the client total balance is not negative.153 6.5 Consistency predicate for checking that a client always have a non-closed checking account. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 6.6 Consistency predicate for checking that a closed ClientAccount has no money.153 6.7 Consistency predicate for checking that a SavingsAccount with no money must be closed.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

6.8 Consistency predicate for checking that a CheckingAccount is in the list of accounts to process by the bank if and only if it has a negative balance. . . 154 6.9 Consistency predicates generated by the DML compiler for checking the multiplicity of the AccountOwnership association. . . . . . . . . . . . . . . . 155

Chapter 1

Introduction The research work that I describe in this dissertation is concerned with the development of domain-intensive object-oriented applications. In particular, I concentrate on the implementation of the domain model of such applications. Simply put, the main contribution of this work is the simplification, in a programmer’s friendly way, of the implementation of robust and complex domain models. In this introductory chapter, I start with a characterization of what I call domainintensive applications. Then, I discuss the general approach that guides me through my research work. These two elements are essential to present my thesis statement, which I shall do next, followed by the description of how I intend to validate the thesis. Then, I introduce some basic notation and terminology. Finally, I present the outline of the dissertation.

1.1

Domain-Intensive Applications

Programmers write computer programs, or applications, to solve particular problems. Over the years, since the early days of programming, the range of problems solved by computer applications expanded enormously, both in their scope and in their domains. Many of the first applications were developed to solve scientific and engineering problems. These applications can be classified as being numeric-intensive: Almost all the data manipulated by these applications is numeric in nature, or else is naturally represented by numbers. The emphasis in these applications is on the algorithms needed to manipulate a limited set of numeric data types. One of the first programming languages— FORTRAN—reflects in its primitives and constructs this emphasis of early programs. Another area that, shortly after the development of the first computers, benefited from the power of computer-based problem solving was enterprise data processing. Many

2

Introduction

Data-intensive applications Complexity of domain logic

Domain-intensive applications Figure 1.1: The continuous space from data-intensive to domain-intensive applications. We move from data-intensive applications, on the left, to domain-intensive applications, on the right, as we add more complexity into the domain logic of the application.

enterprises deal with large volumes of data during their daily operations. Typical examples of the early days are banks, insurance companies, or any company with many clients, suppliers, or employees. Unlike the numeric-intensive applications, the first applications developed for these enterprises do not perform complex numeric computations. Instead, they store, retrieve, and update the data that represent the various entities relevant to the enterprise’s business processes: For instance, the data about each of the enterprise’s clients. Because of this emphasis on the storage and retrieval of large volumes of data, these applications are typically classified as data-intensive. The special needs of dataintensive applications are reflected in the programming language of choice, for many years, for this kind of applications—COBOL—and in the later development of database management systems. Domain-intensive applications are a natural evolution of data-intensive applications. Like in data-intensive applications, in domain-intensive applications there are many different types of entities to deal with. In domain-intensive applications, however, the emphasis is put on the domain logic that governs the behavior of and the interactions among these entities, rather than on the volume of entities processed. Obviously, given this definition, there is no clear-cut separation between data-intensive and domain-intensive applications. Rather, they form a continuous space of applications, as depicted in Figure 1.1: We move from data-intensive applications to domain-intensive applications as we add more domain logic into the application. This trend towards more complex domain logic in the applications evolved alongside the development of the object-oriented paradigm, which have come to displace COBOL in the development of new enterprise-grade applications. In fact, object-oriented design and object-oriented programming languages matured over the last two decades, and are now the de facto standard for most of the industrial software development. Yet, even though I firmly believe that object-orientation is the approach that most

1.2 Evolution Rather than Revolution: an Engineering Approach

effectively copes with the complexity of current applications, I argue that current objectoriented programming languages are still limited in how they support the implementation of rich and complex domain models. Thus, the central subject of this dissertation is the development of domain-intensive object-oriented applications. More specifically, I concentrate on the implementation of one special element of domain-intensive applications—their domain model. The domain model of an application embodies all the knowledge about the domain of the problem that the application is meant to solve; it plays, therefore, a key role on a domain-intensive application. The importance of domain models (and thereby the relevance of this work) is reinforced by the recent surge of interest directed towards the approach of Domain-Driven Design and Development [Evans, 2003].

1.2

Evolution Rather than Revolution: an Engineering Approach

In the previous section, I described the context of my research work and justified its relevance. In this section, I shall describe the general approach that guides my work. Knowing which approach I followed is important because the approach constrains the solutions that can be developed. In fact, more than constraining the solutions, the approach that I describe here shapes them. So, by justifying the underlying approach of my work, I justify the path and many of the options I took in the development of that work.

1.2.1

General Approach

The key idea is that I should strive to propose solutions that are compatible with the way programmers currently work. The pursuit of this goal is essential, if I intend that a significant number of programmers1 ever uses the results of this dissertation. Programmers resist to revolutions, notwithstanding the merit of the newly proposed approaches. And, in most cases, I agree that their resistance is the most rational approach. Programmers invest years of their lives in education, training, and practice to become proficient with a given set of knowledge, technologies, and tools. Revolutions break with most of all this accumulated knowledge, forcing programmers to start it all over again. Naturally, I believe that revolutions are necessary for the advance of science and of human knowledge. Yet, good revolutions are hard to find and, when found, are difficult to bring into common use. In this dissertation I am not aiming at a revolution in the way of programming com1

Programmers other than myself or my research colleagues.

3

4

Introduction

puter applications. Instead, I propose a set of small evolutionary steps on how programming is currently done. I take a more pragmatic stance—an engineering approach, if you will—to the problem of improving the current state-of-the-practice on software development. Likewise, I recognize the value of the enormous amount of effort put into the research and development of current programming languages, tools, development environments, and methodologies. So, instead of trying to replace any of these, I intend to leverage on all of this work by developing solutions that integrate as seamlessly as possible with it.

1.2.2

Guiding Principles

The fundamental tenet of the approach I followed to develop this work is that my proposals should be programmer friendly, in the sense that I should make things easier for programmers, whilst giving them more powerful constructs. To accomplish this goal, I established some guiding principles, which I describe next.

Principle 1 (Target Popular Programming Languages) I should target popular programming languages, even though other languages might be technically more appropriate.

If I want to reach a wide programmer base, it makes sense to make my contributions target popular programming languages; programming languages that are commonly used to develop enterprise applications. This principle has at least one important corollary— that I should not propose radically new languages. Following this principle, I chose to develop my work in the context of the Java programming language [Arnold, Gosling, and Holmes, 2000; Joy, Guy L. Steele, Gosling, and Bracha, 2000]. Java is one of the most (if not the most) popular object-oriented programming language used to develop new enterprise applications. Java is, also, an excellent example of the evolutionary approach I discussed earlier: One of the factors for the rapid acceptance of the language by the programming community at large, was the Java language designers’ decision to adopt the syntax of C for their new language. All the examples that I show in this dissertation, as well as the implementation of my proposals, are in Java. The main results of my work, however, transcend the choice of a particular programming language. All the examples presented and implementations developed can be trivially adapted for any other modern object-oriented programming language that supports concepts similar to Java’s.2 2

As a matter of fact, although not described in this dissertation, part of this work was also implemented in ANSI Common Lisp [ANSI and ITIC, 1996].

1.2 Evolution Rather than Revolution: an Engineering Approach Principle 2 (Let Programmers Use Their Tools) I should not interfere with the tools that programmers use, neither by forcing them to change to other tools, nor by rendering the tools they use less useful. This principle is of particular importance, because the choice of programming tools is one of the most sensitive areas with regard to the “be programmer friendly” goal. Programmers’ productivity is directly related to the tools that they use, and to which they are used to. One of the consequences of this principle, in what concerns the work presented in this dissertation, is that I should not change the programming language. For instance, some of the new constructs I propose would be easier to use—from the programmers’ point of view—if I extended Java’s syntax with new keywords. Unfortunately, changing the syntax of Java interferes with many tools. The compiler is one of them: If we extend the language, then either a new compiler is needed (preventing the use of the existing compilers), or a pre-processor should be provided that transforms the extended Java’s syntax into standard Java (therefore allowing the use of standard compilers). The compiler, however, is not the only tool that depends on the syntax of the language; other such tools are modern development environments that provide editors with syntax highlighting and auto-completion based on the parsing of the program source. These tools are severely impaired by the change in the syntax, also. So, instead of extending the language, most of the extensions proposed in this dissertation were developed so that (1) they can be used by using only standard Java syntax; and (2) they are implemented in pure Java, therefore allowing the use of all the de-facto standard Java tools. The notable exception to this pure-Java approach is the Domain Modeling Language, described in Chapter 5. In this case, I propose an entirely new language, requiring a new compiler to process it. Nevertheless, because of this principle, the language and the compiler are designed to integrate well with the remaining tools, as I shall discuss later in Section 5.1. Principle 3 (Make New Constructs Simple To Learn And Use) I should take into consideration the pragmatics of newly introduced constructs, hiding the complexities inherent to their implementation, and making them easy to learn and use. A construct that is too complex is not used, or, worse, is used in the wrong way. Therefore, I should make my constructs simple. One way to accomplish this goal is to avoid adding many options to each construct: Instead of making an all-purpose construct with many configuration options, I should make simpler, one-purpose-only, constructs that can be combined into more powerful constructs. Another way of reaching the goal of simple-to-use constructs is to design them in such a way that, in the common case, there

5

6

Introduction

is not much to specify. But, if we need more complex behavior, then we can use optional configuration options to accomplish that behavior. Additionally, the difficulty in learning a new construct depends on how well the new construct integrates with the remaining constructs, either new or already existing. For instance, if a new construct replaces existing constructs—or provides an alternative way of doing something—then it should not break the programmers’ expectations on how things work. The key idea here is that, to have simple to use and to learn constructs, I should avoid surprises. The new constructs’ semantics should be natural; the semantics should be what a programmer expects, given her knowledge from the language or other languages where a similar construct exists. Finally, the simplicity of use of a construct is related, not only with its use when developing new code, but also with its effect on other development activities, such as refactoring and maintenance.

Principle 4 (Provide The Most Value With The Minimum Effort) I should find a good compromise between the usefulness of a construct and the effort needed to implement that construct.

This principle, unlike all the previous, aims primarily the concerns of whoever implements the constructs proposed here. As explained above, I implemented all my proposals without extending the language. Nevertheless, I expect that, once tested and proved useful, some of the extensions that I propose become incorporated in the programming language. Therefore, they will have to be reimplemented in that new context. Typically, more powerful constructs are more complex to implement, more costly in terms of computational resources, or both. On one hand, if a construct is difficult to implement, then tool suppliers will resist to implement that construct. On the other hand, if the implementation of a construct is computationally inefficient, then programmers will avoid the construct. Therefore, new constructs should seek a compromise between the power they provide and the effort needed to support them. In general, it is preferable to have less ambitious constructs that, still, provide good value to their users, but that, at the same time, are simple to implement. Note, however, that, despite this principle, I will not restrain myself from presenting new constructs that I find useful just because they are difficult to implement or their implementation is computationally inefficient. Rather, I use this principle as a guide to find the best compromises.

1.3 Thesis Statement

1.3

Thesis Statement

This dissertation’s thesis is that it is possible to simplify significantly the task of implementing an object-oriented domain model by making a small, non-disruptive, and easy to implement set of additions to the current object-oriented programming languages. Namely, that it is possible to achieve that goal by adding the following to a language: • Atomic actions with support for concurrency control and failure recovery • A declarative language for specifying the structural aspects of a domain model • Consistency predicates that validate an atomic action at commit-time To validate this thesis, I give specific proposals in the dissertation for adding each of these elements to the Java programming language. I describe in detail how these new constructs integrate with the Java language and demonstrate, by comparison with the current best-practices in implementing a domain model, what are the benefits of these new constructs for the implementation task. To show that the addition of these new elements may be made with small changes at the programming language level, the specific proposals made in this dissertation have a minimal interface, thereby requiring minimal learning effort from the programmers that intend to start using them. To demonstrate that such additions do not necessarily entail a disruptive change in how programmers work, the proposals made in this dissertation integrate seamlessly both with the Java programming language, and with the existing software development tools and processes. To demonstrate that these new constructs are easy to implement, I implemented all of them and I give, for each of the proposals made, a detailed description of its implementation in this dissertation. One of the additional benefits of implementing all the constructs is that, having a practical implementation for all of them allowed me the opportunity to further validate their effectiveness, by introducing their use in the development of a real-world large web application. This web application is developed by a medium-sized team of programmers and the readily adoption of the new constructs in the development of the application demonstrates both the usefulness of the new constructs and their ease of use. Furthermore, having these proposals deployed in a real-world environment for several years gave me, not only the necessary feedback to make improvements, but also the possibility of collecting statistics about the workload of a typical domain-intensive application. I report on these statistics to demonstrate that they confirm the assumptions that I used in the design of some of the proposals of this dissertation.

7

8

Introduction

result returned by

methodCall

method m1 waiting on lock

horizontal bar represents the execution of

o1.m1(arg)

method called within m1

methodCall(...) result parallel execution of

o1.m1(arg)

o1.m1(arg) and o2.m2()

o2.m2()

result

methodInstruction method call that starts the execution of m2

Time t1

o1 20 o2 "Hello"

instruction executed by m2 at this time

object’s state at instant t1

result returned by o2.m2()

Figure 1.2: Graphical notation used to illustrate the concurrent execution of methods. Time progresses from left to right, and each horizontal bar represents the execution of a method. When I need to refer to method calls within another method, I use a smaller horizontal bar super imposed on the bar of the calling method.

1.4

Notation

Throughout this dissertation, I will need to show source code that implements some part of an application. I shall use always the same language for that: Java. The code shown, however, will seldom be a complete Java class: To simplify the presentation, I will show only the fragments of code that illustrate what is being discussed at the time; it may be a class with just a few of its members, a single method, or even only part of a method. To make clear that something is missing, I will often use the sequence “...” in place of the code that is not relevant to the understanding of the fragment shown. I omit, also, the usual Java access control modifiers when their presence is not relevant to the discussion.

1.5 Outline of the Dissertation

Besides Java code, I will also need to discuss the (eventually parallel) execution of Java programs. To that end, I use the graphical notation shown in Figure 1.2 on the preceding page.

1.5

Outline of the Dissertation

This dissertation comprises eight chapters:

• Introduction. The first chapter establishes what is the subject of concern of this dissertation. Specifically, it introduces the class of domain-intensive applications and their implementation as the target of the work proposed here. Furthermore, it describes a set of guiding principles that influence significantly the type of solutions presented in the dissertation, and presents the thesis statement. • Motivation, Problem Statement, and Approach. Chapter 2 expands on the motivation given in the first chapter and situates this dissertation’s work into the larger contexts of software engineering and software development processes. It presents the domain model as a central element in the development of a domain-intensive application and introduces an extended example of a banking domain model, which is then used throughout the remaining of the dissertation. Finally, it identifies more precisely the problem that this dissertation addresses and describes the general approach that was followed to solve the problems identified. • The Difficulties of Implementing a Domain Model. To illustrate the problems described generically in Chapter 2, this chapter goes through the implementation of part of the rich domain model introduced there, concentrating on the implementation problems for which this dissertation proposes solutions. In particular, it discusses the difficulties of: implementing a domain model that is safe in a concurrent execution environment, implement the associations that are part of a domain model’s structure, and implementing the domain model’s constraints. • Versioned Software Transactional Memory. Chapter 4 proposes the use of a Software Transactional Memory as an enabling technology that allows a rather different way of implementing a domain model. It describes a novel Software Transactional Memory that was designed specifically for the needs of a domain-intensive application, and describes a practical and robust implementation of the system proposed. • Domain Modeling Language. Chapter 5 proposes a new declarative language for the implementation of a domain model’s structure in an object-oriented programming language. It describes the syntax and the semantics of the language in detail, and describes how a domain model’s structure described in that language may be transformed into a conventional object-oriented programming language such as

9

10

Introduction

Java. For that implementation, it proposes a novel pattern for the implementation of bidirectional associations. • Consistency Predicates. Chapter 6 expands on the examples of the banking domain model to introduce a set of development problems that are raised by the current way of implementing domain constraints. To solve those problems, this chapter proposes a new programming construct that allow a much simpler implementation of domain constraints. Finally, it describes how that new construct may be effectively implemented by leveraging on the support for atomic actions given by the use of a Software Transactional Memory. • Validation. To validate the applicability of the work proposed in this dissertation, Chapter 7 describes how it was used in the development of a large real-world web application. Also, it compares the performance of the Software Transactional Memory described in Chapter 4 in a series of benchmarks available in that area. • Conclusions. Chapter 8 summarizes the main contributions of this dissertation and discusses which new areas of research are opened by some of its proposals.

Chapter 2

Motivation, Problem Statement, and Approach In the previous chapter, I stated that the ultimate goal of this work is to simplify the implementation of domain models in object-oriented domain-intensive applications. The reasons for pursuing this goal, however, were only generally addressed. In fact, the only justification that I gave was the increasing importance of domain-intensive applications in the development of software. In this chapter I expand on the reasons that led me to embrace this work. I begin, in Section 2.1, by discussing the role that domain models play into the wider context of a complete software development process; the purpose of this discussion is to clarify why I concentrate on the implementation of domain models in this dissertation. Then, in Section 2.2, I introduce an extended example of a rich domain model, with a twofold purpose: (1) to be used in Chapter 3 to illustrate the problems with the current bestpractices for implementing domain models; and (2) to be used throughout the remaining of the dissertation as a running example to demonstrate the applicability of my proposals. In Section 2.3, I describe, in general terms, what are the specific problems that I am trying to solve in this dissertation. Finally, in Section 2.4, I discuss two general approaches to simplify the implementation of a domain model and identify which one I adopted in the course of this work. Even though the set of domain-intensive applications neither contains nor is contained by the set of enterprise applications, there is a significant overlap between the two. In fact, in what concerns the implementation of a domain model, they may be considered practically the same. So, without loss of generality, in what follows I shall refer to some literature on the development of enterprise applications to describe some of the bestpractices for the development of domain-intensive applications.

Also, given that the

context of this work is already well-established, I shall often use the term application,

12

Motivation, Problem Statement, and Approach

without further qualifications, to refer to a domain-intensive application.

2.1

Domain Models in the Software Development Process

The development of software may range from single-person small programs that comprise just a few lines of code, to ultra-large-scale software systems with millions of lines of code developed by large teams of people. Whereas in the first case it may be relatively simple to develop the program in a completely ad-hoc way, in the latter case, the mere dimension of the system introduces problems in the management of the development process that are very hard to solve [Gabriel, Northrop, Schmidt, and Sullivan, 2006; Polak, 2006]. In general, to be able to develop with success a non-trivial application, it becomes crucial to have some way to tackle the complexity of the software development process [Royce, 1987]; in a very general sense, this is the subject of the software engineering discipline. The software engineering discipline comprises many different subareas of expertise— from requirements engineering to software maintenance, going through project planning, domain analysis, or software testing, to name just a few. Each of these areas of expertise addresses only part of the entire software development process.

2.1.1

Software Development as the Transformation of Artifacts

One way to look at the software development process is as a series of activities that either create new artifacts from scratch or that transform previously built artifacts into other artifacts. For instance, the activity of requirements elicitation typically produces one or more documents detailing the requirements that describe the problem that the program should solve; then, from these requirements, other activities produce other artifacts, such as: a plan for the execution of the project, a set of documents describing the software architecture of the system, or the source code of the program that solves the problem. The exact nature of the activities and the artifacts of a software development process may vary considerably from one process to another. They depend not only on the complexity of the system, but also on the methodology and on the development process model that is adopted by whoever is developing the system. Naturally, some of the activities are simpler than others. In some cases an activity may be performed automatically by some tool, but in most cases they require hard human labor and high expertise. The high complexity of software development results, precisely, from the combined difficulty of performing each of its activities. Thus, a contribution that simplifies one of the activities simplifies, also, the software development process as

2.1 Domain Models in the Software Development Process

a whole. The work described in this dissertation addresses a very specific activity—the transformation of a domain model into the source code that implements it. This activity is commonly called implementation or coding. It is, also, an activity that is always present in a software development process, regardless of the methodology or process model chosen. In fact, the only artifact that a software development process must always produce is a computer program (or set of programs) that may be executed on a computer; it is the execution of this program that solves the problem that led to the software development process in the first place.

2.1.2

Domain Model: A Central Artifact in the Development Process

A domain model, like any model, is a representation of (part of) the real thing; in this case, a representation of an application’s domain. The domain of an application is the sphere of knowledge comprising all the entities and the processes of the real world that is the subject area of that application. A domain model is an abstraction that represents only the parts of the application’s domain that are relevant to the development of the application. A domain model may take many forms and may serve many purposes, but it is indispensable in the development of an application. Even if it only exists in the mind of the person that is developing the application: After all, that person would not be able to write the application if she had no knowledge about the application’s domain. Indeed, for many small programs, the domain model only exists in the mind of the developer, and is never put into a more explicit and available form (other than the program implementing it). Yet, for larger software systems, involving more people, it becomes infeasible not to have the domain model explicitly represented. It may take the form of a document in natural language, a formal representation using some kind of formal language, or some combination of these. In fact, it is common, nowadays, to use the UML language [Booch, Rumbaugh, and Jacobson, 1999] to represent parts of the domain model and complement it with natural language descriptions. The domain model is a useful artifact for most of the activities in a software development process. Its construction starts with the activity of requirements elicitation, evolves through the analysis and the design phases, and influences such late phases in the process as the quality assurance, and the system maintenance. The domain model plays, therefore, a central role in the development of any software system. Yet, despite its importance, only in the late eighties did the research community of software engineering started to address the activity of domain modeling as a research

13

14

Motivation, Problem Statement, and Approach

problem in itself [Iscoe, Williams, and Arango, 1991].

Since then, domain modeling

became a well-established practice in the area of software development; it is now common to see software developers starting the implementation of a program, regardless of its size or complexity, by discussing over a set of UML class diagrams that correspond to part of the application’s domain model. This trend towards domain modeling by the professionals in the area reflects, in my opinion, the increasing demand for applications with broader scopes than ever: As the applications’ scopes increase, so do their domains and, consequently, the need for domain modeling. So, explicit domain models and domain modeling become more important as we move away from data-intensive applications into the development of domain-intensive applications.

2.1.3

Domain Model at Various Levels: Analysis, Design, Implementation

In this dissertation I do not address the task of building a domain model. Rather, I assume that it is already built using standard practices in the area. Once it is built, however, it needs to be expressed in software—that is, implemented in a program that may execute to solve the application’s problem. As mentioned above, the target of the research described in this dissertation is precisely this transformation from a domain model to source code. This is an area where improvements are particularly welcomed, because, in general, the implementation is the most labor-intensive activity during the development of software. Not only because the program is the most detailed artifact of the entire software development process, but also because implementing a domain model is not just a matter of filling in the missing details; rather, it involves the conversion of the concepts of the domain model, which are expressed using the constructs of a particular domain modeling language, into semantically equivalent concepts that are expressed using the constructs available at the programming language level. Thus, the difficulty of implementing a domain model is greater when there is a greater mismatch between the modeling language and the programming language. Traditionally, given the wide gap between these two languages, the common advice in the software engineering area has been to consider intermediate artifacts to facilitate this transformation. So, instead of implementing the domain model—which is primarily an artifact of the analysis phase—the traditional software engineering best-practices prescribe the construction of an intermediate design model. The goal of the design model is to bring the concepts of the domain model closer to the constructs available in the programming language, thereby facilitating the domain model’s implementation. Unfortunately, having yet another language to represent the design model compli-

2.1 Domain Models in the Software Development Process

15

cates the software development process and increases the probability of translation errors among the different models. To solve this problem, recent research in the software engineering area propose to reduce the distance between the analysis model (also called conceptual model) and the design model by using a single modeling language for both models (see, for instance, [Evermann and Wand, 2005]). In fact, one of the goals of the model-driven design approach is to merge the analysis and the design models into only one model [Evans, 2003, Chapter 3]. As the difference between these two types of models is not relevant for what I present in this dissertation, I will not distinguish between both types of models in the remaining of this document. Instead, I shall use the term domain model to refer to the artifact from which software developers start their implementation activity. Moreover, I shall refer generically to the task of passing from the problem description to a domain model as the design task, rather than talk about analysis and design separately. Finally, even though my contributions are at the programming language level and are, therefore, largely independent of the language chosen to represent the domain model, I shall use only UML class diagrams and natural language descriptions to represent domain models.

2.1.4

Standard Architecture for a Domain-Intensive Application

To build an application and have it work as expected, we have to consider several other aspects, other than the implementation of its domain model:

• User interface: There must be code in an application to present an interface to the user, to accept the user’s data, and to validate that data before further processing. These requirements, however, are not part of the domain model.

Rather, they

are usually represented in artifacts such as use cases, presentation models, and navigational models, from which software developers implement the corresponding code in the application. • System qualities: To satisfy architectural requirements such as security, scalability, performance, or availability, an application must include code that implements the tactics necessary to achieve those qualities. • Technological environment: Applications run in some execution environment and must, therefore, have code that implements the interfacing with that environment. For instance, code that accesses a database or that handles network requests.

Obviously, all these elements must be put together with the code that implements the domain model to make a complete application. Whether the code that implements these

16

Motivation, Problem Statement, and Approach

Presentation

Domain

Data Source

Figure 2.1: The layered architecture from [Fowler, 2002]. This is a pure layered architecture, where each layer (represented by a box) may use only the layer immediately beneath it.

various concerns is intermingled with one another or separated in its own module, is a matter of software architecture. The advantages of modularization are well-known in the area of software development [Parnas, 1972]. One of those advantages is the ability to separate by different modules code that we expect to change at different rates, so that changes in one module do not influence significantly code outside that module. The difficulty, of course, is in knowing which parts of an application are expected to change more than others. Yet, over the years, the software development industry converged into the common assumption that changes in the code that implements the domain model of an application occur independently of changes in the code that deals with the user interface, for instance; likewise for the code that interfaces with the execution environment. Whereas the domain model varies only with changes in the application domain, the other two vary either with the available technology or the deployment environment. This difference became more acute, first with the advent of the web-enabled applications, and then with the growing trend towards service-oriented architectures. In both cases, organizations felt the need to deploy their existing applications using either new ways of accessing the application, or using a rather different infrastructure, while retaining the same domain logic in the application. Therefore, it is now a common practice in the area of enterprise software development to adopt a layered software architecture for the development of an application.1 In particular, an architecture that separates the implementation of the domain model from the remaining aspects of the application. For instance, Fowler [2002] proposes the three-layered architecture that I show in Figure 2.1, whereas Evans [2003] proposes the architecture shown in Figure 2.2 on the next page. 1

For a good presentation of the layered architectural style, see [Bass, Clements, and Kazman, 2003] and [Clements, Bachmann, Bass, Garlan, Ivers, Little, Nord, and Stafford, 2003, Chapter 2].

2.1 Domain Models in the Software Development Process

User Interface

Application

Domain

Infrastructure

Figure 2.2: The layered architecture from [Evans, 2003]. Unlike the layered architecture depicted in Figure 2.1, in this case each of the layers may use all the layers below it: The User Interface layer may use all the remaining layers; the Application layer may use both the Domain and the Infrastructure layers; and the Domain layer may use only the Infrastructure layer.

Although the two architectures are not exactly the same, in both cases there is a domain layer—that is, a module that contains the implementation of the domain model separated from the remaining aspects of the application (aspects such as the user interface). Moreover, this layer is restricted to use only the layer beneath it: the layer providing infrastructural support. Thus, to understand the implementation of a domain model we may need to consider also the infrastructural layer, but nothing else.

2.1.5

The Domain Layer Dependence on the Infrastructural Layer

Ideally, the domain layer should remain independent of any other layer in the system. Given that the domain layer implements the application’s domain model, it should not be affected by changes that may occur in other parts of the application that deal with aspects not related to the domain model. In practice, however, in the standard architecture for an enterprise application, the domain layer appears above the infrastructural layer. This dependence results from the fact that the implementation of a domain model for an enterprise application needs to take into consideration a range of other concerns that are extraneous to the application’s domain model. These concerns—often called non-functional requirements—include such things as: the need to support concurrent access to the domain’s entities, the need to support the distributed execution of the application, the need to store persistently the information about the domain, or the need to inter-operate and integrate with other external applications. So, whereas the responsibility of supporting most of these requirements

17

18

Motivation, Problem Statement, and Approach

rests upon the infrastructural layer, the domain layer may still need to use the infrastructural layer’s services to provide an implementation of the domain model that satisfies these requirements. A solution that is commonly used in an enterprise application, and that addresses many of these requirements, is the use of an external database management system to store persistently the information about the domain’s entities. Unfortunately, this common approach has the undesirable consequence of influencing the programming model that is used to implement the application’s domain: Notwithstanding the best efforts of patterns, frameworks, and tools, that try to isolate the implementation of the domain model from the specifics of persistently storing the information about the domain,2 that isolation is not perfect and the usefulness of the object-oriented programming languages becomes severely impaired, thereby further adding to the difficulty of implementing a domain model. Furthermore, by conflating the solutions of all the problems into a single approach makes it more difficult, not only to analyze each of the problems and to improve on each of the solutions separately, but also to employ that solution in contexts where not all the requirements exist—for instance, when no need for persistence exists. Therefore, in this dissertation I shall ignore all the extraneous requirements for a domain model and concentrate instead on how best can we implement the domain model’s core requirements with the current object-oriented programming languages.

2.2

Example of an Application in the Banking Domain

To discuss the difficulties of implementing a domain model, I shall use throughout this dissertation a simple example of an application from the banking domain. The banking domain is a well known example, often used in the literature of objectoriented programming. But, whereas in most cases, the example is given with only a couple of entities and relationships, in this section I extend the example with more entities and relationships; also, entities have more properties and more complex behaviors than usual.

2.2.1

Example’s Rationale

Although this extended example is still an oversimplification of a real problem domain, it exhibits some of the complexities that are typical of rich domain models. Thus, it suffices to illustrate the problems that programmers face when they implement such domain 2

In fact, many of the patterns described, for instance, in [Fowler, 2002; Alur, Crupi, and Malks, 2001] target specifically this problem.

2.2 Example of an Application in the Banking Domain

models. In fact, I want to show that, even in such simplistic examples, we encounter problems that are difficult to solve using current object-oriented languages. By presenting this extended example, however, I do not intend to discuss its implementation in all the details. Also, I shall not discuss all the possible alternatives either for the design of a solution, or for the implementation of a given design, as they are too numerous. Rather, I shall present a design that I deem as reasonable, and, for that design, I shall concentrate only on the implementation aspects that illustrate the problems of the current approaches and the advantages of my proposals. Finally, I shall use this example, not only as a means to introduce the problems that I propose to solve in this dissertation, but also to introduce some terminology related to the development of domain models.

2.2.2

Application’s Functionality

The core functionality of the banking application is the management of the accounts that belong to the clients of a bank. The clients’ accounts may be either checking accounts, or savings accounts. Clients may have as many accounts of each kind as they want, but they must own at least one checking account—thus, when a new client is added to the bank, a new checking account is created for that client. Each account, on the other hand, is owned by a single client. As usual, accounts have a current balance, which corresponds to some monetary amount in some particular currency. Deposits and withdrawals into an account change the account’s current balance by the amount deposited or withdrawn. Yet, when the amount to deposit or to withdraw is not in the same currency as the account’s balance, the amount deposited or withdrawn must be converted to the account’s currency. Moreover, depending on the currencies involved in the exchange and on the account for which the exchange is made, the bank may charge the requesting account with a currency-exchange fee. Whereas clients are allowed to make as many deposits and withdrawals as they want from their checking accounts, the operations on savings accounts are much more restricted. Savings accounts are created for a given time period—for example, seven days, or three months—and with a corresponding interest rate. Clients may create a new savings account whenever they want by choosing one time period, and by transferring some amount from one of their checking accounts. At the end of that time period, the bank automatically transfers the savings account’s balance, plus interest, to the checking account that was used to create the savings account. Therefore, clients may earn money (in the form of interest) by putting their money in savings accounts. The tradeoff is that they cannot make further deposits into the savings account, nor make partial withdrawals.

19

20

Motivation, Problem Statement, and Approach

Clients, however, are allowed to withdraw all of the savings account’s balance before the end of the time period is reached, but in that case the bank does not pay interest. Instead, the bank charges a fee for the anticipated withdrawal. Finally, once the balance of the savings account is withdrawn, either by the client or by the bank, the account is closed. Checking accounts may be closed, also, provided that their balance is zero, and that all of their corresponding savings accounts are closed. Naturally, neither deposits nor withdrawals are allowed for a closed account. The bank allows that checking accounts have negative balances, but the total balance of each client—that is, the sum of the balance of all the client’s accounts—must be greater or equal to zero. A checking account with a negative balance, however, pays interest to the bank, on a daily basis. At the end of each day, the bank calculates the interest due by each of the checking accounts that have a negative balance, and charge each of the accounts accordingly. Only then, it processes the savings accounts that end in that day. If, during this process, a client’s total balance becomes negative, all the accounts for that client are closed. To pay the interest, or to receive the fees and the interest that is due in each of the above mentioned situations, the bank itself has an account. This account has no restrictions on its balance. Moreover, the deposits and withdrawals from this account that need a currency conversion are not charged with a fee. Finally, to issue periodic reports about the bank’s managed accounts, the bank needs to calculate both the total of the client’s checking accounts and the total of the client’s savings accounts. Each of these totals is the sum of all the corresponding account’s balances.

2.2.3

Basic Domain Modeling Terminology

From the description of the banking application’s responsibilities in the previous section, it is straightforward to identify some of the objects that are relevant to the functioning of the application—objects that belong to the application’s domain. For instance, objects such as bank, client, checking account, savings account, monetary amount, currency, and time period result from the simple approach of identifying the nouns used in the description of the functionality. Typical object-oriented analysis and design methodologies3 start with these objects and proceed by identifying the attributes and the behaviors that characterize each of the objects, as well as the associations among them. It is common practice, however, to distinguish between entities and value objects (see, for instance, [Evans, 2003, Chapter 5], [Fowler, 2002, Chapter 18], and [Riehle, 2000, 3

For an example, see [Booch, 1994].

2.2 Example of an Application in the Banking Domain

Chapter 3]). Entities are objects that have a distinguished identity in the program. On the contrary, value objects do not have a distinguished identity. Rather, they are used only for their value. Thus, we can replace an instance of a value object by another that has the same value without affecting the semantics of the program. Whereas both kinds of objects typically have attributes that describe the object state, only entities may have their state changed during the program execution; value objects are immutable and, once created, represent always the same value. Examples of entities from the banking domain are banks, clients, checking accounts, and savings accounts. Also from that domain, examples of value objects are monetary amounts, currencies, and time periods. This distinction between entities and value objects extends naturally to their types. So, I shall use the terms entity type and value type to refer to the type of a particular entity and value object, respectively. As a matter of fact, the types, rather than their instances, are the elements that are most used in the representation of a domain model’s structure. Besides entity types and value types, the other elements which are essential to represent the structure of a domain model are associations.4 Associations represent relationships between entities—for example, the relationship between clients and their accounts is represented in the domain model by an association between the client type and the account type. Binary associations may be either unidirectional or bidirectional, depending on whether they are traversable in only one or in both directions, respectively. A binary association between two types A and B is traversable from A to B if the instances of A can refer to the instances of B with which they relate to. Also, following the terminology used in the UML 2.0 specification [Object Management Group, Visited in 2007], I shall use the term link to refer to an instance of an association—that is, the pair of objects that relate to each other. But, naturally, not all domain knowledge is structural. In a typical domain model, such as the one described above, entities are subjected to constraints, also. A constraint on one or more entities is a condition that those entities must satisfy. For instance, in our example, that the total balance of each client must be greater or equal to zero. Finally, another crucial element of a domain model is the description of its processes. A process is a sequence of domain activities, possibly affecting some entities, that is executed for a given purpose. It is important to know, not only which processes exist, but also what they do, and how they do it. An example of a process in the banking domain is the daily processing of the interest due by each of the checking accounts.

4

Even though some modeling methodologies and languages distinguish between different types of relationships—for example, aggregations and compositions—in this dissertation I do not make such a distinction and consider only associations.

21

22

Motivation, Problem Statement, and Approach

bank 1

client 0..* 1 owner

Bank

1 owner

Client

bank 1

assets 1

Bank Assets

checking accounts 1..*

Checking Account

savings accounts 0..* 0..*

1 checking

savings

Savings Account

Figure 2.3: First design for the banking domain model. In this design, only the classes and the associations that are mentioned explicitly in the problem’s description are used.

2.2.4

Initial Design of the Banking Domain Model

Different software development processes, by definition, advocate different methodologies for the development of an application. Heavy-weight software development processes try to start with a complete elicitation of requirements and with a detailed planning of the development. Agile processes, on the other hand, start by concentrating on the users’ requirements with higher priority and on the rapid development of application prototypes that embody part of those requirements. Yet, regardless of which process is used, software developers have to make at least a minimal set of design decisions before they start implementing an object-oriented application. Indeed, before programmers start making new classes in some object-oriented programming language, they have to decide which classes they need for the problem that they are trying to solve. Thus, a common artifact that is developed during the early phases of an object-oriented development process is the application’s class structure—that is, which classes and which associations exist in the application’s domain model. The de-facto standard used to represent such class structures is the UML’s graphic notation for class diagrams. Often, several class diagrams are used to represent an application’s class structure, each one of the diagrams showing part of the class structure. In this case, however, because the domain is simple, I can show the entire class structure in a single class diagram. For instance, in Figure 2.3, I show the entire class structure of a possible design for the functionality described in Section 2.2.2. The design depicted in Figure 2.3 contains classes for each of the entities mentioned in the description of the banking application functionalities. In fact, I had identified already all but one of these entities in Section 2.2.3. The new class in the diagram is the class bank assets, which represents the account used by the bank to pay and to receive the money that is due as a result of the various banking operations. Regarding the relationships between domain entities, again, the associations used in this design

2.3 Problem Statement

correspond to each of the relationships that are explicit in the problem’s description. But, from an object-oriented programming perspective, this design has a few shortcomings. Checking accounts, savings accounts, and bank assets are all different kinds of accounts, with things in common—for example, all the accounts have a current balance and an operation that allows us to deposit some amount. Yet, this design fails to capture this commonality. Likewise, both the checking accounts and the savings accounts are client accounts, for which fees may be charged when the amounts deposited or withdrawn are in a different currency. Moreover, although with this design we can reach all the accounts of a bank by going through the bank’s clients (who own the accounts), it is easier to implement some of the application’s functionalities if there is an association between the bank and its client’s accounts. As a matter of fact, it is useful to have several of such bank–account associations. For instance, for the daily processing of the checking accounts with a negative balance, it would be helpful to have in the bank the set of all the checking accounts with a negative balance; for processing the savings accounts that reached the end of their time period, it would be helpful to have in the bank a list of savings accounts sorted by their term date; and, for finding a client account given its account number, it would be helpful to have in the bank a map that indexes the bank’s accounts by their number. In Figure 2.4 on the next page, I show an alternative design that takes some of these aspects in consideration. Note that the two associations that, in the first design, exist between the client and the two kinds of client’s accounts, are replaced in this second design by a single association between the client and the class that generalizes both kinds of accounts—the class Client Account. Yet, whereas in the first design it was explicit in the class structure that each client must have at least a checking account, in the second design this structural restriction is no longer represented in the class diagram. Even though the design space for this problem has many other solutions, I shall not discuss them here. I remember that my goal with this exercise is not to discuss which design is best. Rather, I want to discuss the implementation of domain models. Thus, I shall use this second design as a reasonable solution to this problem and base my discussion on it, which I shall do in Chapter 3.

2.3

Problem Statement

To implement a domain model, we must find appropriate representations for all the domain model’s elements in our programming language. In this task lies much (if not most) of the software development complexity.

23

24

Motivation, Problem Statement, and Approach

Account

1

Bank Assets

Client Account

1..*

1 owner

account

Client client 0..*

Checking Account 0..*

1 checking

0..* savings

Savings Account 0..* 1 0..1

Bank

1

0..1

Figure 2.4: Second design for the banking domain model. This design has a more complex structure than the design presented first. Two new classes were added to capture the commonality between the different kinds of accounts. Moreover, in this design there are two redundant associations between the class Bank and the classes Checking Account and Savings Account, to simplify the bank’s daily processing of accounts.

2.3 Problem Statement

The use of the object-oriented programming paradigm helps to some extent. For instance, we may use classes to represent entities’ types, slots to represent entities’ attributes, and methods to represent some of the domain model’s processes. Unfortunately, this apparent ease of implementation does not extend naturally to all the remaining concerns of a domain model’s implementation. Indeed, I argue that what makes the implementation of a domain model such a difficult task is that the current object-oriented programming languages lack the appropriate expressiveness to implement some of the typical requirements of a domain model. In this dissertation, I address specifically the following areas where current objectoriented programming languages have manifest shortcomings: • Representing associations: Even though associations are essential in the representation of a domain model’s structure, their implementation is not as simple as the implementation of other structural aspects of a domain model, as we shall see in Section 3.3. On the contrary, implementing correctly an association between two entities may turn into a difficult task; more so, if we consider bidirectional associations, which must remain consistent even in the presence of failures during the establishment of a link between two objects. • Representing constraints: Like associations, constraints on domain entities are present in every domain model. But, whereas implementing very simple constraints on a single object may be relatively easy, implementing constraints that involve more than one object becomes very difficult on a moderately complex domain, as we shall see in Section 3.4.2. • Concurrent execution: A common requirement for enterprise applications is that they be able to process multiple requests from their users simultaneously. As the common solution for this problem is to have each request processed by a different thread, we have to face the difficulties of concurrent programming. The implementation of a domain model is particularly sensitive to this fact, because domain entities are implemented as shared mutable objects that must maintain their identities throughout the entire application’s life-cycle. Thus, we must ensure that these objects are consistently manipulated in a concurrent setting, which is not at all simple with the current object-oriented programming languages, as we shall see in Section 3.1. • Failure Recovery: Domain operations may fail to complete, either because their execution would violate a domain constraint, or simply because they received erroneous information. Most often, when that happens, nothing should change in the domain’s state. But, given that current programming languages have no provision to undo changes, the implementation of domain operations must verify all the preconditions necessary for the success of the operation before they make any

25

26

Motivation, Problem Statement, and Approach

change, or else, they must explicitly undo the changes in case of failure. This style of programming, however, is very difficult to use in more complex operations, as we shall see in Section 3.2.

2.4

Two Approaches to Solve the Problem

Now that I explained why implementing a domain model is an important and difficult task, and that I want to contribute to its simplification, the question that remains to be answered is: How do I propose to accomplish it? The full answer to this question, of course, lies ahead in the remaining of this dissertation. But, before I conclude this chapter, I present two general and complementary approaches that we may use to solve this problem, so that I make clear which approach I am following in this work.

2.4.1

The MDA approach

One way to simplify the implementation of a domain model is to automate it. The general idea in this case is to use code generation tools to generate automatically all the source code necessary to implement a domain model, much in the same way as we use a compiler to generate an executable program from its source code. This approach is called the model-driven development approach [Selic, 2003] and is best illustrated by the proposal of the Model Driven Architecture (MDA) framework by the Object Management Group [Mellor, Scott, Uhl, and Weise, 2004; OMG, 2007]. The MDA approach proposes that, rather than implementing an application in a specific execution platform, developers should specify instead a platform independent model (PIM) for their application; then, this PIM is transformed into a platform specific model by way of MDA tools that generate all the code. To accomplish this level of automation, however, the PIMs must be much more detailed than domain models typically are, or even than currently used domain modeling languages allow them to be. Indeed, this need for operational models brought up by the MDA approach is one of the major incentives for much of the work on the UML 2.0 specification [Object Management Group, Visited in 2007] and on the idea of an executable UML [Raistrick, Francis, and Wright, 2004]. Yet, even though the MDA approach is enticing, not all researchers agree that it is the best approach to solve the software development process [Thomas, 2003; Thomas and Barry, 2003], or that UML provides a good foundation for it [Hailpern and Tarr, 2006; France, Ghosh, Dinh-Trong, and Solberg, 2006]. In fact, it is not clear whether an MDA approach is feasible with current proposals.

2.4 Two Approaches to Solve the Problem

Also, even if we consider that it becomes a reality, it is questionable whether the development of the PIMs would be any simpler than implementing the source code, given the level of detail necessary to make them fully executable. Finally, if the PIMs are meant to be our domain models, then they loose part of their usefulness of being a higher-level and abstract description of our programs; if, on the other hand, the PIMs are not meant to be the domain models, then we have to face again the task of transforming a domain model into a PIM, bringing us back to where we started. A less ambitious approach, but still with the same general idea of generating code automatically, is to consider only a fragment of the UML language, generate automatically code for that fragment, and let the programmers complement the generated code with the missing pieces that are necessary to complete the program. For instance, some proposals along these lines exist for generating a set of classes in an object-oriented programming language given a UML class diagram describing those classes [Harrison, Barton, and Raghavachari, 2000; Génova, del Castillo, and Llorens, 2003]. One of the difficulties in this case is to avoid round-trip problems—that is, that the code may be regenerated without throwing away the changes that the programmers made manually.

2.4.2

This Dissertation’s Approach: Reduce the Gap between Languages

Most of my proposals in this dissertation follow an entirely different route (with a notable and justified exception that I shall present in the end of this discussion). As we saw before, the difficulty in implementing a domain model is caused by an impedance mismatch between the two languages used in this task: the modeling and the programming languages. So, another way to alleviate the problem is to reduce that impedance mismatch, by bringing the two languages closer together. In principle, if we make our programming language more expressive, we expect that our implementation tasks become easier in return. In fact, it is not only a matter of being more expressive in general, but of being more expressive towards our domain modeling language—that is, of having in the programming language constructs that correspond more closely to the constructs that are used at the domain modeling level. Object-oriented programming languages are an excellent example of the effectiveness of this approach. By adding to procedural programming languages constructs such as classes, objects, inheritance, or polymorphism, programming language researchers created a new programming paradigm that allows us an arguably simpler implementation of our domain models; this happens because the newly added constructs allow a more direct representation of the domain modeling concepts. For instance, implementing an entity type as a class in an object-oriented programming language is much simpler than in a procedural language that does not provide an equivalent construct.

27

28

Motivation, Problem Statement, and Approach

Certainly because of their adequacy for implementing domain models, object-oriented programming languages turned into the mainstream languages for most of the current development of domain-intensive applications. Still, there are plenty of space for improvements. To summarize the general approach that I follow in this dissertation: I propose to extend object-oriented programming languages5 with constructs that facilitate the mapping from a domain model to its implementation. More specifically, I propose the addition of both atomic actions and consistency predicates to the programming language. Atomic actions will be discussed more thoroughly in Chapter 4. My proposal of consistency predicates, which leverages on the support given by atomic actions, is detailed in Chapter 6. Also, I propose in Chapter 5 a new language to implement the structural aspects of a domain model: the DML language. Although the design of DML remains faithful to the approach that I just described, it differs from the remaining proposals in that it is not an extension to an existing programming language. Rather, it is an entirely new language, for which I implemented a compiler that generates Java source code. In this regard, it resembles one of the MDA-like approaches that I described earlier in this section. Yet, whereas in those MDA-like approaches the central idea is to generate code from a domain model expressed in a modeling language such as UML, I propose DML as a programming language. That is, the purpose of DML is not to replace any of the existing domain modeling languages, but to serve as a language to implement domain models. Finally, even though all of my proposals specifically target the implementation task, I expect them to have repercussions in other phases of the software development process as well. A similar effect occurred with the use of object-oriented programming languages: Their use fostered the use of object-oriented analysis and design methodologies.

2.5

Summary

This chapter establishes the context of this research work in the larger context of the software engineering discipline. In particular, it identifies the task of implementing a domain model as the target of the work. After discussing briefly what is a domain model, I introduce an extended example of a rich domain model that is used throughout the dissertation. This example serves also to introduce some basic terminology on domain modeling. 5

Even though all the work in this dissertation is specialized for the Java programming language, most of the results apply equally to any other mainstream object-oriented language.

2.5 Summary

Then, I introduce the implementation problems that I propose to solve with this work. Namely, the difficulty in implementing associations, domain constraints, and domain operations that may fail, all of these in the context of a concurrent program. Finally, I present my approach to solve the problem, which consists in extending the programming languages with new programming constructs that allow them to implement the domain model elements more easily.

29

30

Motivation, Problem Statement, and Approach

Chapter 3

The Difficulties of Implementing a Domain Model The specific problems that I propose to solve with this work, were already introduced in Section 2.3. In this chapter, I get into the details of implementing part of the domain model for the banking application. The goal of this exercise is to illustrate the difficulties that programmers encounter when they have to implement rich domain models with current object-oriented programming languages. Following the same order on which the solutions shall be presented later, I begin, in Section 3.1, with the problem of implementing a domain model in a concurrent program, followed, in Section 3.2, by the problem of implementing operations that may fail. The solution proposed for both of these problems is the use of atomic actions, which are described in Chapter 4. Then, in Section 3.3, I address the problem of implementing the structural aspects of a domain model, giving special attention to the implementation of associations. My proposal to simplify the implementation of a domain model’s structure is to use the DML language, which is presented in detail in Chapter 5. Finally, in Section 3.4, I concentrate on the implementation of the domain model’s behavior and constraints. To solve the problems identified in this latter section, I propose, in Chapter 6, the use of consistency predicates.

3.1

Implementation of a Concurrent Domain Model

To demonstrate the problems that programmers face when they have to implement a concurrent domain model, we do not need to go much farther in the complexity of a domain model. In fact, even in rather simplistic examples we encounter problems that

32

The Difficulties of Implementing a Domain Model

class Account { long balance; Account(long balance) { setBalance(balance); } long getBalance() { return this.balance; } void setBalance(long balance) { this.balance = balance; } void withdraw(long amount) { setBalance(getBalance() - amount); } void deposit(long amount) { setBalance(getBalance() + amount); } } Listing 3.1: A non-thread-safe class Account in Java with the basic getBalance, deposit, and withdraw operations.

are already difficult to solve using current object-oriented languages. So, in this section, I shall use a stripped-down version of the banking application introduced in Section 2.2. I start only with accounts and their balances, ignoring that the accounts’ balances may be in different currencies. As operations, consider that we may consult an account’s balance, deposit some amount, or withdraw some amount.

3.1.1

Basic Thread-Safety

In Listing 3.1 I show a reasonable implementation of the class Account in Java, with no concerns for thread-safety. The lack of thread-safety in the class Account means that, in the presence of multiple executing threads, the interleaving of the various threads can cause inconsistent updates (and readings) of the instance variable balance: We can have an account with a balance of 100, make two deposits of 50 each and arrive at a state where the final balance is 150, instead of 200. This situation is illustrated in Figure 3.1 on the next page, where I show

3.1 Implementation of a Concurrent Domain Model

33

setBalance(150) getBalance() 100 acc.deposit(50) acc.deposit(50) getBalance() 100 setBalance(150) Time t1

t2

t3

acc 100

acc 150

acc 150

Figure 3.1: The possible execution of two concurrent calls to the method deposit for the same Account instance. Because no synchronization exists, the final account’s balance is not correct—one of the deposits was lost. The notation used in this Figure was introduced in Figure 1.2.

class Account { ... synchronized long synchronized void synchronized void synchronized void }

getBalance() {...} setBalance(...) {...} withdraw(...) {...} deposit(...) {...}

Listing 3.2: The thread-safe version of the class Account. Thread-safety is obtained by adding the synchronized modifier to all the class’s methods.

two concurrent executions of the method deposit for the same Account object, acc. Both executions call the method getBalance before any change is made to the balance of the account. Thus, both obtain the same result—the balance of the account before any of the two deposits. Then, each execution of the method deposit adds to that value the amount to deposit and sets the new balance. The problem is that the final call to the method setBalance overwrites the change made by the previous execution. This problem is widely known, of course, just as well as its trivial solution: To make the class thread-safe we just have to add the synchronized modifier to each of the methods in the class, as sketched in Listing 3.2. When a thread executes a synchronized method on a particular object, the thread must acquire, first, an exclusive lock associated with that object, which is released at the end of the method execution. While the thread is holding the exclusive lock, no other thread can acquire the same lock. So, the execution of a synchronized method for a given object cannot be interleaved with the execution, by

34

The Difficulties of Implementing a Domain Model

setBalance(150) getBalance() 100 acc.deposit(50) acc.deposit(50) getBalance() 150 setBalance(200) Time t1

t2

t3

acc 100

acc 150

acc 200

Figure 3.2: The possible execution of two concurrent calls to the method deposit for the same Account instance. In this case, because the method deposit is synchronized, the second thread waits until the first thread finishes its method’s execution.

other threads, of any other synchronized method for the same object. If we make all the methods of the class Account synchronized, then only one thread at a time can execute code for each account object, therefore preventing inconsistencies. The new behavior of the concurrent executions of the method deposit, after the addition of synchronization, is depicted in Figure 3.2. In this case, when the second call to the method deposit is made, the executing thread waits until the first thread releases the lock, before continuing with the execution of the method. Unfortunately, this simple solution brings its share of problems with it, also. For instance, it prevents two threads from executing concurrently the method getBalance for the same account, even though that method only reads the state of the object. In this case, we are limiting the concurrency unnecessarily. In the end, putting too much synchronization into the application eliminates all the parallelism from the application (to the point of generating deadlocks). On the contrary, too few synchronization leads to inconsistencies. Thus, the choice of adding a synchronized modifier should be considered very carefully by each programmer, putting on the programmer’s shoulders a great responsibility. Furthermore, the problems become worse when more than one object needs to be accessed consistently by multiple threads. One of the problems with lock-based approaches, as this one, is that these approaches do not compose: Given two thread-safe objects, we cannot access both of the objects in a thread-safe way without adding more synchronization that, almost always, needs to override the existing synchronization. I shall discuss now an example which illustrates this problem.

3.1 Implementation of a Concurrent Domain Model

Bank

1

belongs to

35

0..* account

Account

Figure 3.3: The UML class diagram with the relationship between the class Bank and the class Account: A bank can have many accounts, but an account belongs to exactly one bank.

class Bank { Set accounts; ... void transfer(long amount, Account source, Account target) { source.withdraw(amount); target.deposit(amount); } long totalBalance() { long total = 0; for (Account acc : accounts) { total += acc.getBalance(); } return total; } } Listing 3.3: The implementation in Java, without any concerns for thread-safety, of the class Bank with the two methods transfer and totalBalance.

3.1.2

Thread-Safety with More than One Object

Continuing with our stripped-down example of the banking application, consider now that the bank accounts are related to the bank, according to the UML class diagram shown in Figure 3.3. Consider, also, that the class Bank has the following two methods: (1) the method transfer, which transfers some amount between two accounts; and (2) the method totalBalance, which calculates the total balance of all bank accounts belonging to the bank. I show, in Listing 3.3, a possible implementation in Java for the class Bank. Unlike the methods of class Account, the methods shown for the class Bank do not change the state of a Bank’s instance; the Bank’s methods just operate over the accounts of a bank. Therefore, even though this implementation has no synchronization code, interleaved executions of the method transfer do not interfere with one another, provided that we are transferring money between instances of the thread-safe implementation of the class Account shown before. Because the class Account is thread-safe, each deposit or withdrawal leaves the corresponding account in a consistent state. If we consider, however, the concurrent execution of the methods totalBalance

36

The Difficulties of Implementing a Domain Model

trg.getBalance() 100 100 src.getBalance() totalBalance()

200

transfer(100, src, trg) src.widthdraw(100) trg.deposit(100) Time t1

t2

t3

src 100 trg 0

src 0 trg 0

src 0 trg 100

Figure 3.4: Execution of the method transfer during the execution of the method totalBalance. The result returned by totalBalance is incorrect: The result is 200, but the correct total balance for the bank, either before or after the transfer, is 100.

trg.getBalance() 0

src.getBalance() 0

totalBalance()

0

transfer(100, src, trg) src.widthdraw(100) trg.deposit(100) Time t1

t2

t3

src 100 trg 0

src 0 trg 0

src 0 trg 100

Figure 3.5: Execution of the method transfer during the execution of the method totalBalance. In this case, the balance of the src and trg accounts are read in such a way that makes the result returned by totalBalance be 0. As in Figure 3.4, however, the correct total balance for the bank, either before or after the transfer, is 100.

3.1 Implementation of a Concurrent Domain Model

and transfer, it is easy to see that the method totalBalance will return an incorrect result, if it executes in the middle of the method transfer—that is, immediately after the withdrawal from the source account and before the deposit on the target account; at that time, the total balance of the bank, as calculated by the method totalBalance, does not correspond to the correct value. In fact, the method totalBalance can give an incorrect result whenever the execution of this method is interleaved with the execution of at least an execution of the method transfer in any of the following ways:

• The method totalBalance reads the balance of the source account before the withdrawal, and the balance of the target account after the deposit. In this case, the result returned will exceed the correct value by the amount transferred. This situation occurs in the execution shown in Figure 3.4 on the facing page. • The method totalBalance reads the balance of the source account after the withdrawal, and the balance of the target account before the deposit. In this case, the result will be incorrect, also, but now the value returned will be below the correct value exactly by the amount transferred, as shown in Figure 3.5 on the preceding page.

Unfortunately, solving this problem with lock-based mechanisms is not that easy. On one hand, the simple solution of making both methods synchronized impairs greatly the concurrency of the application. For instance, it would neither be possible to execute concurrently two transfers between two pairs of unrelated accounts, nor call the method

totalBalance in two parallel threads. The problem here is that the lock on the bank object is too coarse-grained. Moreover, it is reasonable to expect that bank accounts can be accessed in other places other than these two methods of the class Bank. Having the

Bank’s methods synchronized does not help in getting the correct behavior in that case. On the other hand, trying to use more fine-grained locks, by synchronizing on the accounts as shown in Listing 3.4 on the following page, introduces other problems, such as the possibility of deadlocks—for example, consider a thread transferring money from account A to account B, and a concurrent thread transferring from B to A (see Figure 3.6 on the next page). Solving the problems introduced by fine-grained locking (for instance, by acquiring locks in a specific order to avoid deadlocks) makes the code much more complex and is practically unfeasible for any moderately complex domain-intensive application, in which the objects are densely intertwined. Rather, I argue that the only way to deal effectively with this problem is by using the notion of atomic methods (or blocks), as introduced by the work on Software Transactional Memories. In Chapter 4, I propose a new practical implementation of Software Transactional Memory that is specially suited for implementing rich domain models.

37

38

The Difficulties of Implementing a Domain Model

class Bank { ... void transfer(long amount, Account source, Account target) { synchronized (source) { synchronized (target) { source.withdraw(amount); target.deposit(amount); } } } long totalBalance() { return lockAndSum(accounts.iterator()); } private long lockAndSum(Iterator accounts) { if (accounts.hasNext()) { Account acc = accounts.next(); synchronized (acc) { long remainingSum = lockAndSum(accounts); return remainingSum + acc.getBalance(); } } else { return 0; } } } Listing 3.4: An implementation of the class Bank that uses fine-grained locks on the accounts accessed by each method. This implementation, however, does not handle the problem of possible deadlocks.

synchronized (B) synchronized (A) transfer(50, A, B)

Deadlock!

transfer(50, B, A)

Deadlock!

synchronized (B) synchronized (A) Time t1 Figure 3.6: Deadlock caused by the concurrent execution of two calls to the method transfer, which synchronizes on the source and target accounts. The deadlock occurs at time t1.

3.2 Failure Recovery

39

trg.deposit(100) src.widthdraw(100) transfer(100, src, trg) Time t1

t2

t3

src 100 trg 0

src 0 trg 0

src 0 trg 0

Figure 3.7: Failure during the deposit of a transfer operation. The cross in the method deposit represents that the method throws an exception, causing the termination of both the deposit and transfer methods. The state of the program, however, changed at instant t2 and, thus, remained changed after the exception: Whereas at the beginning of the transfer the src account’s balance is 100, at the end the balance is 0 and the account trg remains the same.

3.2

Failure Recovery

In the above discussion, I argued against the use of locks and in favor of using atomic actions to ensure the consistency of a concurrent application. In fact, the concurrency control aspects of atomic actions are the primary concern in most of the published work in Software Transactional Memory research. Yet, for the purposes of this dissertation, the problem of maintaining the consistency when a failure occurs during the execution of an operation is an equally, if not more, important concern. This problem occurs whether we have a concurrent application or not. Thus, it is independent of concurrency and it is not solvable by the use of locks. Consider, again, the method transfer shown in Listing 3.3 on page 35. Except for the problems of concurrency already discussed, this method appears to be correct. Unfortunately, the problem I am discussing in this section manifests itself even in this simple method if the method deposit may fail in some cases. Given the implementation of the class Account in Listing 3.1 on page 32, the methods

withdraw and deposit never fail.1 On a more realistic setting, however, neither can we withdraw an arbitrary amount from an account, nor can we deposit on all accounts. For instance, assume that we cannot withdraw an amount larger than the account balance, and we cannot deposit on a closed account. In Listing 3.5 on the following page, I show the changes made in the class Account to incorporate these new requirements. 1

Unless some runtime error, such as an out-of-memory error occurs. Nevertheless, I am not concerned with runtime errors due to the malfunction of the program. Rather, I am concerned with the normal, domain-dependent, exceptions that may occur.

40

The Difficulties of Implementing a Domain Model

class Account { ... boolean closed; ... void withdraw(long amount) { if (amount > getBalance()) { throw new InsufficientBalanceException(...); } else { setBalance(getBalance() - amount); } } void deposit(long amount) { if (closed) { throw new ClosedAccountException(...); } else { setBalance(getBalance() + amount); } } } Listing 3.5: Reimplementation of the methods withdraw and deposit, for the class Account, to limit the withdrawal of an amount to the balance of the account, and to refuse deposits in a closed account.

3.2 Failure Recovery

void transfer(long amount, Account source, Account target) { if ((source.getBalance() < amount) || target.isClosed()) { throw new TransferException(...); } else { source.withdraw(amount); target.deposit(amount); } } Listing 3.6: Reimplementation of the method transfer to verify, before making any change, that the whole operation will succeed.

With this new definition for the class Account, the method transfer is no longer correct. Consider the case when we are transferring some amount to a closed account. The call to the method deposit on the target account will fail, by throwing the exception

ClosedAccountException. But, because the failure occurs after the source account was withdrawn, it causes the loss of the amount to transfer (see Figure 3.7 on page 39), which is not an acceptable behavior for such an application. Note that this problem is not related to concurrency. The case that I presented here is a purely sequential program. To solve this problem, we have essentially two options:

• We verify before executing any of the operations that they will not fail, as in Listing 3.6. • We save the previous state of the source account and restore it in case of failure, as in Listing 3.7 on the next page.2

I argue that neither of these approaches is satisfactory. In the first case, the need to verify in a method of the class Bank the preconditions for executing the withdrawal and the deposit method calls, breaks the modularity and encapsulation of the class Account: The responsibility of knowing when a withdrawal or a deposit can be made should belong exclusively to the class Account. Ensuring this property is essential, if we want to take advantage of common object-oriented programming facilities such as polymorphism. For instance, consider that we want to have a different kind of account—e.g., a deposit account—in which deposits are further limited. We want that the method transfer remains the same, independently of the new types of accounts in our application. The alternative of adding to the class Account the predicate

canDeposit which receives the amount to deposit and returns whether that amount can be deposited or not, is not viable, either: In the end, we have one such predicate for each operation that may fail. Not only that, all invocations of the operations have to 2

In this particular case, we could just re-deposit the amount in the source account, undoing the original operation. That approach, however, is not generally applicable, because the performed operation first may be irreversible.

41

42

The Difficulties of Implementing a Domain Model

void transfer(long amount, Account source, Account target) { long savedBalance = source.getBalance(); source.withdraw(amount); try { target.deposit(amount); } catch (Throwable t) { // the deposit failed, undo the withdrawal source.setBalance(savedBalance); throw t; } } Listing 3.7: Reimplementation of the method transfer to undo the withdrawal when the deposit operation fails.

be guarded by calls to these predicates. Finally, this approach is not applicable easily in all cases. Sometimes, it is impossible to know in advance—or to factor them out—the conditions that prevent the correct execution of each operation. In the second case, we have similar problems. On one hand, the state of the object may not be publicly available or it may not be easy to save. On the other hand, the complexity of the code needed to save and restore the state increases substantially for more complex methods, when several operations that may fail are executed within more sophisticated control structures. I believe that the difference between the original version of the method transfer and the version corresponding to this approach illustrates very well this latter point.

3.3

Implementation of the Banking Domain Model’s Structure

In this and in the following section, I shall now consider the full domain model introduced in Section 2.2.4. Class diagrams, as those shown in Figure 2.3 on page 22 and in Figure 2.4 on page 24, represent only (part of) the class structure of a domain model. The dynamic aspects of a domain model—its behavior—are not captured by such diagrams. In fact, as I discuss briefly at the end of Section 2.2.4, not even all the structural aspects of a domain model are captured by class diagrams. To model the dynamics of an object model, modeling languages such as UML provide complementary notations, such as sequence diagrams and state charts. This separation, at the modeling level, between the structural and the behavioral aspects of a domain model, becomes diluted when we are at the implementation level, because object-oriented programming languages conflate the two aspects into the same artifact—the class. Yet, for the purposes of this presentation, it is useful to discuss the

3.3 Implementation of the Banking Domain Model’s Structure

implementation of each of these aspects in separate. So, in this section I shall discuss only the implementation of the domain model’s structure. Then, in the next section, I address the behavioral aspects of the domain model’s implementation. The class diagram shown in Figure 2.4 on page 24 is a high-level class diagram. As such, it omits many details which are essential for an implementation of the domain model—for example, no attributes are shown for the classes in the diagram. The omission of such details during the initial design phase is an important abstraction mechanism that allows us to experiment with different designs without spending much time with the details of each one. But, once a design is chosen, the class diagram must be filled-in with the details necessary for an implementation of that design. In Figure 3.8 on the next page, I show a class diagram in which some of the classes previously shown in Figure 2.4 have more information—namely, information about their attributes and their methods. The methods in each class implement that class instance’s behavior; the attributes, on the other hand, are used to hold the state of each class instance. To implement the structure of a domain model, programmers must provide an implementation for each of the following two kinds of domain model elements: (1) the classes, with their attributes; and (2) the associations between classes.

3.3.1

Implementation of Classes

The implementation of the basic structure for each of the classes in a UML class diagram is straightforward in most modern object-oriented programming languages. Each of the classes in a class diagram maps naturally into a programming language construct—for example, a Java class or interface. Furthermore, UML’s generalization relationships3 between classes are implemented by the usual type extension constructs that are available in object-oriented programming languages. Finally, for the UML’s class attributes we have a direct correspondence with the class member slots (or fields) found in object-oriented programming languages. For instance, using Java, each class attribute is implemented, typically, as a private class field, with two accessor methods: one to obtain the field value, and the other to set the field to a new value. The visibility of each of these class members varies, depending on whether they are meant to be used outside the class or not. In Listing 3.8 on page 45, I show a partial implementation, in Java, of the classes

Account and ClientAccount. Each class implements the attribute that is depicted in Figure 3.8 on the next page.

3

Also called inheritance or “is a” relationships.

43

44

The Difficulties of Implementing a Domain Model

Account balance:Money deposit(Money) withdraw(Money)

ClientAccount

BankAssets

Client

closed:boolean

1..*

close() isClosed():boolean

account

1 owner

name:String totalBalance():Money

SavingsAccount CheckingAccount

1 checking

0..* savings

interestRate:Percentage depositPeriod:TimePeriod termDate:Date

Figure 3.8: Implementation-level class diagram for part of the banking application’s domain model. This class diagram shows some of the classes from the design-level class diagram shown in Figure 2.4, but with some implementation details added. Both attributes and methods are shown for some of the classes.

3.3 Implementation of the Banking Domain Model’s Structure

public class Account { private Money balance; public Money getBalance() { return this.balance; } protected void setBalance(Money balance) { this.balance = balance; } } public class ClientAccount extends Account { private boolean closed; public boolean isClosed() { return this.closed; } public void close() { this.closed = true; } } Listing 3.8: Partial implementation of the classes Account and ClientAccount. In this case, each UML class is implemented as a Java class. Also, the generalization relationship between the entities Account and Client Account is implemented by making the class ClientAccount a subclass of Account. Finally, class attributes are implemented as Java class fields with accessor methods.

45

46

The Difficulties of Implementing a Domain Model

3.3.2

Implementation of Associations

Unlike classes, associations do not have a direct mapping into the implementation-level programming language. Thus, their implementation is not as simple as it is for classes. In fact, in some cases, the implementation of associations is far from trivial and becomes a great burden for the programmer. Class associations are common at the modeling level, but they have not found their way into the realms of modern object-oriented programming languages, even though their absence at the implementation level has been noted two decades ago [Rumbaugh, 1987]. The lack of support for associations at the programming language level forces programmers to use other constructs to implement them. In some cases that is easily done; in many others, however, it is not. One of the factors that influences the implementation of an association is its multiplicity. One-to-one associations are simpler to implement than many-to-many associations, whereas one-to-many associations stay somewhere in between. Another factor that affects the implementation of an association is whether the association is unidirectional or bidirectional. Bidirectional associations are harder to implement than unidirectional associations. As a matter of fact, bidirectionality poses the most difficulties on the implementation of associations. So much so that programmers are advised to avoid bidirectional associations—or to limit their use—even though such associations are obviously necessary and useful for expressing the structure of a domain model. For example, consider what Eric Evans wrote on his book on domain-driven design:

In real life, there are lots of many-to-many associations, and a great number are naturally bidirectional. The same tends to be true of early forms of a model as we brainstorm and explore the domain. But these general associations complicate implementation and maintenance. Furthermore, they communicate very little about the nature of the relationship. There are at least three ways of making associations more tractable. 1. Imposing a traversal direction 2. Adding a qualifier, effectively reducing multiplicity 3. Eliminating nonessential associations It is important to constrain relationships as much as possible. A bidirectional association means that both objects can be understood only together. When application requirements do not call for traversal in both directions, adding a traversal direction reduces interdependence and simplifies the design.

[Evans, 2003, page 83]

I agree with the argument that bidirectional associations increase the coupling between classes, and that they may, therefore, reduce our ability to reason about the domain in a modular way. I do not agree, however, with the argument that we should remove

3.3 Implementation of the Banking Domain Model’s Structure

public class Bank { private BankAssets assets; public BankAssets getAssets() { return this.assets; } public void setAssets(BankAssets assets) { this.assets = assets; } } Listing 3.9: Implementation of a unidirectional one-to-one association between Bank and BankAssets. The field assets in the class Bank is used to keep a reference to the only instance of BankAssets that may relate with a bank. The class BankAssets is not shown here because no change is needed in it. bidirectional associations just because they are difficult to implement. Instead, I propose that we make them simple to implement. That is the main goal of the language that I propose in Chapter 5. But first, it is worthwhile to understand why are associations so complicated to implement. In fact, a class association may be implemented in several different ways (see, for example, Guéhéneuc and Albin-Amiot [2004]; Harrison et al. [2000]; Génova et al. [2003]). Here, I do not discuss all the possible implementations. Rather, I present some examples that follow the idioms that are commonly found in the literature of object-oriented programming. Unidirectional one-to-one associations The simplest case is when we have an unidirectional one-to-one association. As an example, consider that we impose a traversal direction on the association between the class

Bank and the class BankAssets—that the association only allows the traversal from the Bank to the BankAssets. The implementation of this association is similar to the implementation of any other class attribute for the class Bank: We add a field to the class

Bank to keep a reference to the only instance of BankAssets that is related with each Bank’s instance, and we add the necessary methods to access that field. In Listing 3.9, I show such an implementation. Note the similarity with the code from Listing 3.8 on page 45. Bidirectional one-to-one associations Unfortunately, the implementation of a bidirectional one-to-one association is no longer that simple: At least, an implementation that gives us some guarantee of domain consistency.

47

48

The Difficulties of Implementing a Domain Model

Consider again the case of the Bank–BankAssets one-to-one association, but now as a bidirectional association. A simple implementation consists in keeping the class Bank as shown in Listing 3.9 on the preceding page, and in changing the class BankAssets so that this class has a reference to an instance of a Bank. The problem with this implementation, however, is that there is nothing in it to prevent domain inconsistencies. For example, this implementation allows that a client of these classes creates an instance of the class Bank that has a reference to an instance of the class BankAssets, which in turn has a reference to a different instance of the class Bank. Yet, in a bidirectional one-to-one association, if an object A refers to another object B, then the object B must refer to the object A also; otherwise, the relationship is not consistent. Of course, it is possible to have such a simple implementation and still do not have inconsistencies, provided that the client code of these classes create and change both classes consistently. But this puts the responsibility on the wrong place. A better implementation of this association should ensure that, when we execute a method call such as bank.setAssets(assets), both the bank and the assets are updated consistently. Moreover, the result of that method call should be the same as executing assets.setBank(bank). In Listing 3.10 on the next page, I show an implementation of the Bank–BankAssets bidirectional association that satisfies these requirements. The code in Listing 3.10 illustrates well enough why programmers avoid bidirectional associations, if possible: Bidirectional associations are much more difficult to implement. Although the implementation shown follows the recommendations of the “Change Unidirectional Association to Bidirectional” refactoring pattern [Fowler, Beck, Brant, Opdyke, and Roberts, 1999], the mere application of the pattern is highly error-prone.

One-to-many and many-to-many associations Going from one-to-one to either one-to-many or many-to-many associations adds complexity into the implementation, because, in the latter case, objects no longer refer to a single object. Rather, they may refer to an indeterminate number of other objects. Therefore, the class fields that implement the associations’ ends that may refer to many objects must be instances of a collection. Fortunately, the Java standard platform provides us with a reasonable API for working with a variety of collections. So, by using some classes from the “Java Collections Framework,” we may implement a unidirectional one-to-many association as shown in Listing 3.11 on page 50. Note that the method getAccounts must return a different set of accounts—a set of accounts that is unmodifiable. Otherwise, clients of this class would be able to modify directly the relationships of a class’s instance. Changing the association from unidirectional to bidirectional, causes a change in the

3.3 Implementation of the Banking Domain Model’s Structure

public class Bank { private BankAssets assets; public BankAssets getAssets() { return this.assets; } public void setAssets(BankAssets assets) { if (this.assets != assets) { if (this.assets != null) { this.assets.friendSetBank(null); } if ((assets != null) && (assets.getBank() != null)) { assets.getBank().setAssets(null); } if (assets != null) { assets.friendSetBank(this); } this.assets = assets; } } } public class BankAssets { private Bank bank; public Bank getBank() { return this.bank; } public void setBank(Bank bank) { if (bank != null) { bank.setAssets(this); } else if (this.bank != null) { this.bank.setAssets(null); } } void friendSetBank(Bank bank) { this.bank = bank; } } Listing 3.10: Implementation of a bidirectional one-to-one association between Bank and BankAssets. The class BankAssets delegates into the class Bank the responsibility of performing the changes needed in both classes. The code in the class Bank, on the other hand, needs to ensure that old links are broken before creating the new one.

49

50

The Difficulties of Implementing a Domain Model

public class Client { private Set accounts; public Set getAccounts() { return new Collections.unmodifiableSet(this.accounts); } public void addAccount(ClientAccount account) { this.accounts.add(account); } public void removeAccount(ClientAccount account) { this.accounts.remove(account); } } Listing 3.11: Implementation of the unidirectional one-to-many association between the classes Client and ClientAccount. The traversal direction is from the Client to the ClientAccount. An instance of the class java.util.Set is used to keep all the client’s accounts. The set of accounts is protected against modifications before returning it to a client of the class.

implementation which is similar to the change occurred in the one-to-one case. The code needed, however, varies with each combination of multiplicities. Furthermore, the choice of which collection to use to implement an association depends on additional properties of the association. If the association is ordered, for instance, then the most appropriate collection might be an ordered set or a list. If, on the other hand, the association is qualified, then maybe a map would be preferable. So, even though there are well-known patterns on how to implement the various types of associations in an object-oriented programming language such as Java, the problem is that employing those patterns is, nevertheless, a burdensome and error-prone task.

3.4

Implementation of the Banking Domain Model’s Behavior

Once we have the code that implements the basic class structure for the banking domain model, we may turn now our attention to the implementation of the remaining functionality. To implement the required functionality, we need to add behavior to the classes that we have in the domain, either by writing new methods, or by adding new code to the methods that already exist. In this section, I shall discuss some of these methods.

3.4 Implementation of the Banking Domain Model’s Behavior

public abstract class Account { ... public void deposit(Money amount) { amount = normalizeCurrency(amount); setBalance(this.balance.add(amount)); } public void withdraw(Money amount) { amount = normalizeCurrency(amount); setBalance(balance.subtract(amount)); } protected Money normalizeCurrency(Money amount) { Currency accountCurrency = getBalance().getCurrency(); if (accountCurrency.equals(amount.getCurrency())) { return amount; } else { return getBank().convertTo(amount, accountCurrency, this); } } public abstract Bank getBank(); } Listing 3.12: Generic implementation of the methods deposit and withdraw for the class Account. Each of the methods calls an auxiliary method that returns the amount in the appropriate currency, converting it, if needed. The abstract method getBank, called by the method normalizeCurrency, must be implemented by each of the concrete subclasses of Account that have some association with the class Bank. An instance of a Bank is needed to perform the currency exchange.

3.4.1

Implementation of the Basic Deposit and Withdraw Operations

Because many of the functionality requirements are related to the deposit and withdraw operations, I start with the implementation of these operations. According to the class diagram shown in Figure 3.8 on page 44, these two methods are implemented in the class

Account. In fact, even though they need to be specialized for some of the subclasses of Account, their basic definition is the same for all the accounts: the method deposit adds the given amount to the account’s balance; the method withdraw subtracts the given amount from the account’s balance. The amount to add or to subtract, however, must be in the same currency as the account’s balance. Thus, in Listing 3.12, both methods use a helper method to perform the conversion of the amount to the appropriate currency, if needed. The Bank’s method convertTo is responsible for making the currency exchange of the received amount to the new currency, charging the account received as the third argument with the exchange fees that are applicable. In Listing 3.13 on the next page, I

51

52

The Difficulties of Implementing a Domain Model

public class Bank { public Money convertTo(Money amt, Currency curr, Account acc) { ExchRate rate = getExchangeRate(amt.getCurrency(), curr); Money newAmount = rate.convertTo(amt); if (acc.isClientAccount()) { charge(acc, rate.getFee()); } return newAmount; } } Listing 3.13: Implementation of the method convertTo. The bank converts the amount using the appropriate exchange rate and then charges the account with a fee if the account in question is the account of a client.

show the sketch of a possible implementation for this method. With this simple implementation, however, we are ignoring one problem: What happens if the call to the method setBalance fails?

Because the call to the method

normalizeCurrency is made first, it may have already charged the account for the exchange fees, even though the operation will not conclude successfully. This is yet another example of the difficulty in dealing with failures in domain operations. Unfortunately, fixing the code to handle this case makes the code much more complex; so, I will leave the code as shown and assume that it is easy to fix it with the solution proposed in this dissertation.

3.4.2

Implementation of the Client’s Total Balance Limit

One of the functionalities described in Section 2.2.2 is that the total balance of each client must be greater than or equal to zero. Therefore, to implement this functionality we must prohibit the operations that, if executed, would cause the total balance of a client to become negative. Because the only operation that decreases the total balance of a client is the withdrawal of money from one of its accounts, we may try to implement this functionality by overriding the method withdraw in the class ClientAccount, as shown in Listing 3.14 on the facing page. The strategy for this implementation is to abort the withdrawal before it occurs, by checking first if the client’s total balance is less than the amount to withdraw. Unfortunately, this solution has two errors. First, because the amount to withdraw may be in a currency different from the currency of the result of totalBalance, the two values may be incomparable.4 Second, because the call to the inherited method withdraw may, in fact, withdraw more than the amount requested, if, for example, some fee is charged by 4

Although the class Money is not described here, I assume that it is not possible to compare directly values with different currencies.

3.4 Implementation of the Banking Domain Model’s Behavior

public class ClientAccount extends Account { public void withdraw(Money amount) { if (getOwner().totalBalance().lessThan(amount)) { throw new ClientNegativeBalanceException(); } else { super.withdraw(amount); } } } Listing 3.14: Checking the client’s total balance before the withdrawal is performed. If the client’s total balance is less than the amount to withdraw, then an exception is thrown. Otherwise, the withdraw proceeds.

public class ClientAccount extends Account { public void withdraw(Money amount) { super.withdraw(amount); if (getOwner().totalBalance().isNegative()) { // assume that the already made side-effects are undone // when the exception is thrown throw new ClientNegativeBalanceException(); } } } Listing 3.15: Checking the client’s total balance after the withdrawal is performed. In this solution, first the withdrawal is performed and only then we check to see whether the client’s total balance became negative. If that is the case, then an exception is thrown and we rely on some failure recovery mechanism to undo the already made changes.

the bank; if that happens, the comparison made in this method is useless to prevent that the client’s total balance after the withdrawal becomes negative. Assuming again that we can undo the changes already performed by an operation that fails, an alternative implementation that solves both problems is shown in Listing 3.15. This implementation relies on the failure recovery capabilities of an operation to perform first the withdrawal and only then check if the resulting state is consistent with the domain requirements. If not, then an exception is thrown and the atomic execution of the method withdraw is aborted, undoing the effect of the withdrawal. Yet, even though this second solution corrects the two errors identified in the first solution, it is not free of problems, either. In fact, there are at least two problems with it. The first problem with this solution is that it may cause a failure, even if the withdrawal is part of a complex banking transaction that, in the end, would leave the total balance of the client positive. The failure happens if the withdrawal causes the total balance to become negative, even if only temporarily. In fact, a simple example illustrates this

53

54

The Difficulties of Implementing a Domain Model

problem. Consider a client with two checking accounts: the first with a negative balance of 100 EUR, and the second with a positive balance of 200 EUR. The total balance for this client is 100 EUR. What happens if the client decides to transfer 150 EUR from the second account to the first? If the transfer operation withdraws the 150 EUR from the source account first, the total balance of the client becomes negative (minus 50 EUR), which causes the withdrawal to fail. Of course that, in this case, it may be possible to reorder the operations so that the deposit occurs before the withdrawal, but this reordering is not always possible. For instance, consider the case of a deposit of an amount that must be converted to another currency. As part of the deposit operation, before the balance of the account is increased with the amount, the amount is converted to the correct currency and the account is charged with a fee. If the client total balance is 0, then the deposit fails, because the fee cannot be withdrawn. So, if we cannot verify that the client’s total balance is not negative at the end of the method withdraw, where can we do it? I argue that we can make this verification only at the end of the outermost domain operation. Unfortunately, we lack the mechanisms to do it. In Chapter 6, however, I propose a solution to this problem. The second problem with the implementation shown in Listing 3.15 is that it is in the wrong place. The responsibility of checking that the client’s total balance is not negative should not belong to the class ClientAccount. Rather, it should be a responsibility of the class Client. Imagine that we extend the domain of our example to support different types of clients. Most probably, each type of client has a different condition regarding its total balance. For instance, the rule we now have may not be applicable to business clients, or to clients with a mortgage. Having the verification of the rule in the method

withdraw of the class ClientAccount does not allow us to differentiate between these cases. If, on the other hand, the responsibility of checking the client’s total balance is on the class Client, then we can specialize it for different types of clients. One way to implement this verification in the proper place is to use the Observer design pattern (see [Gamma, Helm, Johnson, and Vlissides, 1995]), and make the client an observer of all its accounts. But, even if the use of the observer pattern helps in localizing the code in the proper place, it forces a more complex solution and does not solve the first problem either. Again, I argue that the solution that I propose in Chapter 6 is a much cleaner way of solving this problem.

3.5

Summary

This chapter gives several examples of how difficult can be the implementation of a domain model. The four problems identified in Section 2.3 are illustrated in this chapter with concrete examples from the banking application’s domain model.

3.5 Summary

Even though the domain used as example is very simple, I show that its implementation is far from trivial; specially, if we consider that the domain entities are accessed concurrently.

55

56

The Difficulties of Implementing a Domain Model

Chapter 4

Versioned Software Transactional Memory The hardware industry’s commitment to the use of multi-core processors as the only practical way to improve computing power for the new generation of computers, brought concurrent programming, finally, into the realms of mainstream programming. Yet, almost all modern programming languages lack adequate abstractions for concurrent programming. As we saw in Section 3.1, mechanisms such as Java’s synchronized keyword are useful to develop thread-safe single objects, but are of little help when various objects are involved in more complex operations. Ensuring, with lock-based mechanisms, that all the objects accessed during a complex operation are in a consistent state is difficult and highly error-prone. Thus, given that in a domain model objects are typically intertwined in a graph of complex relationships, implementing a concurrent domain model with locks becomes humanly infeasible. Fortunately, there is now a more promising approach to deal with the difficulty of implementing a shared-memory concurrent program: the use of Transactional Memory. Since the seminal paper on Transactional Memory by Herlihy and Moss [1993] and the later proposal of Software Transactional Memory (STM) by Shavit and Touitou [1995], this area of research has developed at an increasingly fast rate during the last few years. Much of the recent work on STMs addresses the problem of concurrent programming in languages such as Java by extending the language with the notion of atomic actions, or transactions [Herlihy, Luchangco, Moir, and Scherer, III., 2003; Harris and Fraser, 2003; Harris, Marlowe, Peyton-Jones, and Herlihy, 2005], which allow a consistent access to shared-memory without the use of locks. Even though many problems remain to be solved in this area, I argue that it is now

58

Versioned Software Transactional Memory

practical to use an STM to develop domain-intensive applications. In fact, I argue that by doing so we will be able to simplify significantly the development of such applications. In this chapter, I propose a novel STM that, unlike previous STMs, keeps multiple versions of each transactional location. This new multi-version STM was conceived with the development of domain-intensive applications in mind. I present the versioned STM model independently of a specific programming language, and then describe an implementation in the Java programming language. I argue not only that this new versioned model is suitable for the development of domain-intensive applications, but also that it is adequate for an effective and practical implementation that allows its immediate use in real-world domain-intensive applications.

4.1

Introduction to Software Transactional Memory

The key idea underlying the work on Software Transactional Memory is that programmers specify which operations should execute atomically, rather than protect accesses to the data with locks. The intended semantics for such atomic actions is that they execute atomically, independently of which data is accessed by the operation—that is, that their execution occurs as if nothing else is executing at the same time. A simple solution to accomplish the intended semantics for atomic actions is to prohibit parallelism in the application when the atomic action is executing. This is similar to using a global exclusive lock that all the atomic actions must acquire before executing. Yet, the goal of having STMs is to allow concurrency, rather than to eliminate it. Thus, the challenge of an STM is to allow the maximum parallelism in a program while ensuring that atomic actions execute with their intended semantics. Even though most of the literature on STMs uses the terms atomic action and transaction interchangeably, in this dissertation I use them with a slightly different meaning: I use the term atomic action to refer to the definition of a series of operations that should be executed atomically, whereas the term transaction refers to the execution of an atomic action. There are many places, however, where this distinction is not relevant, in which case I will use either of the terms.

4.1.1

Atomic Actions and the Property of Atomicity

The property of atomicity has two distinct aspects, which are of special interest to us. First, that all the changes performed by the execution of an atomic operation are seen at the same time by the rest of the system. Second, that either all the changes made by the operation occur or none occurs; there is no in-between case. The first aspect of atomicity is what ensures the consistency of the data in the case of concurrent executions. The

4.1 Introduction to Software Transactional Memory

second aspect of atomicity is what ensures the consistency of the data in the case of failures: Atomic actions give us the ability to recover from failures. Having atomic actions in the programming language, the correct implementation of the class Bank from Listing 3.3 on page 35 becomes much easier: We just have to say that the two methods (transfer and totalBalance) are atomic actions; then, it is the programming language responsibility to ensure that the methods execute atomically. With atomic actions, no locks (or synchronized modifiers) are necessary, either in the class Bank, or in the class Account. Obviously, this way of programming a concurrent application is much easier than using lock-based mechanisms. It is easier to specify which actions should execute atomically, than to make sure that all the relevant locks are acquired, in the correct order, to ensure the exclusive access to a set of data. Furthermore, atomic actions give us the added bonus of failure recovery.

4.1.2

Read Sets, Write Sets, Commits, and Aborts

Even though the various STMs proposed so far vary considerably in how they ensure the atomicity property, there are a few concepts that are common to most, if not all, of the STMs. Two such concepts, which are useful to explain how STMs work, are the read set and the write set. From the point of view of an STM, a transaction is a series of operations that either read values from shared locations or write values to shared locations.1 Thus, the read set of a transaction is the set of locations read during that transaction. Likewise, the write set of a transaction is the set of locations written during that transaction. Given that transactions may occur in parallel, it may happen that two or more of them access the same shared location. Whereas it may be safe that several transactions read the same shared location, having one transaction writing to a location that is read by another would probably break the atomicity property. Thus, it is the job of an STM to prevent this from happening, usually by keeping track of the read sets and the write sets of each transaction, and checking for conflicting accesses. Another important concept regarding STMs is the commit of a transaction. As we saw, a transaction may execute an arbitrarily large number of operations. Yet, given that it should execute atomically, none of the values written during the transaction should become accessible to other transactions until it finishes. The commit of a transaction is the final operation in a transaction that indicates that all the values written during the transaction should become accessible to others. 1

The size of a shared location may vary from system to system, from a word of memory to an entire object, for instance.

59

60

Versioned Software Transactional Memory

It may happen, however, that a transaction must fail, in which case none of the values written by the transaction should become accessible. In that case, the final operation of the transaction is an abort, rather than a commit. An abort may be either executed deliberately during the transaction as part of an atomic action, or issued by the STM system when it determines that a transaction should not commit successfully.

4.1.3

Transaction Linearizability

The expected correctness condition that a set of successfully committed transactions must satisfy is that they are linearizable [Herlihy and Wing, 1990]. Informally, a set of transactions is linearizable if they satisfy the following conditions: (1) each transaction appears to occur instantaneously at some point in time between its start and its end; and (2) the execution of non-concurrent transactions preserve their order of invocation—that is, a transaction that starts only after another ends should never appear to execute before the latter. A system that ensures that transactions are linearizable allows us to reason more easily about our transactional programs, because we know that each transaction sees a consistent view of the program’s data. Using locks, it is possible, by carefully programming the acquisition of locks at the beginning of each transaction, to ensure that each transaction will be able to view a consistent view of the data, because no other transaction will be allowed to change any of the data that the transaction has locked. This method of concurrency control is often called pessimistic. Yet, with STMs the programmer is free of the burden of acquiring locks. So, in contrast with the previous solution, STMs typically use an optimistic approach to concurrency control. In the optimistic method, the transaction proceeds without acquiring any locks and then verifies at some point that the transaction is valid. A consequence of this method, however, is that a transaction may need to abort because of a conflict with another transaction. Typically, a conflict between two concurrent transactions occurs when one of the transactions needs to read some data that is modified by the other. In such cases, lockbased approaches would probably force one of the transactions to wait for the other, leading eventually to a deadlock. With most STMs, no wait exists. Instead, the conflict is detected, and at least one of the transactions restarts. The basic assumption for most of the STM approaches is that conflicts are rare. Thus, when they occur, it is acceptable to retry the execution of the transaction, given that the probability of another conflict is low. If, on the other hand, the probability of a conflict is high, this strategy may lead to the starvation of some transactions.

4.2 The Rationale for the Versioned STM

To conclude this brief introduction to STMs, consider again the problematic examples shown in Section 3.1: Figure 3.1 on page 33, Figure 3.4 on page 36, Figure 3.5 on page 36, and Figure 3.6 on page 38. In which way are these examples changed if we use STMs? Assume that each of the methods’ executions in these examples corresponds to a transaction in some STM-based system, and that no other synchronization exists. Then, most probably, in all of the examples, the two transactions conflict with each other. Note that in all the examples there is at least an object that is read by one transaction and modified by the other. The advantage of using STMs in these examples, of course, is that the conflict is detected and one the transactions is restarted, whereas in the previous case incorrect results (or deadlocks) occurred.

4.2

The Rationale for the Versioned STM

The goal that I established for the STM proposed in this chapter was that it should be suitable for implementing domain-intensive applications. Therefore, I started the design of the STM with a set of assumptions regarding the characteristics of this type of applications: • That the number of updating transactions (transactions that change any data) is low when compared with the total number of transactions; probably, less than 10%. • That most of the transactions are medium-sized; that is, that the average size of their read sets and write sets have hundreds to thousands of objects. • That occasionally there are long-running transactions that need to access thousands to millions of objects. • That an updating transaction may read many objects, but typically changes just a few. When I started, most of the research on STMs did not deal well with this kind of workload. In fact, even now, most of the examples and the benchmarks are for very short transactions: transactions that access from a couple to a few tens of objects before they commit. The problem with larger transactions is twofold. First, the need to keep track of large read sets and write sets may cause serious performance problems. Second, the probability of a conflict increases both with the number of objects accessed by a transaction, and with the transaction’s duration; this problem becomes more dramatic when we have longrunning transactions that access a significant portion of the object space. Given this set of assumptions, I established the following set of requirements for the STM proposed in this dissertation:

61

62

Versioned Software Transactional Memory

• The read-only transactions should be made as fast as possible. • Accessing an object for reading should have a low overhead. • The performance of an updating transaction is not very important. • Long-running transactions should be able to execute without halting the entire system.

In the following section I describe my proposal for an STM that addresses these requirements.

4.3

The Versioned STM Model

In the previous introduction to STMs, I have described in very general terms how the majority of the existing STMs work. Yet, there are many differences among the various proposals—for example, different STMs vary in the number of conflicts they generate, use different conflict detection algorithms, or differ in where and how they store the read and write sets. In this chapter I propose a new model for STMs that is significantly better than the existing approaches in handling particular workloads: Workloads which I argue are typical of domain-intensive applications. The distinctive element of my approach to STMs is the use of versioned boxes to hold the mutable state of a concurrent program. Versioned boxes can be seen as a replacement for memory locations [Harris and Fraser, 2003] or transactional variables [Harris et al., 2005].

4.3.1

Model Elements and Terminology

Transactions, as usual, serve to delimit blocks of operations that should execute atomically. A transaction is associated with exactly one thread—the thread that started it—and lasts until that thread either aborts or commits the transaction. A transaction Tc that starts in the context of another transaction Tp is a nested transaction. Moreover, I say that Tp is the parent of Tc and that Tc is a child of Tp. Transactions that have no parent are called top-level transactions. When a transaction starts, it becomes the thread’s current transaction until it finishes or another (child) transaction starts. When a transaction finishes, its parent, if any, becomes the thread’s current transaction.

4.3 The Versioned STM Model

63

B 2 1 0

87

23

5

Figure 4.1: Graphical representation of a versioned box. The history’s values are presented in decreasing order of their version number, which is the number at the lower right corner of each rectangle. At the top of the history’s values I put the name used to refer the box in the text. Because child transactions are executed by the same thread of their parents, when a child transaction is executing, the parent transaction is not. Likewise, it is not possible to have sibling transactions executing simultaneously. In fact, a sibling transaction of a transaction Tc can start only after Tc finishes. This model of transaction nesting corresponds to what Moss and Hosking [2005] call linear nesting. Each transaction has a version number, which is assigned to the transaction when the transaction starts. This number comes from a global counter that is incremented only when a top-level transaction that changed some data successfully commits. So, all the top-level transactions that start between two commits of such transactions have the same transaction number. When a nested transaction starts, its number is set to the number of its parent. During their execution, transactions may access versioned boxes. Unlike conventional locations, which keep only a single value, a versioned box is a container that keeps a tagged sequence of values—the history of the versioned box. Each of the history’s values corresponds to a change made to the box by a successfully committed top-level transaction and is tagged with the number of that transaction; this tag number is the value’s version. For instance, if a box B is created in transaction number 5, and then changed twice, in transactions numbered 23 and 87, the history of B will have three values, each tagged with one of the previous transactions’ numbers. I represent such a box graphically as shown in Figure 4.1. There are two operations that a transaction can execute on a versioned box B: • The read operation, BoxRead(B), which returns the current value of box B in the transaction. • The write operation, BoxWrite(B, v), which sets the value of box B to v in the current transaction. The behavior of these operations and how they interact with transactions will be explained below. For now, it is important to note that each of these operations must execute in the context of some transaction. If the executing thread has no current transaction, then

64

Versioned Software Transactional Memory

a new transaction is created immediately before and is committed immediately after the operation. I say that a box B is read (respectively, written) in the context of a transaction T, when

T is the current transaction of the thread executing a read (respectively, write) operation on B. Finally, besides a reference to its parent, each transaction keeps a record of the following information: (1) a transaction number, (2) a set of versioned boxes that were read in the context of the transaction, and (3) a map that maps boxes that were written in the context of the transaction to the values the boxes were set to. To simplify the presentation below, given a transaction T, I use the notation T-number, T-readSet, and T-writeMap, to refer to each of these values, respectively.

4.3.2

Operations on Versioned Boxes

A write operation, BoxWrite(B, v), that executes in the context of a transaction T does not change the history of B immediately. Instead, it adds a mapping from B to v into the map T-writeMap. This map corresponds to the private storage of transaction

T, and is used to keep track of the writes performed during T. If the same box is written more than once during the same transaction T, later values replace earlier values in

T-writeMap—that is, each transaction keeps only the last value written to a box. A read operation, BoxRead(B), that executes in the context of a transaction T must return the current value of B. Obviously, the expected behavior for this operation is that it returns the last value wrote to B in the context of T, if any. To accomplish this behavior, the read operation searches for an existing mapping for B in T-writeMap. If such a mapping exists, the corresponding value is returned. Otherwise, the search continues recursively on T’s parent, until either one mapping is found or no parent exists. If no mapping is found, then the value to return is obtained from the history of B and B is added to T-readSet. The value obtained from the history of B by the read operation executed in T is the value tagged with the highest number which is less than or equal to the number of

T—for example, a transaction numbered 30 will read the value 1 from the box B shown in Figure 4.1 on the preceding page, provided that no value was written to B in the transaction. As we shall see below when I describe the commit of a transaction, the value that T reads from B was the last value wrote to B when T started. I say that a transaction T is a read-only transaction if T-writeMap is empty, regardless of the state of T-readSet—that is, a read-only transaction T is a transaction during which no box was written, whether some box was read by T or not.

4.3 The Versioned STM Model

Similarly, I say that T is a write-only transaction if T-writeMap is non-empty and

T-readSet is empty. Note that the boxes read during a transaction are added to the transaction’s read-set only if they were not written previously in the transaction or in any of its ancestors; this is one of the distinguishing elements of this model. Thus, a write-only transaction may actually have read some boxes during its execution, but, if it did, then it follows that all the boxes read were previously written by the transaction (or by one of its ancestors). In the remaining case, when both T-readSet and T-writeMap are non-empty, for some transaction T, I say that T is a read-write transaction. Finally, I say that a transaction T is a write transaction when it is either a write-only or a read-write transaction.

4.3.3

The Transactions’ Life-Cycle

Transactions start with both their read-set and their write-map empty. Also, when they start, they get a number that will remain the same during the entire transaction, until the transaction commits. Only when a transaction commits, and only in certain cases, can this transaction number be changed to a number greater than the originally assigned to the transaction. Intuitively, transaction numbers serve to position the successfully committed top-level transactions within a serial order of execution. As we shall see, however, the numbers assigned to each transaction do not unambiguously specify a total order among the transactions. Rather, these numbers specify only a partial order among the transactions, because several transactions may get the same number. I shall discuss how this partial order is related to the linearizability of top-level transactions in Section 4.3.4. Before that, however, I shall describe the rest of the transactions’ life-cycle. As transactions execute in parallel, accessing shared resources, they may conflict with one another. Conflicts are detected by examining what is read and written by each transaction. Other STMs check for conflicts whenever a shared location is accessed during a transaction. In my model, however, conflicts are detected only at commit time. Until the transaction commits, it accumulates values in its read-set and write-map, which will then be used at commit-time to detect conflicts. When a conflict occurs, naturally, the commit of the transaction fails and the transaction is aborted. A conflict—and, thus, a commit failure—occurs when it is not possible to linearize a transaction. In principle, a transaction can be linearized at any time between its start and its end. So, we need to know, for each different kind of transaction, when and whether it can be linearized.

65

66

Versioned Software Transactional Memory

The use of versioned boxes, with the definition of the read operation given in the previous section, ensures that all the transactions access a consistent view of the program state during their execution: After a transaction T starts, and its number is set, it uses this version number to read the appropriate value of each box; even if several transactions commit during the execution of T, the changes made by those transactions are not visible to T. So, ideally, we could linearize each transaction exactly at the time the transaction starts. In fact, if T is a read-only transaction, then, by definition, no boxes are changed during the execution of T. Thus, if T has no effect on the state of the program, we may safely assume that T executed instantaneously when it started—that is, that T is linearizable at the time that it started. Moreover, as nothing is changed by a read-only transaction, there is nothing to be done in the commit of a read-only transaction. If, on the other hand, T is a write transaction, then at least one box is changed during the execution of T. Thus, if T successfully commits, its changes should become visible after the commit. Because of this, T can be linearized only at its commit-time. To see why, imagine that T is linearized before its commit and that between that time and the commit of T a read-only transaction Tr executes. Then, if Tr reads some box written by

T, it should read the value written by T, because it linearizes after T. But, because the commit of T is not done yet, Tr cannot read that value. Therefore, either the commit of

Tr should fail, or T should be linearized after Tr. As the main design decision underlying this model is that the commit of read-only transactions never fails, it follows that write transactions should be linearized at the time of their commits. Now, the problem that we have is that, even though a write transaction is linearizable only when it commits, the consistent view of the program state seen by the transaction corresponds to the program state at the time that the transaction started. Therefore, to ensure that the transaction can be linearized at its commit time, we have to make sure that the (already performed) transaction’s execution is equivalent to the (hypothetical) transaction’s execution at commit time. Assuming that the transaction is deterministic, its execution does not change if the publicly available values read during the transaction remain the same. Thus, to ensure that a top-level write transaction T is linearizable, T can commit successfully if and only if none of the boxes in T-readSet changed after T started—in this case, I say that T is valid. Note that, by this definition, a write-only transaction is always valid, because its read-set is empty. After ensuring that a top-level transaction T is valid, the commit of T proceeds by renumbering the transaction, so that the new T’s number is one greater than the last successfully committed top-level write transaction. Then, each of the values in T-writeMap is added to the corresponding history, tagged with this new number. Of course that the

4.3 The Versioned STM Model

check for transaction validity, the renumbering, and the additions to the boxes’ histories should be all executed atomically, so that no concurrent commit executes between the validity check and the end of the commit operation. The commit of a nested transaction, regardless of its type, is much simpler: it just needs to propagate the changes made during the nested transaction to the parent’s context. More formally, when a nested transaction Tc with parent Tp commits, the elements of Tc-readSet are added to Tp-readSet, and, also, the mappings in Tc-writeMap are added to Tp-writeMap, overriding any existing mapping in the parent’s map. Therefore, the commit of a nested transaction always succeeds, regardless of its type. Finally, aborting either a top-level transaction or a nested transaction simply ends the transaction and does not have any effect on the existing boxes. All the transaction’s values in its write-map, if any, are lost. A key result for this model is that a transaction T1 may conflict with another transaction T2, only if T1 is a top-level read-write transaction and T2 is an already successfully committed top-level write transaction. If T1 is a top-level read-only transaction, a toplevel write-only transaction, or a nested transaction, then T1 will never conflict with any other transaction.

4.3.4

The Linearizability of Top-Level Transactions

In Figure 4.2 on the next page, I present the graphical notation that will be used to illustrate the execution of transactions. In this figure, I show four top-level transactions, which all start with number 3. Transaction T1 is a read-only transaction that corresponds to the execution of the method call o1.m1(); it returns true. Transaction T2 is a write transaction that commits successfully at instant t1 and it is, therefore, renumbered with the number 4. Transaction T3 is a write transaction, also, but its commit fails— presumably, because it conflicts with T2. Finally, T4 is a transaction that aborts at instant t2. The successfully committed transactions in this simple example are the transactions

T1, with number 3, and transaction T2, with number 4. So, in this case, the transactions’ numbers induce a total order on their executions. First we have T1, followed by T2. This is the only linearization possible in this case. In general, however, several transactions may have the same transaction number. When that happens, often there are various possible equivalent linearizations for the transactions. Nevertheless, in either case, each of the possible linearizations respects the partial order specified implicitly by the transactions’ numbers. In fact, given the partial order induced by the transactions’ numbers of all the successfully committed toplevel transactions, it is possible to determine all the equivalent linearizations of those

67

68

Versioned Software Transactional Memory

true

T1 o1.m1() 3 T2 3

4 T3 3 T4 3

Time t1

t2

Figure 4.2: The graphical notation used to represent transactions. This notation is an extension of the notation introduced in Figure 1.2 on page 8. Each horizontal bar represents a complete transaction. The number at the beginning of the bar represents the transaction number when it starts. A bar with a black ending corresponds to a write transaction, and the black portion of the bar corresponds to the commit of the transaction. If the commit succeeds, the number set at the end of the bar is the new number of the transaction, after the commit; a commit failure (because of a conflict) is represented by a white cross over the end of the black bar. A black cross at the end of a white bar identifies the abort of the transaction. Additionally, on the left of each bar, with a gray background, is an id that can be used to refer to the transaction.

transactions. First, note that the definition of the commit operation forces each write transaction to have a new number after its commit—a number which is greater than the number of any of the previously committed transactions. Thus, it is not possible to have two write transactions with the same number. Yet, it is possible to have various transactions with the same number. For instance, in Figure 4.3 on the facing page, I show an example with eight successfully committed top-level transactions, which have only four distinct numbers: Transaction T1 has the number 2; Transactions T2 and T7 share the number

3; transactions T4 and T8 are both numbered 5; and transactions T3, T5, and T6 are all numbered 4. Nevertheless, for each transaction number, there is at most a write transaction; all the remaining transactions are read-only transactions. Second, given a read-only transaction Tr and a write transaction Tw with the same number, it follows from the semantics of the read and the commit operations that the transaction Tw must be linearized before Tr: Because, if the transaction Tr reads some box written by Tw, it must read the value written by Tw. Therefore, from the combination of these two properties, we can derive the first condition that a total order among transactions needs to satisfy to ensure that the transactions are linearizable: For each set of transactions with the same number, the (only one possible) write transaction must be linearized before all the other transactions; the remaining

4.3 The Versioned STM Model

69

T1 2 T2 2

3

T3 4 5

T4 2 T5 3

4

T6 4 T8 5

T7 3 Time t1

t2

t3

t4

Figure 4.3: Example of 8 successfully committed top-level transactions. Transactions T2, T4, and T5 are write transactions. All the others are read-only transactions.

(read-only) transactions in the set can be linearized in any order. Moreover, if we require that a transaction with a number N must linearize before any other transaction with a number greater than N , we have the two necessary conditions to define all the possible linearizations of a set of transactions. More formally, given a set of successfully committed top-level transactions, S, all the total orders on S that respect the following condition represent possible and equivalent linearizations of S’s transactions:

 

∀Ti ,Tj ∈S,i ,j n (Ti ) < n (Tj ) ∨ n (Ti ) = n (Tj ) ∧ w(Ti ) ⇒ (Ti ≺ Tj ) where n (T ) represents the number of the transaction T after the commit, w(T ) is true if and only if T is a write transaction, and Ti ≺ Tj means that Ti appears before Tj in the total order. For the example shown in Figure 4.3, the serial order obtained by assuming that read-only transactions are linearized at the time they start and that write-transactions are linearized at the end of their commit is: T1 ≺ T2 ≺ T7 ≺ T5 ≺ T3 ≺ T6 ≺ T4 ≺ T8 This serial order is one of the two that satisfies the previous condition. The other equivalent serial order is obtained by swapping the order of T3 and T6 .

70

Versioned Software Transactional Memory

T1 3 T2 3

T3 4

4

5

T4 4 Time t1

t2

A

t3

A 1

3

t4

t5

A 2 1

4

3

3 2 1

5

4

3

Figure 4.4: Example of unreachable values. The transactions T2 and T4 increment the value of the box A. At instant t2, the commit of transaction T2 adds a new value, 2, to the history of box A. The value 1, however, does not become unreachable at instant t2, because transaction T1 is still active. The value 1 becomes unreachable only at instant t3. Likewise, the value 2 becomes unreachable only at instant t5, when transaction T4 ends.

4.3.5

Garbage Collection of Old Values

With versioned boxes, old values are not lost; they are kept in the history of the box. This is necessary so that running transactions can read the correct values even after later transactions commit new values to a box. But, as old transactions finish, older values become unreachable and, therefore, may be garbage collected. To know precisely when old values may be garbage collected, I introduce the notion of an active transaction. I say that a transaction is active if and only if it is either the current transaction of some thread or the parent of an active transaction. Moreover, I say that a value v of a history h is reachable if and only if: (1) v is the most recent value in the history h, or (2) there is an active transaction T such that versionOf (v) ≤ T-num < versionOf (successor (v)) where the function versionOf returns the version number associated with a value, and the function successor returns the next value in a box’s history. Values that are not reachable are unreachable and may be garbage collected. Note that, once a value v becomes unreachable, no future transaction may make it reachable again, because all the new transactions must have a version number that is at least equal

4.3 The Versioned STM Model

71

setBalance(150) getBalance() 100 T1 acc.deposit(50) 3

4

getBalance() 150

T2 acc.deposit(50) 3

T3 4

5

getBalance() 100 setBalance(200) setBalance(150) Time t1

acc 100

t2

1

acc 150 100

t3

4

1

acc 200 150 100

5

4

1

Figure 4.5: Parallel deposits on the same account using the STM model based on versioned boxes. Transaction T2 conflicts with transaction T1, causing the deposit to be restarted. The reexecution of the deposit operation corresponds to the transaction T3, which commits successfully.

to the version number of the successor of v. In Figure 4.4 on the facing page, I show an example that illustrates when old values become unreachable.

4.3.6

Examples: The Bank Revisited

In Section 3.1, I used the banking domain to illustrate, with some examples, the difficulties of concurrent object-oriented programming in Java. Now, after presenting my proposal for an STM model, I shall discuss briefly to what extent this proposal solves the problems identified. Because the implementation of the versioned STM model is presented only in the next chapter, here I discuss the examples at the model level only, by assuming the two following changes: (1) the balance of each account is stored in a versioned box; and (2) the methods deposit, withdraw, transfer, and totalBalance are atomic—that is, the execution of each of these methods occurs within a transaction. In Figure 4.5, I show the case of two concurrent deposits of an amount of 50 into the same account acc. Like in the original case (see Figure 3.1 on page 33), when no

72

Versioned Software Transactional Memory

trg.getBalance() 0 src.getBalance() 100 T1 totalBalance() 3

100

T2 transfer(100, src, trg) 3

4

src.widthdraw(100) trg.deposit(100) Time t1

src 100

t2

src 0

1

100

trg 0

2

4

1

trg 100 0

4

2

Figure 4.6: Execution of the method transfer during the execution of the method totalBalance using the STM model based on versioned boxes. The use of versioned boxes ensures that the execution of the method totalBalance can complete successfully with the correct answer, even though transaction T2 commits before the totalBalance method reads the balance of the account src.

synchronization exists, the two deposits’ executions proceed in parallel, each one setting the balance of the account to 150; those executions correspond to transactions T1 and

T2 in this case. But now, one of the deposits—the deposit that corresponds to transaction T2—fails, when the commit of the transaction T2 detects a conflict. Transaction T2 is a write transaction because it changes the value of the box that contains the balance of the account acc.2 But, to make the deposit, transaction T2 needs to read the value of the balance before changing it, also. Thus, the balance’s box is added to the T2-readSet. Meanwhile, at instant t2, transaction T1 commits and changes the balance of account acc. So, when T2 commits later, the validity check for

T2 detects that a box in its read-set was changed by other transaction after T2 started. So, the commit of T2 fails and the deposit operation is restarted. The restart of the deposit is shown in the figure as transaction T3, which completes successfully at instant t3, changing the balance of acc to 200, as expected. 2

In Figure 4.5, I make a simplification of the notation. I use the same name acc to denote both the object of the class Account and that same object’s versioned box that keeps the balance. This same simplification is used in subsequent figures, whenever the meaning is clear from the context.

4.4 Implementation of the Versioned STM Model

The second example that I present here, in Figure 4.6 on the preceding page, shows the major advantage of an STM model based on multiple versions of data. Whereas the original version (shown in Figure 3.5 on page 36) produces incorrect results, and traditional STM models cause a conflict between the two transactions, in the STM that I propose in this chapter both operations complete successfully with the correct result. Note that, even though transaction T2 commits before transaction T1 reads the balance of account src, the changes made by T2 do not discard the old value of any of the boxes changed by T2; the old values of the boxes are still available, so that T1 can access them. The old values of the boxes that keep the balance of accounts src and trg become unreachable only after T1 finishes. Finally, the example of a failure during the deposit of a transfer operation, as given in Figure 3.7 on page 39, is trivially solved using an STM: The failure of the deposit causes the transfer transaction to abort. Given that transactions do not make changes to the public state of the program until the transaction commits, there is nothing to undo when the transaction aborts.

4.4

Implementation of the Versioned STM Model

To implement the STM model proposed in the previous chapter, I need to provide answers for two questions:

• How will programmers use the STM in their programs? • What implementing structures do we need to support the model semantics in the programming language?

Obviously, the answers to these questions are not independent of each other—for example, the difficulty of implementing a particular set of constructs influences the choice of constructs to use to support the model at the language level. Moreover, each of the questions raises other questions that deserve as much consideration as these ones. For instance, is the implementation efficient enough to be used for real examples? In this chapter, I describe an implementation of the STM model based on versioned boxes that serves as a reference implementation. This implementation is not meant to be the most efficient, or to be as well integrated as possible with the programming language. Rather, the implementation I describe here was designed to accomplish a good tradeoff among the guiding principles described in Section 1.2.2 on page 4. For instance, regarding the integration of the model with Java—the target programming language of this implementation—the best solution would be, probably, to augment

73

74

Versioned Software Transactional Memory

public class VBox { public VBox(E initial) { ... } public E get() { ... } public void put(E newE) { ... } } Listing 4.1: Skeleton of the generic class VBox. Besides the constructor, which creates a new versioned box, the only public operations for this class are the methods get and put that implement the read and write operations described in Section 4.3.2, respectively.

the syntax of Java with new keywords to declare versioned boxes and atomic blocks. This solution, however, interferes with the tools that programmers use. Likewise, from a performance-only standpoint, it would be preferable to implement the support for the STM model at the compiler and virtual-machine level. But, again, this solution interferes with the tools that programmers use. Furthermore, this solution is harder to implement than others. Therefore, the implementation that I describe here—the JVSTM—is a pure-Java implementation of the model. Besides the guiding principles already mentioned, the design of the JVSTM was influenced by the two primary goals that I want to accomplish with this implementation. First, that the implementation serves as an operational semantics of the STM model proposed. Second, that it can be used readily and effectively in the implementation of real examples. I do not present in this dissertation all the details of the JVSTM implementation. In particular, I do not show the complete Java source code that implements the JVSTM.3 Rather, I describe the most significant design decisions and skip over the implementation details that do not add much to the work presented in the remaining of this dissertation.

4.4.1

The JVSTM’s API

I start with a description of the JVSTM’s API, which is all the information that programmers need to know to be able to use the JVSTM in their programs. One of the major design decisions regarding the JVSTM was that its interface should be simple and easy to use. JVSTM is implemented as a pure-Java library that provides only two visible interfaces for the programmers that use it. One of the two interfaces is the public interface of the VBox generic class, which is shown in skeletal form in Listing 4.1. This class implements the versioned boxes of the 3

The complete source code for the JVSTM is freely available at the JVSTM page [JVSTM].

4.4 Implementation of the Versioned STM Model

public class Transaction { public static void begin() { ... } public static void abort() { ... } public static void commit() { ... } } Listing 4.2: Skeleton of the class Transaction. The three methods shown are the basic operations needed to demarcate transactions when using the JVSTM.

STM model proposed in the previous section: Each instance of this class is a versioned box, capable of holding a history of values. The method get corresponds to the read operation, returning the current value of the box. The method put corresponds to the write operation, changing the value of the box to a new value. Furthermore, according to the semantics described in Section 4.3.1, if any of these methods is called by a thread without a current transaction, then the method begins a new transaction, accesses the box, and commits the just created transaction—that is, the execution of these methods is atomic. Because the transactions created by these methods are either read-only or writeonly, they never conflict with other transactions. So, the commit of these transactions never fails. The class VBox allows us to create and use versioned boxes. Now, we need to be able to create more complex atomic actions that use these boxes. For that, the JVSTM supplies the class Transaction, which provides the basic operations shown in Listing 4.2. The method begin creates a new transaction and makes it the new current transaction for the executing thread. The new transaction is a top-level transaction if the thread that calls the method begin has no current transaction; otherwise, it is a nested transaction, child of the thread’s current transaction. The method commit tries to commit the current transaction. If the commit operation detects a conflict between the current transaction and any previously committed transaction, then the method throws an exception—an instance of CommitException, which is a subclass of Java’s RuntimeException— and the commit operation fails. The method abort aborts the current transaction, as expected. These three operations are the basic building blocks needed to create atomic actions. But, as we shall see below, often programmers will use simpler constructs to create the atomic actions that they need in their programs. To illustrate the basic usage of the JVSTM I show, in Listing 4.3 on the next page, the JVSTM’s version of an HelloWorld program. When this program is executed, it produces the output shown in Listing 4.4 on the following page. The HelloWorld program creates a versioned box with an initial value and then changes the box twice, printing the contents of the box between each change. The first change to the box is made inside a transaction that aborts. Thus, the change has no effect, and the second value printed is the same as the first. The second change, however, succeeds, and the third value printed shows the

75

76

Versioned Software Transactional Memory

import jvstm.*; public class HelloWorld { public static void main(String[] args) { // creates a box that holds a string VBox box = new VBox("Hello world!"); // print the box’s contents System.out.println(box.get()); // begin transaction to change the box... Transaction.begin(); box.put("Hi!"); // ...but abort the transaction Transaction.abort(); // the value did not change System.out.println(box.get()); // change the box again... box.put("Hello, again!"); // ...and the new value is printed System.out.println(box.get()); } } Listing 4.3: Complete implementation of JVSTM’s version of the HelloWorld program. This program illustrates the basic usage of the two classes provided by the JVSTM: The classes VBox and Transaction.

# java HelloWorld Hello world! Hello world! Hello, again! Listing 4.4: Output produced by the execution of the HelloWorld program. The second value printed is equals to the first, meaning that the abort of the transaction undid the write to the box.

4.4 Implementation of the Versioned STM Model

class Account { final VBox balance = new VBox(0L); ... long getBalance() { return this.balance.get(); } void setBalance(long balance) { this.balance.put(balance); } ... } Listing 4.5: Changes needed in the class Account to use a VBox to hold the Account’s balance. This listing shows only the parts of the class (shown previously in Listing 3.1 on page 32) that need changes. Besides changing the field balance from a long to a VBox, all the methods that access that field need to be changed also—in this case, just the methods getBalance and setBalance.

new value of the box. Note, also, that the calls to the methods get and put do not need to occur inside a transaction, as explained above. The program HelloWorld, however, is not a typical example of the programs that use (or need) the JVSTM; in fact, this program is not even concurrent. Rather, the JVSTM is meant to be used to create transaction-aware domain objects, which are then manipulated within atomic actions. For instance, we may reimplement the classes Account and Bank (originally shown in Listing 3.1 on page 32 and in Listing 3.3 on page 35), to make the instances of the class Account transactional objects and the methods of the class Bank atomic. In general, to use the JVSTM, programmers need to do only two things: (1) use versioned boxes (instances of the class VBox) to hold all the state of the program that may change during the program’s execution, and (2) use the Transaction’s methods to delimit the operations that should execute atomically. For instance, in Listing 4.5, I show the minimal changes needed to transform the class Account into a transaction-aware class: We need to replace the class’s field by an instance of the class VBox, and, consequently, replace the expressions that read and assign to the field by calls to the appropriate get and put methods. The new class Account, however, is not complete, because the methods deposit and withdraw should be atomic, or else we may have the problem depicted in Figure 3.1 on page 33. To make these methods atomic, we may use the methods from the class

Transaction, but it is not sufficient to call the method begin on method entry, and to call the method commit on method exit. Remember that the call to the method commit can throw a CommitException, because of a conflict. Thus, we need to handle that

77

78

Versioned Software Transactional Memory

class Account { ... void deposit(long amount) { while (true) { Transaction.begin(); boolean txFinished = false; try { setBalance(getBalance() + amount); Transaction.commit(); txFinished = true; return; } catch (CommitException ce) { Transaction.abort(); txFinished = true; } finally { // handles other kinds of non-normal termination if (! txFinished) { Transaction.abort(); } } } } ... } Listing 4.6: Implementation of an atomic version of the method deposit. This implementation uses only the basic begin, commit, and abort operations to implement the intended atomicity semantics. This code handles the case of a transaction conflict, which makes the method commit to throw a CommitException, by reexecuting the method again. If the method’s original body fails for any other reason, then the transaction aborts.

exception and to restart the execution of the method, as many times as needed, in case of a conflict. In Listing 4.6, I show an implementation of the method deposit that implements the expected atomicity semantics of the STM model, by using the basic JVSTM transaction operations. The original body of the method is shown in boldface; the rest of the method’s new body is the idiomatic code needed to handle possible transaction conflicts, in which case the original method’s body is reexecuted.4 Because the code needed to implement atomic actions correctly is so verbose, the JVSTM provides a method annotation to make it simpler: the annotation Atomic. Classes that use this annotation should be post-processed to wrap the annotated methods’ bodies with the necessary Transaction’s method calls. In Listing 4.7 on the facing page, I 4

In fact, when the original body of the method may change any of the method’s parameters, this idiom is not correct. In that case, we should use an auxiliary method to execute the original method’s body.

4.4 Implementation of the Versioned STM Model

class Account { ... @Atomic void deposit(long amount) { setBalance(getBalance() + amount); } ... } Listing 4.7: Use of the annotation Atomic to make the method deposit atomic. This implementation uses only a method annotation to implement the same semantics of the implementation shown in Listing 4.6. The necessary code to support the atomicity is introduced by post-processing the byte code.

show an implementation of the method deposit that uses this annotation. Using this annotation, it is now trivial to make the method withdraw, as well as the Bank’s methods

transfer and totalBalance, atomic: We just need to annotate each of the methods with the Atomic annotation. Finally, note that the execution of an atomic method creates either a nested or a toplevel transaction, depending on whether the method was called during an existing current transaction or not, respectively. For instance, when the method deposit is called by the method transfer, its execution creates a nested transaction, because both the method

deposit and the method transfer are atomic. Naturally, when the nested transaction commits, its changes are merged into its parent—the transfer’s transaction, in this case—as expected. This compositional nature of transactions is a fundamental property to implement rich transactional domain models.

4.4.2

Interaction with the Java Memory Model

The implementation of the JVSTM relies on the revised semantics for the Java Memory Model that is now part of Java 5.0 [Gosling, Joy, Steele, and Bracha, 2005, Chapter 17]. In this section, I give an overview of the fundamental concepts of this new memory model that are necessary to understand the implementation of the JVSTM. The original semantics of the Java memory model was designed to give some safety guarantees regarding the execution of a multithreaded Java program while allowing, at the same time, an efficient implementation in multiprocessor machines. This original semantics of the Java memory model, however, suffered from a series of flaws that made its use awkward and error-prone [Pugh, 1999]. Thus, to solve these problems, Java 5.0 adopted a new and improved version of the memory model [Manson, Pugh, and Adve, 2005]. The Java memory model specifies how memory operations in a multithreaded Java program appear to take effect. For instance, to allow for compiler and hardware optimiza-

79

80

Versioned Software Transactional Memory

tions, not every write to a memory location performed by a given thread needs to become immediately visible to reads occurring in parallel threads—for example, because the write operation was reordered with respect to other operations, or because the write was performed to the processor cache which has not been flushed to main memory yet. Thus, to understand how two or more threads may communicate through shared-memory, it is crucial to understand the semantics of the memory model. In this aspect, the new memory model for Java is much more intuitive and easy to use than the original. The most important guarantees of the Java memory model in what concerns the implementation of the JVSTM are the following:

• Writes to and reads of references and primitive types (other than long and double) are always atomic. This means that a thread that reads from an int variable, for instance, will always get a value that was necessarily assigned to that variable, even if it may not be the last value. • The fields declared as final that are correctly initialized during the construction of an object are immutable and thread-safe. That is, once the constructor returns, all the writes to final fields must be visible to all the threads and, thus, it is safe to access these fields from other threads without synchronization. • Variables declared as volatile provide a synchronization point in a program: A write to a volatile variable synchronizes with all the subsequent reads of that variable by any thread. An important consequence of this semantics is that, if a thread t writes to a volatile variable v, then all the writes made by t (even for normal variables) before the write to v become visible to any other thread that reads v after the write made by t.

This semantics of volatile variables is crucial to allow us to guarantee that the changes made to memory by one thread are consistently seen by the remaining threads. The use of volatile variables, however, should not be made lightly. Because of their semantics, reads of and writes to volatile variables increase the cache-coherency traffic in a multiprocessor system, which may result in a significant performance penalty.

4.4.3

Implementation of Versioned Boxes

To implement versioned boxes, I use a variation of the Handle/Body idiom [Coplien, 1992]. Each versioned box is separated into a handle and a series of bodies. The handle instance—an instance of the class VBox—represents the versioned box’s identity, and its single field is a reference to its most recent committed body. Each body—an instance of the class VBoxBody—represents a particular version of the box’s state, a value in the versioned box’s history. The body has fields to represent the box’s value, a version

4.4 Implementation of the Versioned STM Model

81

class VBox { VBoxBody body; ... } class VBoxBody { final E value; final int version; final VBoxBody next; ... } Listing 4.8: Structure of the classes VBox and VBoxBody. These classes follow the Handle/Body idiom to implement versioned boxes. All the VBoxBody’s fields are final to ensure that bodies are immutable and, thus, thread-safe.

B body:

next: value:

next: 2

version:

value:

next: null

87

1

version:

value: 23

0

version:

5

Figure 4.7: Structure that represents a versioned box with three values in its history. The box on the left is the VBox’s instance. The three boxes on the right are the VBoxBody’s instances. These objects represent the versioned box B depicted in Figure 4.1 on page 63.

number, and a reference to the body that maintains the previous value in the box’s history. Through this reference, the bodies of a versioned box form a linked list, sorted in descending order of their version number. The version number in each body is the number of the transaction that committed the body. In Listing 4.8, I show the structure of these two classes, and, in Figure 4.7, I show the four objects needed to represent a versioned box with three values in its history (see Figure 4.1 on page 63). All the fields in the class VBoxBody are final, which makes the bodies immutable and, therefore, thread-safe. This is essential to ensure that any thread that accesses a body sees that body properly initialized; in particular, with the correct version number. The sequence of bodies accessible through a box represent only the values successfully committed to that box. Thus, if a box was written during a transaction, but that transaction has not committed yet, the value wrote to the box is not in its sequence of bodies. Rather, the new value is kept in the private memory of the transaction, as we shall see in the next section. This new value, however, is the current value of the box for that transaction, and it should be the value returned by a call to the method get, if that call is made during the same transaction. So, none of the two VBox’s methods, the get and the put methods, accesses the body of a box directly. Instead, each one delegates its work to the current transaction.

82

Versioned Software Transactional Memory

class Transaction { static volatile int lastCommitted = 0; int number; Transaction parent; Map readMap = ...; Map writeMap = ...; } Listing 4.9: Some of the fields of the class Transaction.

The method get asks for the current value of the box to the current transaction. The value returned by the transaction may be one of the values of the box’s history, if the box was not written yet during the current transaction, or a value that exists only in the private memory of the transaction, if the box was previously written during that transaction. The method put simply asks that the transaction records a new value for the box in the transaction’s private memory.

4.4.4

Implementation of Transactions

In Listing 4.9, I show some of the fields of the class Transaction. The static field

lastCommitted, which keeps the number of the last committed transaction, works as a global counter to give an initial number to each new top-level transaction. The value of this static field changes only when a top-level write transaction successfully commits, in which case it is incremented by one. The volatile field lastCommitted works as a synchronization point that affects all the transactions in the JVSTM. The implementation of the JVSTM ensures that: (1) a write transaction writes to the lastCommitted field only after it has performed all the changes in shared-memory that are needed for a commit; and (2) a new transaction reads of the lastCommitted field before executing any of its shared-memory operations. Given the semantics of volatile variables, this implementation ensures that all the changes made by the commit of a transaction t become visible to all the transactions that start after t has written to the field lastCommitted. The (atomic) write to the field

lastCommitted marks, thus, the commit of a transaction. The field writeMap corresponds to the transaction’s private storage. The transaction uses this map to record the new value of each of the boxes written during the transaction. When, during a transaction, a box is written for the first time, a new entry is added to

writeMap, mapping the box to its new value. If, during the same transaction another value is written to the same box, then that new value overrides the previous value in the map. The writeMap does not need to be thread-safe because no other thread can access this map.

4.4 Implementation of the Versioned STM Model

When a box is read, the transaction consults first its writeMap to see whether a new value exists for the box. This search is performed recursively in the transaction’s parent until either a value is found in some writeMap, or no parent transaction exists. If no value is found in any of the ancestors’ writeMaps, then the transaction searches for the appropriate value in the box’s sequence of committed bodies. The search for the correct value in the sequence of bodies of a versioned box is a linear search in that sequence. The value to return corresponds to the body with the higher version number which is less than or equal to the current transaction’s number. Because the sequence of bodies is sorted in descending order of version number, the search stops once a body with a version number less than or equal to the transaction’s number is found. Typically, the search stops at the first element of the sequence; only if a concurrent transaction committed a new value for the box after the current transaction started, will the search need to go further in the sequence of bodies. If the box has already a new value committed by a concurrent transaction, then this transaction will read an older value, which may cause a conflict for this transaction if it ever writes to some box and then tries to commit. But, if the current transaction is a read-only transaction, it may proceed and commit successfully. Thus, the only case in which we could cause the failure of the current transaction (if we read a box with newer committed values), is when we know already that the current transaction is a write transaction. It is not clear, however, that it is worthwhile to perform this test every time we read a box. If the probability of a conflict is low, then it might be best to skip that test and detect the conflict only at commit time; this is the approach that I use in the current implementation of the JVSTM. The field readMap implements a transaction’s read set. A transaction inserts a mapping into the readMap, only when a box’s public body is consulted by the transaction, after trying first all the transaction’s writeMaps. In this case, the transaction records in the readMap that the box was read, and what was the body read for that box. In Figure 4.8 on the next page, I show an example of one box, identified as box A, and two transactions, T1 and T2, that access the box A. Note that two new values exist for box A: one in each of the transaction’s writeMap. Nevertheless, these values are not accessible through the box, because neither of the transactions committed yet. Thus, these values are only visible in their corresponding transactions. The values stored in the writeMap of a transaction are added to the sequence of bodies of their corresponding boxes, only when the transaction commits, and only if the transaction is valid. In Figure 4.9 on page 85, I show what happens if transaction T2 commits. Transaction T2 is valid, because it is a write-only transaction (its readMap is empty). So, the commit of the transaction proceeds by renumbering the transaction (from

4 to 5), and by committing a new body for each of the entries in the writeMap (in this

83

84

Versioned Software Transactional Memory

A body:

T1 number:

4

next: null value:

readMap:

100

version:

writeMap:

3

T2 number:

150

4

readMap: empty writeMap:

0

Figure 4.8: Values stored in the Transaction’s fields readMap and writeMap. The box A contains only one body in its history, with the value 100. That value was read by transaction T1 before the new value 150 was written to the box. Because the readMap of transaction T2 is empty, either T2 never read the box A, or the read was done only after writing the value 0 into it.

case, only one). To commit a new body for an entry in the writeMap, the transaction creates a new body (initializing it with the correct values) and then replaces the current body of the entry’s box by this new body. In the case shown in the figure, the transaction T2 creates a new body with the value 0, the version 5, and pointing to the current body of box A; then, it sets the body of A to this new body. Note that, given that the bodies are immutable, when the new body replaces the current body of the box A, it is already properly initialized and may be accessed by other threads, which will see the final values in the slots of the body. After the commit of transaction T2, however, the transaction T1 is not able to commit anymore. To commit transaction T1, we need to check first whether T1 is valid. The validity check verifies that all the bodies in the readMap correspond, still, to the most recent body of their corresponding box. Thus, because the current body of box A is not the body read by T1, the check fails. Note that, if the transaction T1 committed before transaction T2, then the commit of T2 would still succeed, because T2 has not read anything. This discussion of a transaction’s commit applies only to top-level transactions. The commit of a nested transaction is much simpler, because it only needs to propagate the contents of both the readMap and the writeMap to the transaction’s parent. The JVSTM implements this by adding all the entries of the child’s maps into the corresponding maps of the parent.

4.4 Implementation of the Versioned STM Model

85

A body:

T1 number:

4

next: null value:

readMap:

100

version:

writeMap:

3

150

next:

T2 number:

5

value:

readMap: empty writeMap:

version:

0 5

0

Figure 4.9: Final result after transaction T2 commits. The cross on the link between the field body of box A and its original body represents the fact that this link no longer exists. Instead, the field body now points to the new body created by transaction T2, which is shown on the bottom-right corner of the figure. The structures of transaction T2 are shown in gray to represent the fact that the transaction finished.

4.4.5

Atomic Commits

The commit of a top-level write transaction is the only place in the JVSTM where the shared state of the program is changed. Thus, at least during such commits there must be some form of synchronization. In fact, as we saw in Section 4.3.3, the commit of a write transaction must execute atomically. But, what does this need for synchronization entails? And to which extent do we need synchronization between the various JVSTM’s transactions? The careful use of the volatile field lastCommitted, as described in the previous section, allows that transactions proceed without any other form of synchronization during their entire execution, up to (but excluding) the commit operation. In fact, even when a transaction is committing new bodies for the boxes that it wrote to, the remaining transactions may continue to access those boxes without any kind of synchronization; not even some form of volatile-like memory barrier, given that the field of a box is not volatile. To see why, consider what may happen if a transaction t1 is committing a new body for a box b while another transaction t2 is reading that box’s history to access the box value. Given that writes to and reads of references in Java are atomic, either the transaction t2 gets the new body for box b, or the old one; there is no other alternative. The value of b in t2 should be the old value, given that t2 started before t1 finished its commit. So, the problem, if it exists, is when t2 gets the new body for b. If that happens, however, t2 will skip over that body and access the old body either way because the version of the new body is necessarily higher than the version of t2. Also, the access of t2 to the new body is perfectly safe because bodies are immutable and, therefore, thread-safe.

86

Versioned Software Transactional Memory

3

T1 2

commit() T2 2

4

commit() 5

T3 2

commit() 6

T4 2 T5 2

7

commit() Time t1

t2

t3

t4

t5

Figure 4.10: Several write transactions committing at the same time. This figure shows that, if multiple top-level write transactions try to commit at the same time, then all but one of the transactions must wait until it gets hold of the exclusive commit lock.

This lack of synchronization during the execution of a transaction extends farther in the case of read-only transactions, because their commit is essentially void. So, an important property of the JVSTM implementation is that the execution of transactions, when restricted to read-only transactions, is wait-free [Herlihy, 1991]. In fact, even though the system as a whole is not wait-free when we consider also write transactions, all readonly transactions satisfy the wait-freedom property that they make progress in a finite number of steps. The synchronization of commits for top-level write transactions, however, is crucial to ensure that transactions are linearizable. We cannot allow that—after checking that a transaction is valid, but before the commit finishes—any of the boxes read by the transaction is modified in any way by a parallel commit. In my current implementation of the JVSTM, this is accomplished by forcing all the commits of top-level write transactions to synchronize on a single object. Thus, the commit of top-level write transactions execute sequentially, as shown in Figure 4.10. Clearly, as Figure 4.10 illustrates, this solution may not scale well for machines with many processors and workloads with a high percentage of write transactions. Yet, I chose this solution for the current implementation of the JVSTM not only because of its simplicity, but also because it fits well with the assumptions underlying the design of the JVSTM; in particular, that there are many more reads than writes, and that write

4.4 Implementation of the Versioned STM Model

transactions change a small number of objects. Nevertheless, I emphasize that neither the execution of concurrent transactions (other than their commit), nor the commit of read-only transactions are affected by this implementation choice. There are, however, alternatives to this simple implementation that may increase the parallelism of the commits. For instance, rather than acquiring a single exclusive lock for the entire commit, use read-write locks per object and acquire, in a well-defined total order, read locks for each of the boxes in the readMap, and write locks for each of the boxes in the writeMap. Then, while holding the locks, proceed with the commit, and, at the end of the commit, release all the locks. The problem with this alternative is that it may well be that the overhead of managing the locks offsets the gains of the increased parallelism, at least for the currently available machines with a limited number of processors. A better alternative may be to continue to execute the commits one after another, but execute each commit in parallel. The commit of a transaction needs to validate each element of the transaction’s read set and then commit new bodies for each of the boxes in the transaction’s write set. Obviously, each of these operations may be performed in parallel. So, the key idea here is that a committing transaction puts itself into a queue of commit operations and then continuously helps the first operation in the queue by doing part of that operation’s commit, until its own commit is processed. By using well-known lock-free implementations of queues, it becomes relatively simple to implement such a scheme for the commit of write transactions, which would make the JVSTM lock-free. In fact, I made some experiments with such an implementation, but the overall performance of the JVSTM in my benchmarks was much lower than with the single-lock solution described above. Yet, given the limited extent of my experiment, the results obtained were only anecdotal and far from conclusive.

4.4.6

Speculative Read-Only Transactions

The implementation of transactions described above registers all the boxes that are read during the transaction in the transaction’s readMap . This recorded information is necessary to validate the top-level write transactions when they commit. But, for read-only transactions, all this registering is useless work, and, worse, constitutes a significant overhead, both in memory, to store all the map’s entries, and in time, to update that map. Moreover, when we read a box, the transaction searches through the hierarchy of transactions for an existing body in any of the hierarchy’s writeMaps, again introducing a significant overhead. Therefore, I would like to eliminate these overheads for the common case of read-only transactions. Unfortunately, we do not know beforehand whether

87

88

Versioned Software Transactional Memory

Transaction

ReadWriteTransaction

NestedTransaction

ReadTransaction

TopLevelTransaction

Figure 4.11: The hierarchy of transactions implemented by the JVSTM.

a transaction is read-only or not. We can, however, speculatively assume that a top-level transaction is read-only when it starts. If, contrary to this assumption, a write operation is attempted during the execution of the transaction, then we must abort and restart the transaction as a generic read-write transaction. In programs where read-only transactions largely outnumber read-write transactions, the cost of restarting the erroneously assumed read-only transactions may be much less than the cost of unnecessarily using read-write transactions. One way to improve on this strategy is to give to the programmer the possibility of giving hints about the nature of a transaction. Another, is to adaptively change the nature of a transaction in runtime based on prior executions of the same atomic action. This speculative strategy can be applied to nested transactions as well, but requires some special considerations. If the parent transaction is not read-only, the nested transaction still has to register all the boxes read, so that it can add them to the parent at commit time. Therefore, we can use a nested read-only transaction only when the parent is a read-only transaction. Moreover, when a write operation is executed in a nested read-only transaction we should restart the top-level transaction, rather than the nested transaction itself. To implement these different kinds of transactions, the JVSTM provides several subclasses of the class Transaction, as shown in Figure 4.11.

4.4.7

Implementation of Garbage Collection

To conclude the discussion of the most relevant aspects of the JVSTM implementation, I describe in this section how the JVSTM implements the garbage collection of old values

4.4 Implementation of the Versioned STM Model

from the boxes’ histories. As we saw in Section 4.3.5, old values are needed only as long as there are active transactions that may need to access them. Thus, when an old value becomes unreachable, we can discard that value from the box’s history—that is, from the sequence of bodies. A corollary of the definition of unreachable values is that, if a box’s history value is unreachable, then all the previous versions in that same history are unreachable, also. Therefore, to discard an unreachable value, we may simply trim the tail of the sequence of bodies, from the point where the unreachable value occurs forward. This is accomplished by setting the field next of the preceding body to null: After this trimming, all the

VBoxBody’s objects that were in the tail of the body that was trimmed become garbage collectable by the Java runtime. In the following, I refer to this trimming process as the cleaning of a body. Now that we know how to garbage collect unreachable values, we just need to know how to find those unreachable values, and when to run the garbage collection process. Starting with the latter problem of knowing when to run the garbage collection process, it follows from the definition of an unreachable value that values may become unreachable only when a top-level transaction finishes (either with a commit or an abort). Thus, in the JVSTM, the cleaning of unreachable values occurs as part of the finishing of a top-level transaction: When a top-level transaction finishes, it checks whether its finishing has made some values unreachable and, if that happens, it cleans those values. Yet, the finishing of a top-level transaction makes some values unreachable only if certain conditions are met. Namely, assuming that the finishing transaction is T, both of these conditions must hold: (1) the current value of the lastCommitted counter is greater than T-num, and (2) there is no other active transaction with a number less than or equal to T-num. If the first condition holds, then we know that all the future transactions will have a number greater than T-num. Moreover, if the second condition holds, it means that the finishing transaction is the last active transaction with the number T-num. So, after the finishing of the transaction, some values will necessarily become unreachable. The question now is to find which values become unreachable. The obvious brute force solution to this problem is to sweep all the boxes, cleaning up any unreachable values found. Yet, this solution forces us to keep a list of all the boxes in the program. Moreover, it is unnecessarily inefficient, because most of the boxes do not need any sweeping at all: For instance, boxes with only one value do not have unreachable values. So, an improvement over the previous strategy is to use a list of boxes which are candidates for sweeping. We put boxes into this list when they get a new value, and take them out of the list when they are cleaned up (if they end up with only one value). Nevertheless, this strategy may still examine too many unneeded boxes.

89

90

Versioned Software Transactional Memory

The key observation that allows the JVSTM to implement a more efficient garbage collection process is the following: All the boxes written by one transaction need cleaning exactly at the same time. To see why, consider that the transaction Ti writes new values for the boxes B1 , B2 , . . . , Bn . Then, when Ti commits, each of the boxes B1 through Bn will have a new value tagged with the version number assigned to Ti by the commit operation. Assume that this number is i. Thus, the previous value of each of the boxes B1 through Bn becomes unreachable when all the active transactions with a number less than i finish— that is, the previous values become unreachable all at the same time, because the new value of each box was created at the same time, also. This answers our question of knowing which values become unreachable when the finishing of a transaction T triggers the garbage collection process: We need to clean up, at least, all the bodies committed by the transaction T-num + 1. I say at least, because the finishing of a transaction T may, in fact, cause that various other values become unreachable. Consider that, while T was running, n top-level write transactions committed successfully. Then, the finishing of T may cause that all the values that were updated by the commits of all those n transactions become unreachable. Therefore, the JVSTM implements the garbage collection of unreachable values by keeping track of which transactions are active at a given instant and which boxes need cleaning for a given transaction number. The difficulty of implementing this approach is in doing it without introducing two much synchronization overheads in the bookkeeping of this information. We need synchronization for this bookkeeping because transactions start and finish concurrently. The JVSTM uses the class ActiveTxRecord to keep track of all the bookkeeping information necessary for the garbage collection of unreachable values. The fields of an

ActiveTxRecord, shown in Listing 4.10, hold the following information:

• A transaction number, in the field txNumber . The field txNumber is initialized when the record is created and never changes thereafter. Its value identifies to which transaction number pertains the information of the record. • A list of bodies to clean, in the field bodiesToClean. This list contains all the bodies that were created by the commit of the write transaction with number txNumber. These are the bodies that need cleaning when we know that there is no active transaction with a number less than txNumber. When the bodies get cleaned up, this field is set to null. • A count of the active transactions, in the field running. The field running is atomically incremented when a new transaction with number txNumber starts, and is atomically decremented when that transaction finishes. So, when the value of this field is zero, we know that there are no active transactions with number txNumber.

4.4 Implementation of the Versioned STM Model

class ActiveTxRecord { final int txNumber; final AtomicReference bodiesToClean; final AtomicInteger running = new AtomicInteger(1); volatile ActiveTxRecord next = null; ... } Listing 4.10: The fields of the class ActiveTxRecord. This class uses the atomic variables provided by the standard java.util.concurrent.atomic package.

There may, however, be active transactions with a lower number; so, it is not sufficient to look at the value of this field to know when to clean up unreachable values. • A reference to the next record, in the field next. The field next starts with the null value when the record is created, and is assigned at most once, when a new write transaction commits and creates a new record, in which case this field is assigned to that new record. So, this field creates a linked list of records, with older records pointing to newer records.

The key idea underlying the JVSTM implementation is to keep track of the count of active transactions for each possible transaction number and, when that count reaches zero, to clean up the bodies of the next transaction record that point, now, to unreachable values. To accomplish this, the JVSTM updates this bookkeeping data structure in each of the following cases:

• When a new top-level transaction starts, in which case we must increment the

running count on the record that corresponds to the starting transaction’s number. • When a top-level transaction finishes (either after commit, or on abort), in which case we must decrement the running count on the record that corresponds to the finishing transaction’s number. • When a top-level write transaction commits, in which case we create a new record and link it to the most recent one. When a top-level write transaction commits, it creates a new record with the new number of the transaction and with the list of bodies committed by the transaction. The running count starts with the value 1 because there is one transaction running: the transaction that is committing. When later in the commit process the transaction finishes, it decrements the running count as any other finishing transaction.

The condition that triggers the cleaning process when a top-level transaction finishes is reaching a combination of values for three of the record’s fields: (1) the field running has

91

92

Versioned Software Transactional Memory

a count of zero, which means that no other transaction with the same number is running; (2) the field next has a non-null value, which means that all the future transactions will have a number greater than the currently finishing; and (3) the field bodiesToClean has a null value, which means that this record was already cleaned up and, thus, there is no older transaction running, also. As these values are all read and modified concurrently as part of the starting and the finishing of transactions, we need to ensure that no data races exist, or that, if they exist, they are safe. One way to guarantee the safety of these processes is to use locks to protect the access to these fields in the critical regions. This solution, however, entails too much synchronization. So, the JVSTM implements a lock-free alternative instead. The lock-free algorithm that the JVSTM implements uses the atomic variables provided by the package java.util.concurrent.atomic, which is part of the standard Java libraries. In particular, the JVSTM implementation depends on the compare-and-swap operations provided by these atomic variables to detect and recover from eventual data races. The highlights of the implementation are the following:

• When a new top-level transaction starts it needs to get a transaction number and to increment the corresponding record’s running count. Therefore, it starts with the most recent record and speculatively increments its running count. By doing so, it prevents that other transactions clean up this record (if the record has not been cleaned up already). Then, it must check whether the next field is nonnull. If it is, then there is already a more recent transaction number. As each new transaction must start with the newest transaction number, it backs off of its speculative increment by decrementing the running count again (which may trigger the cleaning process, given that the field next is non-null), and tries again with the next record. The implementation of this operation is shown in Listing 4.11. This algorithm may cause starvation on a thread that is trying to start a new transaction, but only if successive write transactions commit in between. Thus, this algorithm is lock-free, rather than wait-free. • When a top-level transaction finishes, it decrements the running count of the record that corresponds to its number. If the count reaches zero, it needs to check whether it needs to clean up the next record. So, it checks if the field next is null; if it is, then there is nothing to clean up yet. If, on the other hand, the value of next is non-null, then no new transactions may start for this record (given the implementation described in the previous point) and it may have to clean the next record. To see whether this is true, it checks again if the running count is zero5 and if this record was cleaned up already, cleaning up the next record if both 5

It may not be, because between the first check and the check for the next field, one transaction may

4.4 Implementation of the Versioned STM Model

class ActiveTxRecord { // returns the number to give to the new transaction int startTransaction() { ActiveTxRecord rec = this; while (true) { rec.running.incrementAndGet(); if (rec.next == null) { // if there is no next yet, then it’s because the rec // is the most recent one and we may return its number return rec.txNumber; } else { // a more recent record exists, so backoff rec.decrementRunning(); // and try again with the new one rec = rec.next; } } } } Listing 4.11: The operation used during the start of a new transaction to find the transaction’s number.

conditions are true. Finally, if the transaction cleaned up the next record, it should see whether it needs to propagate the cleaning to the record following it. This is needed because a late finishing transaction may be responsible for cleaning up a series of records that were all waiting for its finish. The implementation of this operation is shown in Listing 4.12. • Finally, the commit of a top-level write transaction is easier to deal with. When a top-level write transaction commits, it creates a new record and sets the field next of the most recent record to the newly created record. This change occurs within the mutual-exclusion lock used for the commit operation and, so, is thread-safe with regard to other commits; also, given that the field next is volatile, this change synchronizes with any other thread reading the same field. Furthermore, changing the field next in the commit of a transaction cannot trigger the cleaning process, because the committing transaction is still running and its number is either equal to the record that will see its next field changed (in which case, that record must have a running count greater than 0), or it belongs to an older record (in which case, the record with the next field updated is not cleaned up). Yet, after changing the next field, the commit operation decrements the running count for its old number, which may trigger now the cleaning process.

have started, thereby incrementing the count, and another transaction may have committed, setting the next field to a non-null value.

93

94

Versioned Software Transactional Memory

class ActiveTxRecord { void finishTransaction() { if (running.decrementAndGet() == 0) { // when running reachs 0 maybe // it is time to clean our successor ActiveTxRecord rec = this; while (true) { // it is crucial that we test the next field first, // because only after having the next non-null, // do we have the guarantee that no transactions // may start for this record if ((rec.next != null) && (rec.bodiesToClean.get() == null) && (rec.running.get() == 0)) { if (rec.next.clean()) { // if we cleaned up, move to the next rec = rec.next; // and repeat the test continue; } } break; } } } boolean clean() { List toClean = bodiesToClean.getAndSet(null); // // // // if

the toClean may be null because more than one thread may race into this method yet, because of the atomic getAndSet above, only one will actually clean the bodies (toClean != null) { for (VBoxBody body : toClean) { body.clearNext(); } return true; } else { return false; } } } Listing 4.12: The operation used when a transaction finishes to clean up unreachable values. The method clean is an auxiliary method that cleans up each of the bodies.

4.5 Related Work

4.5

Related Work

Since the seminal paper on Transactional Memory from Herlihy and Moss [1993], and the later proposal of a software realization of the same idea by Shavit and Touitou [1995], the research on Software Transactional Memory remained mostly dormant until 2003, when a couple of influential papers spurred again the interest in the area [Herlihy et al., 2003; Harris and Fraser, 2003]. Unlike the original STM from Shavit and Touitou, which was lock-free, both the DSTM proposed by Herlihy et al. [2003] and the STM from Harris and Fraser [2003] are based on a weaker non-blocking guarantee: obstruction-freedom. But, by being obstruction-free, these STMs are also simpler and more efficient. Since then, the research on Software Transactional Memory has been immensely active, with many researchers proposing new STM implementations. As a consequence of all this activity, there is now a large number of STM proposals, covering a significant portion of the design space. In [Marathe and Scott, 2004; Marathe, Scherer, and Scott, 2004], Marathe and colleagues make an initial comparison of the then existing approaches to STM, but many other proposals exploring other options were made since then. The versioned STM that I propose in this dissertation, originally presented in [Cachopo and Rito-Silva, 2005, 2006], was the first STM to propose the use of multiple versions for each transactional location. By using multiple versions, this STM is better suited for long-running read-only transactions than the rest. This comes at the expense of more memory overheads. The use of multiple versions to increase the concurrency of transactional systems is well known in the area of database management systems. Since the seminal work of Bernstein and Goodman [Bernstein and Goodman, 1983] on multi-version concurrency control, the technique of using multiple versions as the basis for optimistic concurrency control was applied in several contexts. One such application was made by Graham and Barker [Graham and Barker, 1994]. They presented a formalism for describing multiversion object base systems which is similar, at the model level, to my proposal. However, their work is in the context of object databases, which have different concerns compared to a programming language level software transactional memory. The idea of keeping a history of values for each object whenever it is changed, rather than replacing the old value, was used by Reed in the context of the distributed execution of atomic actions [Reed, 1978, 1983]. However, many of his concerns regarding the difficulty of synchronization on a distributed system do not apply in the context of STM. Moreover, in Reed’s approach, when an object is read, the history of the object may be updated by that read operation, thereby introducing a point of synchronization among concurrent read-only transactions, which defeats somehow the benefits of this approach.

95

96

Versioned Software Transactional Memory

A notable aspect of my implementation is that it uses a single lock to ensure mutual exclusion during the commit, whereas others—e.g. the DSTM from Herlihy et al.—rely on a single CAS operation to perform the commit, because all the values are put in place during the transaction’s execution. My approach is simpler and allows for a natural implementation of nested transactions, which the DSTM and other STMs based on the DSTM design do not support. The problem with my approach, of course, is that it may cause scalability problems if the number of write transactions and the number of available processors rise significantly. Yet, there is nothing in the versioned model that prevents a non-blocking implementation of the commit operation, as I discussed in Section 4.4.5. The support for nested transactions is an important feature in an STM that aims to be usable for implementing a rich domain model, because otherwise it becomes very difficult to make composition of atomic actions. The paper on Composable Memory Transactions by Harris et al. [2005] is well-known for discussing this problem. The semantics of nesting in my STM corresponds to what Moss and Hosking [2005] describe as linear nesting. This simplified model allows for a simpler implementation of nesting and is sufficient to support the retry and the orElse operations from Harris et al. [2005]. Although I did not address these operations in this dissertation, their implementation in my model is quite straightforward. Two important goals for my STM implementation were that it should be easy to learn and use, and that it allowed programmers to use it without having to change their tools. So, I implemented the JVSTM as a pure Java library with a minimal interface. Most of the work on STMs, however, does not share this concern for ease of use. In fact, in many cases, either the STM was implemented as an extension of the language (interfering with the tools) or as a library that has an awkward interface. More recently, however, Herlihy, Luchangco, and Moir [2006] proposed a new version of their original DSTM, the DSTM2, that has a simpler programming interface based on transactional factories. Finally, another STM approach based on multi-versions was proposed recently by Riegel, Fetzer, and Felber in [Riegel et al., 2006b; Riegel, Felber, and Fetzer, 2006a]. This STM borrows much of its implementation from the original DSTM design. Like the DSTM, it uses an intermediate locator object to keep track of the changes made to each transactional object. Yet, whereas the DSTM keeps at most two versions of an object in its locator (the new and the old state of the object), the SI-STM proposed by Riegel et al. keeps track of multiple versions. Unlike the JVSTM, the SI-STM does not support nested transactions, but shares with the JVSTM the fact that it may access a consistent view of the data by reading older versions of the objects, if necessary. Thus, it performs reasonably better than the non-versioned STMs for workloads with long-running transactions.

4.6 Summary

4.6

Summary

This chapter describes a new Software Transactional Memory (STM) that, unlike other STMs, uses multi-version transactional locations called versioned boxes. The chapter starts with a brief introduction to some fundamental concepts of STMs and then describes the rationale underlying the development of this new STM: To create an STM suitable for the implementation of domain-intensive applications. More specifically, to create an STM that handles well workloads that have a high read/write ratio and that are composed mostly by medium to large transactions. After describing the rationale, it describes the proposed Versioned STM model and gives a detailed description of the model’s implementation in the Java programming language. The proposed STM model ensures the linearizability of a set of successful transactions, and supports (linear) nested transactions. Moreover, the model defines in which conditions may a garbage collector free the old versions of each box. The implementation described—the JVSTM—is a full-fledged implementation of the STM model that provides a simple and easy to use API, thereby facilitating its immediate adoption for the implementation of rich domain models. Moreover, the implementation of the JVSTM shows that the versioned STM model is amenable to simple and efficient implementations. The chapter describes, also, a lock-free algorithm for keeping track of old values so that they may be garbage collected when they become unreachable. Finally, the distinctive features of this new STM model and its implementation are that read-only transactions never need to synchronize with any other transactions, and that, also, read-only transactions have the guarantee of being able to complete successfully.

97

98

Versioned Software Transactional Memory

Chapter 5

Domain Modeling Language At least since the introduction of the Chen Entity-Relationship Model [Chen, 1976], well before object-oriented programming became mainstream, relations are an integral part of domain modeling languages. Therefore, we would expect that recent programming languages should have constructs to facilitate the implementation of relations. In fact, about twenty years ago, Rumbaugh argued convincingly for the inclusion of relations as a primitive declarative construct in object-oriented programming languages: Object-oriented languages express classification (the grouping of objects into classes) and generalization (the refinement of classes into subclasses) well, but do not contain syntax or semantics to express relations directly. Any program can implement particular relations on an ad hoc basis, but the abstraction may get lost in the implementation mechanisms. [...] Object-oriented languages have built-in constructs for generalization because it is a natural concept that people use in ordinary discourse; it allows algorithms to be written more concisely and more clearly; and it is common enough to justify building it into a language. Relations are also natural, productive, and common in abstracting applications. An object-oriented language is more expressive if relations are a primitive declarative construct, on the same footing as classes. [Rumbaugh, 1987]

And yet, surprisingly, none of the more recently developed object-oriented programming languages provide such a construct. Unfortunately, the consequence of this oversight in the design of new programming languages is an undesirable burden for the programmers that need to implement domain models, as we have seen in Section 3.3. Thus, given the current state of the object-oriented programming languages, a possible approach to simplify the implementation of domain models is to extend those languages with a set of new constructs for implementing relations. This solution, however, has the inconvenient that interferes with the development tools that programmers use and goes,

100

Domain Modeling Language

therefore, against the guiding-principles that I established in Section 1.2.2. The approach that I propose in this chapter follows a different route. Rather than extending a programming language, I propose a new language—the Domain Modeling Language (DML)—that complements and integrates with the Java programming language. The DML language is a micro-language designed specifically to implement the structure of a domain model: It has constructs for specifying both entity types and associations between entity types. Associations are the primitive declarative construct for relations that was argued for by Rumbaugh [1987]. I start with the rationale underlying the development of this new language, and then I describe the language in detail, showing both its syntax and how it integrates with the Java language. I discuss related work at the end of the chapter, in Section 5.8.

5.1

DML’s Rationale

The objects of a domain model have special needs, compared to the remaining objects of an application, as we have discussed already in Section 2.1.4 and Section 2.1.5. These special needs may include, for example, that they are stored in a database whenever they change, or that they be accessed by multiple concurrent threads in a consistent way. Most often, because of these needs, the classes that represent the domain objects must follow some coding conventions, to ensure that their instances behave as expected. For instance, if we are using the JVSTM described in Section 4.4 to make the domain objects transactional, we must use instances of the class VBox for all the mutable fields of a domain class. In some cases, as when using the JVSTM, the code conventions are simple to follow. But, as simple as they may be, they still must be applied manually by the programmers, introducing unnecessary effort into the programming task, and opening the possibility of errors. Thus, the idea of automating these programming tasks was the first incentive that led me to the development of the DML: I wanted a simple way to specify that a class is a domain class, and to have this class transformed automatically into a class that uses versioned boxes for all its fields. Nevertheless, this reason alone is not sufficient to justify the need of a new language. In fact, transforming each of a class’s field into a field of another type and changing the code that accesses that field to use a different access expression, is well within the reaches of Java’s annotations and post-processing technologies. The most important reason for the development of the DML language, however, was the lack of support in Java for the implementation of associations between classes. To solve this problem in a convenient way, I would need to extend the syntax of the Java language

5.1 DML’s Rationale

with new top-level syntactic constructs to represent associations. Unfortunately, the extension mechanisms available in Java do not allow syntax extensions. The top-level constructs of a Java program are the class and the interface, and no provisions exist to create other syntactic constructs. So, I decided to create a new language—the DML—with the appropriate constructs to represent both the classes and the associations of a domain model. Yet, it is not the goal of DML to replace Java. Rather, it should integrate with Java, so that programmers can still leverage on all the advantages of the Java programming language. The key idea is that this new language should be as small as possible, by providing constructs to represent a domain model’s structure, but leave all the rest to Java. In particular, the behavior of the domain model’s classes should be programmed in Java, as before. Furthermore, the code that implements the classes’ behavior should be the same, regardless of how the class structure is developed (either with DML or in plain Java). Therefore, DML is designed to represent only the structural aspects of a domain model, using, for that, constructs that are as close as possible to the constructs used at the modeling level—constructs such as class and association. But also, because the domain model is not complete without its behavior, which is programmed in Java, the model’s structure expressed in DML must integrate with the Java code that implements the model’s behavior. Both the syntax and the semantics of DML reflect this requirement of seamless integration. As the DML is designed to target specifically Java programmers, its syntax borrows from the syntax of Java whenever possible. Moreover, the semantics of DML is specified by describing the set of Java classes that result from a DML domain specification: To implement the domain model’s behavior, programmers must know the interfaces of these resulting classes. Thus, implicit to the semantics of DML is that there is a compilation process that transforms a DML domain specification into a set of Java classes. Finally, the last requirement that influences the design of DML is that, even though programmers must know the interfaces of the classes generated by the DML compiler, they should not need to know the implementation details of those classes. In fact, to avoid round-trip problems, the source code of the class generated by the DML compiler should not be accessible to the programmers: If programmers could modify the code generated by the DML compiler, the DML compiler would not be able to regenerate the code when some part of the DML domain specification changes. In Figure 5.1, I illustrate the expected effect of using the DML in the tool chain typically used for the development of Java applications. The DML compiler is the only element that is new. It reads a set of DML source files, which specify the structure of a domain model, and generates a set of Java source files that implement the structure of that

101

102

Domain Modeling Language

Java Source

DML Source

Standard Java Tools

DML compiler

Java Classes

Domain Classes’ Sources

May be used by IDEs

Java Source

Standard Java Tools

Java Classes

Figure 5.1: The effect that the DML has on the tool chain typically used for Java application development. The dashed line separates the two different scenarios of tool chain usage. The scenario above the line corresponds to the situation where the DML is not used. The scenario below the line illustrates the changes caused in the tool chain by using the DML. Rectangles represent implementation artifacts of the development process. The rectangles with a white background represent the artifacts that are edited by the programmers, whereas the rectangles with a gray background are tool-generated artifacts that the programmers never modify manually. The large arrows represent the transformation of one kind of artifacts into another kind of artifacts, performed by the execution of the development tools indicated inside the arrow. Finally, the single arrow pointing from the Domain Classes’ Sources to the Java Source indicates that the former artifacts may be used by IDEs to help programmers in the development of the latter.

domain model. I decided to generate Java source code, rather than compiled classes, to eliminate dependencies between this transformation process and other Java classes. As we shall see, the Java source code generated by the DML compiler may need additional application-specific classes to compile. Yet, the DML compiler itself does not need any other application-specific Java class to perform the transformation. So, by generating only source code, the DML compiler may be executed at any time during the development process.

5.2

Grammar Notation and Lexical Structure

The syntax of the DML language is defined using a context-free grammar that I introduce in the following sections. To present this grammar, I use the notation that is used in the Java Language Specification [Gosling et al., 2005, Chapter 2]:

• Nonterminal symbols are shown in italic type, with no spaces between words and with the first letter of each word capitalized. For example, the following are nonter-

5.2 Grammar Notation and Lexical Structure

minal symbols: Identifier, DomainSpecification, and EntityTypeDeclaration. • Terminal symbols are shown in fixed width font. Examples of terminal symbols are: class, }, ;, and playsRole. • Production rules are typeset with the left-hand side nonterminal—the nonterminal symbol defined by the rule—followed by a colon, on a line by itself. Then, on each of the following lines, there is a sequence of symbols that defines an alternative right-hand side of the rule. Thus, the rule

QualifiedName: Identifier QualifiedName . Identifier defines the nonterminal QualifiedName as either an Identifier, or a QualifiedName, followed by the terminal . and an Identifier. This rule is part of the syntactic grammar for the DML language and defines the nonterminal QualifiedName as a sequence of one or more identifiers (explained below) separated by dots. • The suffix opt, which may appear on the right-hand side of a rule, indicates that the symbol that precedes the suffix is optional—that is, that in fact the right-hand side corresponds to two alternative right-hand sides, one where the symbol occurs and another where the symbol does not occur. A lexical grammar for the Java programming language is given in Chapter 3 of the Java Language Specification [Gosling et al., 2005]. This grammar defines how sequences of Unicode characters are translated into a sequence of input elements, which may be white space, comments, or tokens. The tokens, which consist of identifiers, keywords, literals, separators, and operators, form the terminal symbols for the syntactic grammar of Java. The DML is a much simpler language. For instance, in DML there are no literals nor operators. DML uses only identifiers, a few keywords, and operators. Yet, because DML must integrate seamlessly with Java, I use the lexical grammar defined for Java to define the lexical structure for the DML language. Thus, the syntax of DML for such things as comments, identifiers, and white space is the same as in Java. Note that, even though the DML language does not use most of the keywords of Java nor any literals, the identifiers used in a DML specification must follow the restriction that they cannot be a keyword, a boolean literal, or the null literal, because these identifiers will appear as identifiers in the Java source code generated by the DML compiler. In the grammar of DML presented in the following sections I use the nonterminal symbol Identifier as it is defined in the Java lexical grammar: Informally, as a sequence

103

104

Domain Modeling Language

DomainSpecification: DomainDeclarationsopt DomainDeclarations: DomainDeclaration DomainDeclarations DomainDeclaration DomainDeclaration: ValueTypeDeclaration EntityTypeDeclaration AssociationDeclaration Listing 5.1: Syntactic rules for a domain specification. DomainSpecification is the goal symbol for the DML syntactic grammar.

of Unicode characters that start with a letter, and is followed by any number of letters or digits. Furthermore, I use the nonterminal symbol DecimalNumeral as it is defined in the Java lexical grammar, also: Either as the digit 0, or as a sequence of digits starting with a digit from 1 to 9. Finally, I use the nonterminal QualifiedName as defined by the production given above. All the remaining nonterminal symbols are defined in one of the grammar fragments shown in the following sections.

5.3

Domain Specification

The DML language allows us to represent only the structural aspects of a domain model. It has no constructs to describe the behavior of a program. So, what the DML compiler reads and processes should not be called a program. Rather, I call it a domain specification. The syntax of a domain specification is defined by the grammar rules shown in Listing 5.1. The nonterminal symbol DomainSpecification is the goal symbol of the DML syntactic grammar. According to these rules, a domain specification is a sequence of zero or more domain declarations, which, in turn, are either a value type declaration, an entity type declaration, or an association declaration. The following sections describe, in detail, the syntax and the semantics of each of these domain declarations. Each domain declaration introduces a new name that may be used in other parts of the domain specification to refer to the domain element introduced by this declaration. Thus, the only restriction to the order of domain declarations is that the names are introduced before they are used.

5.4 Value Types

ValueTypeDeclaration: valueType ValueTypeName AliasClauseopt ; AliasClause: as ValueTypeName ValueTypeName: QualifiedName Listing 5.2: Grammar rules for the syntax of a value type declaration.

5.4

Value Types

In the DML language, I distinguish between value objects and entities, as described in Section 2.2.3. Value objects, unlike entities, are immutable, are not persistent by themselves (only as part of entities), and do not participate in bidirectional associations. Therefore, the code used to implement a value type is significantly different from the code used to implement an entity type. Furthermore, often the value types used in a domain model are not specific of that domain model. Instead, they may be used across many different domains. So, in many cases value types are provided already by a third-party framework or toolkit. If, however, a value type does not exist yet, the implementation of its structure is mostly trivial, because value objects are immutable. For these reasons, the DML language does not allow the specification of new value types. Yet, value types are needed to define new entity types: All the attributes of an entity type must be of some value type. That is why the DML language has value type declarations. New value types are introduced in a domain specification by value type declarations, which follow the syntax specified by the grammar productions shown in Listing 5.2. Each value type declaration introduces into the domain specification the name of a value type. The definition of this value type, however, is made outside the DML’s domain specification. In its simplest form, a value type declaration is only the keyword valueType followed by the fully-qualified name of a new value type. That name becomes a new valid name that may be used wherever a value type is expected. When the alias clause is used, the name that is introduced into the domain specification as a new value type is the name that follows the keyword as; the real name of the value type is used only by the DML compiler to generate the code with the appropriate types.

105

106

Domain Modeling Language

valueType valueType valueType valueType valueType valueType valueType valueType valueType valueType valueType valueType valueType valueType valueType valueType valueType valueType

boolean; byte; char; short; int; float; long; double; java.lang.Boolean java.lang.Byte java.lang.Character java.lang.Short java.lang.Integer java.lang.Float java.lang.Long java.lang.Double java.lang.Number java.lang.String

as as as as as as as as as as

Boolean; Byte; Character; Short; Integer; Float; Long; Double; Number; String;

Listing 5.3: Default value types in the DML language. The first eight value type declarations introduce the names of the eight primitive types in Java as value types. The following eight declarations introduce the wrapper reference types, which, because of the alias clause, should be used without the java.lang package qualifier. Likewise for the last declaration, which introduces the name String.

As an example, I show in Listing 5.3 the value type declarations for the value types defined by default in the DML language.

5.5

Entity Types

Entity types are the basic elements of a domain specification. Each entity type describes the structure of a set of similar entities, which are the objects that represent the domain’s state.

Entities hold their state in a set of attributes, which may change during the

execution of an application as the state of an entity changes. The value of each attribute, however, must be a value object—that is, the type of each attribute must be a value type. Entities may refer to other entities only through the traversal of the associations between their types.

5.5.1

Syntax of Entity Type Declarations

A new entity type is introduced in a domain specification by an entity type declaration, which follows the syntax specified by the grammar rules shown in Listing 5.4 on the next page.

5.5 Entity Types

EntityTypeDeclaration: class EntityTypeName Superopt EntityTypeBody EntityTypeName: QualifiedName Super:

extends EntityTypeName EntityTypeBody:

; { AttributeDeclarationsopt } AttributeDeclarations: AttributeDeclaration AttributeDeclarations AttributeDeclaration AttributeDeclaration: ValueTypeName Identifier ; Listing 5.4: Grammar rules for the syntax of an entity type declaration.

An entity type declaration is a stripped-down version of a class declaration in the Java programming language. In fact, many entity type declarations in DML are valid class declarations in Java, even though only a few class declarations in Java are valid entity type declarations in DML. As in Java, after the keyword class comes the name of the new entity type. But, whereas in Java the name of the new class must be an Identifier, in DML the EntityTypeName may be a qualified name. When the new entity type is a subtype of another entity type, we represent that inheritance relationship by using the keyword extends followed by the name of the entity type of which the new type is a subtype. Note that the name after the keyword extends must be the name of an entity type introduced by a previous entity type declaration. In DML, an entity type cannot be a subtype of a value type. The body of an entity type declaration is used in DML only to specify the entity type’s attributes, using a syntax similar to the syntax of field declarations in Java. If an entity type does not have new attributes (other than those inherited), the body of its declaration may be replaced by a semicolon. An example of such a simple entity type declaration is

class Bank; Yet, in general, entity types have one or more attributes. So, typical examples of entity type declarations are shown in Listing 5.5 on the following page.

107

108

Domain Modeling Language

class Client { String name; } valueType a.business.api.Money as Money; class Account { Money balance; } class ClientAccount extends Account { boolean closed; } Listing 5.5: Examples of entity type declarations in DML. Both Client and Account are top-level entity types that do not inherit from any other type. The ClientAccount type, however, is a subtype of Account. Also, the Account entity type uses a value type introduced in the previous declaration.

5.5.2

Semantics of Entity Type Declarations

I specify the semantics of a domain specification by prescribing the minimal set of Java classes that a compiler of the DML language must produce when it compiles the domain specification. Also, I prescribe which fields and methods each generated class must have. Different compilers for the DML language may vary in the classes they produce. Yet, all the compilers must conform to the minimal interface specified here, because programmers depend on this interface to develop the rest of the domain model. Therefore, in this section, I specify the semantics of an entity type declaration by describing the minimal Java interface that a DML compiler must produce when it compiles an entity type declaration. We have seen in Section 3.3.1 that classes from a UML class diagram are implemented naturally as classes in Java: Typically, each class of a class diagram—corresponding to an entity type in DML—is implemented by a single class in Java. In DML, however, an entity type is implemented by two classes, as depicted in Figure 5.2 on the next page. Each class implements one of the two aspects of a domain entity:

• The first class—the state class—is abstract and implements the domain entity’s structural aspects. The DML compiler generates this class from the domain specification. Thus, programmers should not edit this class manually. • The second class—the behavior class—extends the state class and implements the domain entity’s behavioral aspects. This class, unlike the state class, cannot be

5.5 Entity Types

109

AState class A { ValueType1 attr1; ... ValueTypeN attrN; }

DML Compiler

getAttr1(): ValueType1 setAttr1(ValueType1 val) ... getAttrN(): ValueTypeN setAttrN(ValueTypeN val)

A

Figure 5.2: The result of compiling an entity type in DML. An entity type A is transformed into two Java classes AState and A. The abstract class AState has a getter and a setter for each of the entity type’s attributes. The gray background in the class AState indicates that the DML compiler generates this class and, thus, that programmers should not edit the class.

generated by the DML compiler, because the domain specification does not have any behavior specification.1 Instead, the implementation of this class is the responsibility of the domain model programmers: This is the class where programmers implement the behavior of the entity.

The DML compiler uses this compilation strategy of separating a class in two to avoid round-trip problems: Whenever we change the domain specification, we must reexecute the DML compiler to update the classes generated in previous compilations, but, because programmers do not edit the state classes, the DML compiler may regenerate those classes from scratch each time it runs. The state class that the DML compiler generates from an entity type has both a getter method and a setter method for each of the entity type’s attributes. The DML compiler forms the names of these methods by concatenating the prefixes get and set, respectively, with the name of the attribute, after capitalizing the first letter of the attribute. So, given an entity object obj, the method call obj.getAttrName() returns the obj’s value for the attribute named attrName, whereas the call obj.setAttrName(newVal) sets the value of the attribute to the value newVal. No other methods can access the attribute. Even though a state class has all the attributes of an entity type and no abstract method, it must be abstract, because it does not represent an entity. An entity contains both state and behavior, but the instances of a state class contain only the state. Therefore, the state class is abstract to prevent the creation of instances of an incomplete type. 1

In fact, the DML compiler generates an empty behavior class if the class does not exist yet, so that we may compile the domain model without errors. If the class exists, however, the DML compiler does not modify it.

110

Domain Modeling Language

AState ...

class A { ... }

A DML Compiler

class B extends A { ... }

BState ...

B

Figure 5.3: The result of compiling an hierarchy of entity types in DML. The Java state class that results from the compilation of an entity type extends the behavior class that corresponds to its supertype: In this case, the class BState extends A.

The class that represents both the state and the behavior of an entity is the behavior class, which implements the behavior and inherits the state from the state class. Note that the methods in the behavior class may access the state of an instance by using the getter and the setter methods inherited from the state class. As a matter of fact, the state class is meant to be used only as the superclass of its corresponding behavior class. No other classes should extend it. Also, no fields or methods or any other part of the code should refer to a state class as a type. Instead, all the remaining code should use the types that correspond to behavior classes. For instance, in Figure 5.3, I show that, if B is a subtype of A, the class that implements the structure of B, the class BState, must extend the class A, rather than the class AState. If the class BState extended the class AState instead of A, it would not inherit the behavior of A, as expected. Finally, there is only one restriction regarding the implementation of a behavior class— that it should not have any state of the entity type; the state belongs in the state class. Besides that restriction, programmers are free to implement the behavior class in whatever way they want. For instance, programmers may make the class abstract, if it is not meant to have any instances; they may make the class implement any number of interfaces; or they may add the methods they need to implement the entity’s behavior. In particular, they may override any of the methods that are specified in this section, which the behavior class inherits from the state class.

5.6 Associations

5.6

Associations

In UML, relations are called associations and they may connect two or more classes to represent a relationship that exists among the objects that the connected classes describe. The DML language uses the name association, also, to represent relationships. But, unlike UML, DML does not allow the specification of associations with an arbitrary arity; in DML all associations must be binary. I chose to support only binary associations in the DML language mostly for pragmatical reasons. First, because even though the semantics of a generic n-ary relation is naturally defined in mathematical terms as a set of tuples, the semantics of n-ary relationships as a domain modeling construct is either ill-defined, or confusing and error-prone [Génova, Llorens, and Martínez, 2002]. Second, because the implementation of n-ary associations in the object-oriented programming paradigm is not that simple. In an object-oriented program, programmers use binary associations to navigate in the object graph, going from one object to another by traversing a link—that is, an instance of an association—between the two objects. Typically, programmers implement such binary links in an object-oriented domain model as references or pointers from one object to the other. When the association is not binary, however, to find the object at the other end of a link, we need more than an object: We need all the objects in the link but one. Thus, we no longer can use simple references between objects to implement such associations. But this is more of a concern for the implementation of the DML compiler, which would need to generate the code to implement the correct semantics (provided that we could agree on one). If we ignore the implementation problems raised by non-binary associations, however, there are still usage problems: Traversing a binary link in a typical implementation of an object-oriented domain model may be linguistically quite different from traversing links that relate more than two objects. The third pragmatical reason for limiting the DML to binary associations, was the observation that associations among three or more entity types are seldom used in the design or in the implementation of a domain model: Most probably because of the two reasons above, programmers shy away from n-ary associations when they develop their domain models. The fourth and final reason is that limiting the DML to binary associations does not prevent the implementation of a domain model. In the rare cases where programmers use a non-binary association, it is possible to replace that association (often with benefits to the design of the domain model) with an entity type and several binary associations: The new entity type represents, as first-class entities in the domain model, the links of the non-binary association that it replaces.

111

112

Domain Modeling Language

Therefore, following the fourth guiding-principle of Section 1.2.2, I considered that the eventual benefits of supporting n-ary associations in DML were not worthy of the effort necessary to implement them.

5.6.1

Syntax of Association Declarations

An association declaration, which adds an association to a domain specification, is a toplevel construct in the DML language—that is, it is a construct that stands by itself, rather than being part of, or having to appear subordinated to another construct. This syntax for associations contrasts with how other approaches to the problem of making associations explicit in the programming language propose to represent associations. Many of such approaches propose to represent associations with new constructs that programmers should use in each of the associated classes. This solution, however, splits the information about the association between the two classes, making both the understanding and the maintenance of a domain model more difficult. Thus, I argue that an association declaration should be a single construct, as it is in the DML language. In Listing 5.6 on the next page, I show the grammar rules that specify the syntax of an association declaration. This set of rules concludes the syntactic grammar of the DML language. The keyword association introduces an association declaration and is followed by the name of the association, an identifier. As we shall see in the following section, the DML compiler uses this identifier to name a static field in each of the classes that participate in the association. Therefore, this name must be unique throughout the inheritance hierarchy of each of those classes. To avoid name clashes, programmers should adopt a naming convention that distinguishes the names of associations from the names of other members of a class (e.g., inner classes). In a relationship among several entities, each of the entities plays a role in the relationship. Thus, the DML language uses role declarations to identify the entity types that participate in an association. As in DML all the associations are binary, an association declaration has exactly two role declarations. Each role declaration specifies the type of the entities that play the declared role in the association, the name of the role, and the multiplicity of the role. Yet, both the name and the multiplicity of the role are optional. If the name of a role is not specified, then the association is not traversable in that direction—that is, it is not possible to reach the entities that play that role from the entities in the other end of the association. I shall discuss this further in the following section, where I present the semantics of an association declaration. If the multiplicity of a role is not specified, however, a default multiplicity of 0..1

5.6 Associations

AssociationDeclaration: association Identifier RoleDeclarations RoleDeclarations: { RoleDeclaration RoleDeclaration } RoleDeclaration: EntityTypeName playsRole Identifieropt RoleBody RoleBody:

; { MultiplicityOption } MultiplicityOption:

multiplicity MultiplicityValues ; MultiplicityValues: MultiplicityRange MultiplicityValues , MultiplicityRange MultiplicityRange: MultiplicityUpperBound MultiplicityLowerBound .. MultiplicityUpperBound MultiplicityUpperBound:

* DecimalNumeral MultiplicityLowerBound: DecimalNumeral Listing 5.6: Grammar rules for the syntax of an association declaration. Each association declaration has exactly two role declarations, identifying the two entity types that participate in the relationship. The nonterminal symbol DecimalNumeral is defined in the Java lexical grammar [Gosling et al., 2005] either as the digit 0 or as a sequence of digits that start with a non-zero digit.

113

114

Domain Modeling Language

association AccountOwnership { Client playsRole owner { multiplicity 1; // it is equivalent to 1..1 } ClientAccount playsRole account { multiplicity 1..*; } } association AccountGroup { CheckingAccount playsRole checking { multiplicity 1; } SavingsAccount playsRole savings { multiplicity *; // it is equivalent to 0..* } } Listing 5.7: Examples of association declarations in DML. Both associations are bidirectional one-to-many associations. Note that the role names and the multiplicities match those specified in the class diagram shown in Figure 2.4 on page 24.

is assumed. Moreover, the multiplicity range * is a shorthand for 0..*. Finally, a multiplicity range of the form N, for any positive integer N , is a shorthand for N..N. Thus, every role has a multiplicity value, either implicitly or explicitly declared, which is a sequence of one or more ranges with a lower bound and an upper bound. Note that the syntax for an association declaration could be made simpler. For instance, we could eliminate the keywords playsRole and multiplicity, as well as some of the curly braces. The reason for having this syntax, however, is that I designed it with extensibility in mind. For example, I foresee the need to add other options to a role declaration. Yet, given the current syntax for the multiplicity option, the syntactic changes to add other options would be minimal. To conclude this section I show, in Listing 5.7, two association declarations that represent two of the associations from the banking domain.

5.6.2

Semantics of Association Declarations

Like before, when I specified the semantics of an entity type declaration, I specify the semantics of an association declaration by prescribing the minimal Java interface that a DML compiler must generate for each association declaration. In this case, however, the DML compiler does not need to generate any new classes to

5.6 Associations

implement an association declaration. Rather, it must generate new methods for each of the entity types that play a role in the association. The semantics that I specify here tries to respect the pragmatics of object-oriented programming. Object-oriented programmers do not, as a common practice, implement a binary association or the association’s instances—the links—as first-class objects in the program. Instead, the object-oriented common practice is to implement a binary association between two classes A and B, by adding to each of the classes A and B a reference to the elements of the class in the other side of the association: In the class A we add a reference to elements of B; in the class B we add a reference to elements of A. These references, however, are not typically accessible by the classes’ clients. As we have seen in Section 3.3.2, the usual approach is to provide methods that access these references. Thus, because programmers should use the public methods, rather than the private references, the semantics that I specify here prescribes only which methods must a DML compiler generate. It does not specify how the compiler should implement those methods. It does not specify either, how the generated code should implement the association’s links, whether with references in each of the classes, or with any other solution. To simplify the presentation below, given an association declaration with two role declarations, I use the term opposite type of a role declaration to refer to the entity type of the other role declaration. For example, given the following association declaration

association Rel { A playsRole role1; B playsRole role2; }

I say that the opposite type of the first role declaration is B, and that the opposite type of the second role declaration is A. A compiler for the DML language must generate at most two sets of related methods for an association declaration—one set of methods for each of the role declarations that have a role name specified. If a role declaration has no name, then the DML compiler should not generate any methods for that role declaration. The methods generated from a role declaration belong to the opposite type of that role declaration. They allow the traversal from an instance of the opposite type to one or more instances of the role declaration’s type. Therefore, an association where both role declarations have a name is a bidirectional association, whereas if only one role declaration has a name, the association is unidirectional. An association declaration where none of the role declaration have a name is meaningless for a domain specification, because no code will be generated from it.

115

116

Domain Modeling Language

The signature of the methods that a DML compiler must generate for a role declaration depends only on the properties of that role declaration:

• The role’s multiplicity determines the set of methods to generate. Although the multiplicity option may have many different values, the DML compiler separates the roles into two disjoint classes: (1) the roles that have a multiplicity upper-bound of one; and (2) the roles that have a multiplicity upper-bound greater than one.2 The multiplicity upper-bound of a role declaration is the maximum of the upper-bounds of all the multiplicity ranges (there is at least one) in the role declaration. The upperbound of a multiplicity range of the form M..N is infinite, if N is the terminal symbol

*, and the value of the integer N, otherwise. I shall present below, separately, the set of methods that must be generated for each of these cases. • The role’s name determines the exact name of each method. The name of the role with the first letter capitalized appears in all the methods’ names, either with a prefix only, or with both a prefix and a suffix. Below, when I present the methods to generate, I show in italic the part of the name of each method that must be replaced by the role’s name. • The role’s type determines the type of the argument, or the return type of some of the methods to generate.

Regardless of the multiplicities involved, the methods that implement an association must allow us to do each one of the following tasks: (1) create a new link between two objects, (2) remove an existing link between two objects, and (3) traverse from one object in one end of a link to the object on the other end. The specific signature of the methods that allow us to do this, however, depends on the multiplicity of each role. I shall present each case separately.

5.6.2.1

Roles with a multiplicity upper-bound of one

If the multiplicity of a role declaration has an upper-bound of one, then an object of the opposite type may refer to at most one object of the role’s type. This is the simplest case of an association, which is typically implemented with a getter and a setter method. In Figure 5.4 on the next page, I show the signature of the methods that a DML compiler must generate for a role declaration with a multiplicity upper-bound of one. Only the first two methods, shown in boldface, are necessary to implement the association. The other two methods may be trivially implemented by using the mandatory methods. Yet, they make the class more convenient to use, and, therefore, more programmer-friendly. 2

Roles with a multiplicity upper-bound lesser than one do not make sense.

5.6 Associations

association RelAB { A playsRole role { multiplicity 0..1; } B playsRole; }

117

BState DML Compiler

... setRole(A role) getRole():A hasRole():boolean removeRole()

Figure 5.4: Methods that a DML compiler must generate for a role declaration with a multiplicity upper-bound of one. The part of a method’s name that depends on the name of the role is shown in italic. The methods in boldface are the core methods. The other methods are optional methods.

The method setRole is the setter method. It allows us to create or to eliminate a link between an instance of class B and an instance of class A. To create a link, we should call the method with an instance of class A as argument. In this case, the method execution creates a new link between the receiver and the argument of the method call. But, because we can have at most an instance A linked to each instance of B, if there was already a link between the receiver of the method call and another instance of class

A, the method must eliminate that link before it creates the new link. Finally, the call to the setter method with a null argument eliminates any current link that may exist for the receiver of the method call. The method getRole is the getter method. Given an instance of the class B, b, the method call b.getRole() returns the instance of the class A that is related to b, if such an instance exists, or null, otherwise. This method is the method that allows us to traverse the association. The method hasRole returns true if there is a link between the receiver of the method call and an instance of class A. Otherwise, it returns false. Calling this method on an object obj is equivalent to evaluate the Java expression obj.getRole() != null. The method, however, may be implemented more efficiently by the DML compiler. Finally, the method removeRole removes an existing link that may exist for the receiver of the method call. It is equivalent to call the setter method with a null argument.

5.6.2.2

Roles with a multiplicity upper-bound greater than one

The difference between this case and the previous is that the object of the opposite type may have multiple links, at the same time, with objects of the role’s type. So, when we create a new link, we do not remove any existing link, as in the previous case. The existing links must be removed explicitly, by specifying which object is on the other end of the link that should be removed. Furthermore, when we traverse the association, we may reach multiple objects. Thus, the getter method, in this case, must return a collection of

118

Domain Modeling Language

AState

association RelAB { A playsRole; B playsRole role { multiplicity 0..*; }

DML Compiler

}

... addRole(B role) removeRole(B role) getRoleSet():Set getRoleCount():int hasAnyRole():boolean hasRole(B b):boolean

Figure 5.5: Methods that a DML compiler must generate for a role declaration with a multiplicity upper-bound greater than one. The part of a method’s name that depends on the name of the role is shown in italic. The methods in boldface are the core methods. The other methods are optional methods.

objects. In Figure 5.5, I show the signature of the methods that a DML compiler must generate for this case. Like in the previous case, there is a set of core methods, and a set of convenience methods. The method addRole creates a new link between the receiver of the method and an instance of the class B passed as argument to the method. If the argument of the method is null, the method does nothing. The method removeRole eliminates the link between the receiver of the method and the instance of the class B passed as argument to the method. If no such link exists, or if the argument is null, the method has no effect. The method getRoleSet returns the set of instances of the class B that have a link with the receiver of the method. The value returned by this method is always an instance of a class that implements the java.util.Set interface, but it must satisfy some conditions that are specified below. Finally, the remaining three methods may be defined by the expressions to which they are equivalent:

// the expression obj.getRoleCount() // is equivalent to the expression obj.getRoleSet().size() // the expression obj.hasAnyRole() // is equivalent to the expression (! obj.getRoleSet().isEmpty()) // the expression

5.6 Associations

obj.hasRole(b) // is equivalent to the expression obj.getRoleSet().contains(b)

5.6.2.3

Bidirectional associations

When a new link is created for a bidirectional association, that new link must become traversable in both directions, rather than in only one, regardless of how that link is created. Likewise for the removal of a link. A link is an instance of an association between two objects—that is, it represents that the two objects are related by the relationship that the association represents. A link, however, is typically implemented as two separate references: Each of the objects in the link keeps a reference to the other. Yet, when a link is created or eliminated both references must be updated to reflect the change. Thus, in a bidirectional association, we should be able to create the same link by calling either one method for the object in one end of the link, or calling an equivalent method for the object in the other end. For instance, if we merge the two declarations for the association RelAB (see Figure 5.4 on page 117 and Figure 5.5 on the facing page), the execution of the code a.addRole(b) should be equivalent to the execution of the code

b.setRole(a). As a final example, consider that we have the following association declaration

association Rel { A playsRole a { multiplicity 0..1; } B playsRole b { multiplicity 0..1; } }

Moreover, consider that we have two instances of the class A, a1 and a2, and two instances of the class B, b1 and b2. Between a1 and b1 there is a link, and another link exists between a2 and b2. Thus, a1.getB() returns b1, b1.getA() returns a1,

a2.getB() returns b2, and b2.getA() returns a2. Consider now that we execute either a1.setB(b2), or the equivalent b2.setA(a1). Either of the calls creates a new link between a1 and b2, but, as the multiplicity upperbound of both roles is one, the previous links must be eliminated. Thus, after the execution of either of the calls, we have that a1.getB() returns b2, b1.getA() returns

null, a2.getB() returns null, and b2.getA() returns a1.

119

120

Domain Modeling Language

5.6.2.4

Sets returned by the method getRoleSet

The specification for the method getRoleSet, presented above, does not specify the exact nature of the value returned by the method—for example, whether the value returned is an immutable set or a mutable set. Yet, it is important to specify these details, so that programmers know what they can and cannot do with the result returned. In the DML language, the value returned by a call to the method getRoleSet is a mutable set that is backed up by the association and the receiver of the method call. This means not only that programmers may add and remove elements from the set, but also that those changes correspond, in fact, to the creation and elimination of association links. Furthermore, if a new link is added to or removed from the object that backs up the set, the set reflects that change. The advantage of having this specification is that we may use all the methods available in the java.util.Set interface to manipulate the links of an object. For instance, we may remove all the links of an object, o, with the code o.getRoleSet().clear(). Or, probably more useful, we may iterate over the set and remove the elements that satisfy some condition. Finally, a consequence of this specification is that the method call o1.addRole(o2) is equivalent to o1.getRoleSet().add(o2). Likewise for the method that removes a link. Thus, the method getRoleSet is sufficient to implement a role declaration with a multiplicity upper-bound greater than one.

5.6.2.5

Association objects and their listeners

Having different ways to create and to remove a link makes the domain model more programmer-friendly, because programmers may choose what is more convenient for them in each situation. Yet, all these equivalent approaches make the domain model more complex, and, eventually, more confusing, also. In particular, having various entry points for the same functionality raises the question of which method should we override if we want to customize that functionality. Imagine that we want to execute some code whenever a new link for the association

RelAB is created between an instance of the class A and an instance of the class B. Should we override the method setRole in the class B? That would work for the links created by that method. But a link may be created by calling the method addRole in the class A, also, and we do not know whether that method calls the method setRole or not. Yet, if we override the method addRole instead, we have similar problems. Even if we override both methods, we are not sure to catch all the link creations because links may be created when we add objects to a set returned by the method getRoleSet.

5.6 Associations

interface Association { void add(C1 o1, C2 o2); void remove(C1 o1, C2 o2); Association getInverse(); void addListener(AssocListener listener); void removeListener(AssocListener listener); } interface AssocListener { void beforeAdd(Association assoc, C1 o1, C2 o2); void afterAdd(Association assoc, C1 o1, C2 o2); void beforeRemove(Association assoc, C1 o1, C2 o2); void afterRemove(Association assoc, C1 o1, C2 o2); } Listing 5.8: The Java generic interfaces Association and AssocListener. The DML compiler uses objects of the Association type to implement an association in Java. Programmers may customize the association operations by adding listeners to an association object.

To solve this problem, I propose to add yet another way to create and to remove links from an association: A common point in the code that must be executed whenever a link is created or removed, regardless of how that is done. The key idea is to have an object that represents an association. The basic operations of that object are a method to add a new pair of objects to the association, and a method to remove a pair of objects from the association. The first method creates a new link, the second removes an existing link. These methods must be called to create or to remove a link. In fact, all the methods described previously that create or remove links may call the add or the remove operations of this new object. The semantics must be the same. The association object gives us a central point of execution for the operations that change an association. Now, we need some mechanism to specialize those operations. Because each association is represented by an object, rather than by a class, we cannot use standard class inheritance and method overriding to do that specialization. Instead, we may use listeners to do it. In Listing 5.8, I show both the interface Association and the interface AssocListener that will allow us to specialize the creation and the removal of a link. The method addListener adds a new listener to an association object, whereas the method removeListener removes a listener. Even though these methods may be used at runtime to change the set of listeners for an association object dynamically, the common usage is to add one or more listeners to an association object at class-load-time and use that set of listeners throughout the entire program. When an association object has some listeners registered, the methods add and

121

122

Domain Modeling Language

association Rel { A playsRole a { multiplicity 0..1; }

abstract class AState { static Association Rel; ... } DML Compiler

B playsRole b { multiplicity 0..*; } }

abstract class BState { static Association Rel; ... }

Figure 5.6:

Static fields that a DML compiler must generate to hold the Association objects. The name of the static field in each class is the name of the association, as given in the association declaration. The association object in each class is the inverse of the association object in the other class.

remove call the appropriate methods of the registered listeners, following the same order by which the listeners were added to the association object. The method add first calls the method beforeAdd of each listener, then creates the link, and, finally, calls the method afterAdd of each listener. Mutatis mutandis for the method remove. Note that a listener may cancel the creation (or the removal) of a link, if its beforeAdd (or

beforeRemove) method throws an exception. Using custom association listeners, it is easy to specialize an association. But, before we can do that, we need to know how to access the objects that represent associations in a domain model. My proposal is to make them accessible through static fields in the classes that participate in the association, as depicted in Figure 5.6. With this final piece in place, we may now specialize the creation of a link, as intended. In Listing 5.9 on the next page, I show a sketch of the code that we may add to the behavior class A to accomplish that.

5.7

Implementation of a Domain Specification

The specification of the DML language prescribes in detail the interface of the classes that a conforming DML compiler must generate; it does not, however, dictate how that interface should be implemented, even though it gives, now and then, some hints about possible implementation strategies. Therefore, different compilers for the DML language may generate different source code to implement a domain specification: Either because they must generate domain models with different properties, or simply because they use different implementation strategies. In fact, the simplicity of the DML language is a conscious design decision to promote experimentation with different implementation strategies, which may affect significantly

5.7 Implementation of a Domain Specification

class A extends AState { static class RelListener implements AssocListener { void beforeAdd(A a, B b) { // do something } // ... implementation of remaining methods } static { // specialize the Rel association Rel.addListener(new RelListener()); } } Listing 5.9: Specialization of an association using an AssocListener. An inner class is used to specify the listener. Then, a static initializer registers a listener with the association object that is stored in the static field Rel of the superclass AState.

the performance and the memory footprint of an application. Given the reduced number of constructs in the language, creating a new code generator backend for a DML compiler should be well within the reaches of any software development team. In this section, I give an outline of one of the code-generator backends that I implemented for the DML. I will not describe how I implemented the compiler or the codegenerator.

Rather, I describe which code is generated by the backend for a domain

specification. The backend that I describe here corresponds roughly to the one used in the FénixEDU project [FenixEDU], which we shall see in Chapter 7. The code that this backend generates tries to accomplish a good equilibrium between two different requirements for the generated code: efficiency, and readability. Efficiency, because it is used in a large project with many entities, where efficiency was a concern. Readability, because programmers may need to read the code generated by the DML compiler when they are debugging the application. Even though many other strategies exist for implementing a domain specification, I believe that the implementation that I outline here provides a good starting base for many projects.

5.7.1

Using the JVSTM to Make a Transactional Domain

Throughout this dissertation, I propose to implement domain models that exhibit transactional properties. So, it comes out naturally that the implementation that I describe here generates a transactional domain, by leveraging on the JVSTM—the Java implementation

123

124

Domain Modeling Language

of the STM model that I propose in Section 4.3. By using the JVSTM to implement the domain specification, we obtain a domain model that is transactional, a domain model that has all its state stored in transactional memory. We are, therefore, able to use atomic actions to implement some of the operations on the domain model. Those operations may be methods implemented in the behavior classes, or more coarse-grained operations that are defined in the layers above the domain model. In fact, because the JVSTM supports nested transactions, we may have atomic actions at both levels: We may use atomic actions to implement some of the behavior methods, which may then be called inside more coarse-grained atomic actions. Note that, to implement the behavior classes of a domain model, programmers need to know which methods exist in the state classes from which the behavior classes inherit. Also, they need to know what those methods do. But they should not need to know the implementation details of the state classes. Yet, programmers need to know whether those classes are transactional or not, if we want that they take advantage of the transactional properties of the domain model. I argue that, to build rich domain models, it is indispensable that the domain model be transactional. So, the intended semantics for a DML domain specification is that it specifies a transactional domain model, even though the language may have more relaxed implementations in some cases. It turns out, however, that the implementation of a domain specification as a transactional domain model is quite simple, if we are using the JVSTM to do it.

5.7.2

Implementing Entity Types

The DML specification already prescribes that an entity type must compile to a pair of classes—the state class and the behavior class—with a getter and a setter for each of the entity type’s attributes. What is missing, then, is how to implement those methods. The usual approach used to implement the attributes of an entity is to use one field for each attribute in the class implementing that entity type. Each field holds the value of an attribute. To make the class transactional, however, we need to wrap each value with a VBox, so that a change in an attribute may become part of an atomic action. For instance, in Figure 5.7 on the facing page, I show the implementation of the first entity type declaration from the example given in Listing 5.5 on page 108. The methods generated for an entity type declaration do not need the annotation

Atomic. They only read or write the box that corresponds to an attribute, so they are atomic already.

5.7 Implementation of a Domain Specification

public class ClientState { private VBox name;

class Client { String name; }

String getName() { return this.name.get(); }

DML Compiler

void setName(String name) { this.name.put(name); } } Figure 5.7: JVSTM-based implementation of the class ClientState. This class is the state class that results from the compilation of the first entity type declaration shown in Listing 5.5 on page 108, the declaration of the entity type Client.

5.7.3

Implementing Associations

The straightforward implementation of an entity type is a consequence of the natural mapping between entity types and Java classes. Unfortunately, this ease of implementation does not extend to the implementation of associations. Therefore, the implementation of associations requires more work. To implement an association, we must generate a set of methods for each of the association’s roles. We need, also, to create an instance of a class that implements the interface

Association (which is shown in Listing 5.8) for each of the classes participating in the association; as prescribed by the DML specification, these Association instances are stored in static fields, and, so, there is not much variability here. Where we may have different implementation strategies is on the implementation of the

Association interface and on the implementation of the role’s methods, provided that the implementation chosen satisfies the requirements of the DML specification. Namely, that, regardless of which methods are used to create or remove links from an association, they invoke the appropriate methods in the association listeners that were previously added to the association’s instance. In fact, the DML specification prescribes a set of redundant ways for creating and removing links, but requires that all those redundant ways of doing things be semantically equivalent. The simplest way of ensuring this equivalence is to implement the creation (or removal) of a link in a single kernel method, and make all the remaining equivalent methods call that kernel method. The kernel method is responsible for creating (or removing) a link, ensuring that both sides of the link are updated consistently. In my implementation, the kernel methods are the methods add and remove from the interface Association.

125

126

Domain Modeling Language

association AccountOwnership { Client playsRole owner { multiplicity 1; } ClientAccount playsRole account { multiplicity 1..*; } } DML Compiler

public class ClientState { private AssocSet accountSet; ... } public class ClientAccountState { private VBox owner; ... } Figure 5.8: Fields used to store the links of an association. Each role generates a new field in the class of the opposite type to store the objects that are related with the object at that end of the link.

5.7.3.1

Storing the Association’s Links

When we create a link between two objects, we need to store the information that they are linked somewhere. Two common choices are: (1) making the link a first-class object and keeping a global set of links for each association, and (2) storing with each object the object or set of objects with which it relates. The first solution may be preferable in terms of memory consumption when relations are sparse, but incurs in a performance penalty when we need to know which objects are related to another object. The second solution is faster in this latter case, but may require more memory and makes other types of relation accounting more difficult to implement. Given the interface prescribed by the DML specification, I opted for the second solution. Thus, each role declaration will generate a new field in the role’s opposite class. The purpose of that field is to hold either a single object or a set of objects, depending on whether the role’s multiplicity upper-bound is one or greater than one, respectively. Additionally, to ensure the proper transactional semantics, the field must be wrapped with a VBox, if for a single object, or use a transactional set, otherwise. In Figure 5.8, I show an example where both types of fields are generated when an association declaration is compiled. The class AssocSet is a transactional set that ensures the semantics required by the DML specification for the sets returned for a role

5.7 Implementation of a Domain Specification

association AccountOwnership { Client playsRole owner { multiplicity 1; } ClientAccount playsRole account { ... } } DML Compiler

public class ClientAccountState { static Association AccountOwnership = ...; private VBox owner; Client getOwner() { return this.owner.get(); } void setOwner(Client owner) { AccountOwnership.add((ClientAccount)this, owner); } }

Figure 5.9: Implementation of the methods for a role with a multiplicity upper-bound of one. The setter simply calls the kernel method add from the Association’s instance.

with a multiplicity upper-bound greater than one. We shall see this class in more detail in Section 5.7.3.3.

5.7.3.2

Implementing the Role Methods

The set of methods to generate for each role depends on the multiplicity of the role. There are two cases: roles with a multiplicity upper-bound of one, and roles with a multiplicity upper-bound greater than one. In either case, however, there is a set of core methods and a set of optional methods. Here, I describe the implementation of the core methods only; the implementation of the optional methods results trivially from the core methods. In the first case—roles with a multiplicity upper-bound of one—we have only two methods: a getter and setter. To implement the getter, we just have to return the value of the VBox that is used to store the value. The setter, however, creates or removes a link and must, therefore, call the appropriate kernel method to accomplish that. I show in Figure 5.9 an example of the code generated in this case. In the second case—roles with a multiplicity upper-bound greater than one—the implementation is similar, except that now we have one more method to remove a link, whereas in the previous case the setter took care of that too. I show in Figure 5.10 an example of the code generated in this case.

127

128

Domain Modeling Language

association AccountOwnership { Client playsRole owner { ... } ClientAccount playsRole account { multiplicity 1..*; } } DML Compiler

public class ClientState { static Association AccountOwnership = ...; private AssocSet accountSet; Set getAccountSet() { return this.accountSet; } void addAccount(Account account) { AccountOwnership.add((Client)this, account); } void removeAccount(Account account) { AccountOwnership.remove((Client)this, account); } }

Figure 5.10: Implementation of the methods for a role with a multiplicity upperbound greater than one. The methods addAccount and removeAccount simply call the kernel methods add and remove from the Association’s instance.

5.7.3.3

Implementing the Association-Aware Set

According to the DML semantics, the set returned by the method getAccountSet, shown in Figure 5.10, must be backed up by the association and the instance of the class Client on which it was called. That is, changes in the set should give rise to changes into the association’s links and vice-versa. The class AssocSet, which I outline in Listing 5.10, implements this semantics. We create an instance of AssocSet, by passing it an object corresponding to an end of a link and an instance of an Association. The set stores these objects internally, and when an element is added to or removed from the set, it calls the kernel methods add or remove of the association object with that element and the stored end of the link. The association object, however, must be able to add and to remove elements of the set, as part of the creation or removal of a link. Yet, calling again the method add (or

remove) of the set would lead to an infinite loop. Thus, the class AssocSet provides an alternative interface for adding and removing elements: the methods justAdd and

justRemove. These methods effectively add or remove the element from the underlying set: an instance of the class VSet, which is a transactional set provided by the JVSTM. Obviously, these low-level methods are meant to be used only by the association class.

5.7 Implementation of a Domain Specification

public class AssocSet extends AbstractSet { private Set set = new VSet(); private E1 linkEnd; private Association assoc; public AssocSet(E1 linkEnd, Association assoc) { this.linkEnd = linkEnd; this.assoc = assoc; } void justAdd(E2 elem) { set.add(elem); } void justRemove(E2 elem) { set.remove(elem); } public int size() { return set.size(); } public boolean contains(Object o) { return set.contains(o); } public boolean add(E2 o) { if (set.contains(o)) { return false; } else { assoc.add(linkEnd, o); return true; } } public boolean remove(Object o) { if (set.contains(o)) { assoc.remove(linkEnd, (E2)o); return true; } else { return false; } } ... } Listing 5.10: Implementation of the association-aware class AssocSet.

129

130

Domain Modeling Language

5.7.3.4

Implementing the Association Class

All the methods shown above delegate into the association object the responsibility of actually creating and removing links, in its add and remove methods, respectively. To create or remove a link, these methods must perform several operations. For instance, to create a link for a bidirectional association, we have to update both of the objects that we want to link. Furthermore, in some cases, the creation of a link may result in the elimination of previous links. The exact sequence of operations depends on the multiplicities at both ends of the association. So, we need to distinguish, at least, the three following cases: one-to-one, one-to-many, and many-to-many associations. In fact, given that we need to update each of the two objects of a link so that they point to the other object, these three cases result from the composition of only two distinct ways of updating an end of a link: when we have a single object, and when we have a set of objects. Therefore, we may implement an association class in an entirely generic way, as shown in Listing 5.11. The DirectAssociation class delegates the work of updating the ends of a link to objects of the Role type. Creating different instances of this class, with appropriate instances of the Role type, allows us to implement the various types of associations. Before we look at the Role type, note the use of the annotation Atomic to ensure that all the operations affecting a link are executed atomically. With this implementation, the code of each listener executes within the atomic action, also. So, if some listener throws an exception, either before or after creating or removing a link, the entire operation is aborted. This gives us an increased expressive power to develop the domain model. Finally, for completeness, I show in Listing 5.12 an outline of the implementation of the class InverseAssociation, which is used in the implementation of the class

DirectAssociation.

5.7.3.5

Implementing Different Role Types

The final piece to conclude the implementation of an association declaration is the implementation of the classes that update each of the association’s ends. These classes must implement the interface Role, shown in Listing 5.13. An association object calls the method add to create a new link between o1 and o2. The responsibility of the method add of the Role interface is to change the object o1, only; that is, this method performs only half of the work. The assoc argument is the association object that made the call. This object may be needed to remove an existing

5.7 Implementation of a Domain Specification

class DirectAssociation implements Association { private Association inverse = new InverseAssociation(this); private List listeners = ...; private Role firstRole; private Role secondRole; DirectAssociation(Role first, Role second) { this.firstRole = first; this.secondRole = second; } @Atomic void add(C1 o1, C2 o2) { for (AssocListener l : listeners) { l.beforeAdd(this, o1, o2); } firstRole.add(o1, o2, this); secondRole.add(o2, o1, inverse); for (AssocListener l : listeners) { l.afterAdd(this, o1, o2); } } @Atomic void remove(C1 o1, C2 o2) { for (AssocListener l : listeners) { l.beforeRemove(this, o1, o2); } firstRole.remove(o1, o2); secondRole.remove(o2, o1); for (AssocListener l : listeners) { l.afterRemove(this, o1, o2); } } Association getInverse() { return inverse; } ... } Listing 5.11: Implementation of the generic class DirectAssociation.

131

132

Domain Modeling Language

class InverseAssociation implements Association { private Association inverse; InverseAssociation(Association inverse) { this.inverse = inverse; } void add(C1 o1, C2 o2) { inverse.add(o2, o1); } void remove(C1 o1, C2 o2) { inverse.remove(o2, o1); } Association getInverse() { return inverse; } ... } Listing 5.12: Implementation of the class InverseAssociation.

interface Role { void add(C1 o1, C2 o2, Association assoc); void remove(C1 o1, C2 o2); } Listing 5.13:

The generic interface Role. This interface is used by the DirectAssociation class to update each of the association’s objects in a link.

5.7 Implementation of a Domain Specification

abstract class RoleOne implements Role { void add(C1 o1, C2 o2, Association assoc) { if (o1 != null) { VBox o1Box = getBox(o1); C2 old2 = o1Box.get(); if (o2 != old2) { assoc.remove(o1, old2); o1Box.put(o2); } } } void remove(C1 o1, C2 o2) { if (o1 != null) { getBox(o1).put(null); } } abstract VBox getBox(C1 o1); } Listing 5.14: The implementation of the class RoleOne.

link for the object o1, in the case when it cannot have more than one link. The case for the method remove is similar, except that this method does not need the third argument. We need two distinct implementations of this interface: one for when we have a role with a multiplicity upper-bound of one, another for when the multiplicity upper-bound is greater than one. The first class is called RoleOne, and is shown in Listing 5.14. The second class is called RoleMany, and is shown in Listing 5.15. These classes are abstract because, to perform the changes, they need to access either the VBox or the AssocSet that holds the value that should be changed. The box or set to access, however, is different from one association to another, which means that we need a different subclass of either RoleOne or RoleMany for each role declaration. These subclasses simply override the methods getBox or getSet to return the appropriate object, given an object o1. These latter subclasses are generated as anonymous inner classes for each role declaration. Note that, unlike these, none of the classes AssocSet, DirectAssociation,

InverseAssociation, RoleOne, and RoleMany are generated by the DML compiler. Rather, they are part of a runtime library provided by the DML compiler to facilitate the implementation of an association declaration.

133

134

Domain Modeling Language

abstract class RoleMany implements Role { void add(C1 o1, C2 o2, Association assoc) { if ((o1 != null) && (o2 != null)) { getSet(o1).justAdd(o2); } } void remove(C1 o1, C2 o2) { if ((o1 != null) && (o2 != null)) { getSet(o1).justRemove(o2); } } abstract AssocSet getSet(C1 o1); } Listing 5.15: The implementation of the class RoleMany.

5.7.3.6

Enforcing Multiplicity Constraints

In the implementation given above I have not addressed one question: How to guarantee that the multiplicities of an association are not violated? I have not addressed this problem yet, because it is not possible to implement this requirement adequately until I introduce the consistent predicates in the next chapter. I shall return to this topic again in Section 6.5.

5.8

Related Work

To the extent of my knowledge, the approach that I propose in this chapter is novel, even though there is plenty of work that addresses the same problem. I propose a new language specifically designed for implementing a domain model’s structure and to complement another general-purpose programming language. The key idea is to allow the implementation of a domain model’s structure in a language that is midway between the languages used to represent a domain model and the languages used to implement them. This language, however, is fully operational, allowing a complete implementation of a domain model’s structure via automated code generation. In contrast with this approach, all the previous work on the subject of reducing the gap between a domain model and its implementation fits into one of the following approaches:

• Support associations directly as a first-class construct in the programming language, which is then used to implement both the structure and the behavior of a domain model.

5.8 Related Work

• Facilitate the implementation of associations in a programming language by providing libraries and patterns that capture the best-practices of implementing associations in that language. • Automate the generation of the code that implements part of a domain model, starting with the domain model expressed in a modeling language such as UML.

The last approach is the one that more closely relates to the work in this dissertation. It is, however, worthwhile to discuss all of these approaches, given that all of them are trying to solve the same problem.

5.8.1

Associations as First-Class Language Constructs

Proposals to add to object-oriented programming languages associations (or relationships) as a first-class language construct date back to twenty years ago, to the work of Rumbaugh. First in [Rumbaugh, 1987] and later in [Shah, Hamel, Borsari, and Rumbaugh, 1989], Rumbaugh and colleagues argued for the inclusion of relations as a core construct of object-oriented programming languages, on par with the constructs already available for classes. In that work they describe an object-oriented programming language called DSM, which extended the C language with constructs for objects and relationships, among other things. In the DSM language, relations are declared with a top-level construct that gives a name to the relation and identifies the classes that participate in the relation, associating a role name and a cardinality with each participating class. Thus, apart from the syntax, association declarations in the DML are very much alike relations in the DSM. Moreover, declaring a relation between two classes in DSM generated access methods in each of the participating classes, named after the participants’ roles, that allowed the navigation from one object to those with which it related to. This approach to relationship implementation is still the current practice in the object-oriented programming world. In this area, the DML followed the common practice in the area to provide a language that is familiar to the programming community. Besides Rumbaugh, several other researchers proposed to add explicit relationship constructs to their languages. For instance, Albano, Ghelli, and Orsini [1991] criticize the usual approach of implementing associations between objects by adding properties to each of the participants in the association. Instead, they propose a language for an object-oriented database which offers constructs for representing explicitly associations, as well as for establishing constraints on those associations. More recently, in the context of the Java programming language, Bierman and Wren [2005] proposed the language RelJ, which includes relationships as a first-class language

135

136

Domain Modeling Language

construct. RelJ is an extension of a small subset of the Java programming language, so that the authors could formalize the language and prove certain safety properties on the relationships’ types. In RelJ, not only relationships are first-class entities, but also their instances are. Also, relationships may have fields and methods, may extend other relationships, and may participate in other relationships. The various approaches to support relationships at the programming language level vary considerably in the details. Whereas some (as the DSM of Rumbaugh) support arbitrary n-ary relations, others support binary relationships only (RelJ, for instance). Also, some support qualified relations, whereas others do not. Studying and discussing all those variations, however, is beyond the scope of this dissertation.

5.8.2

Patterns for Implementing Associations

Even though, theoretically, adding support for relationships to the programming language is the best approach, practically, these proposals are of little help if they never reach mainstream programming languages. Another, more practical route, is to stay within the limits of current programming languages, and see how we can implement associations more easily. I call this approach the patterns approach, even if not all the proposals may qualify as patterns (in the sense of [Gamma et al., 1995]). In [Noble, 2000], Noble proposes a set of basic relationship patterns for an objectoriented programming language. He describes several patterns to implement relationships, from the simplest Relationship As Attribute, which implements a small, simple, one-to-one unidirectional relationship, to the more complex Mutual Friends, which implements a bidirectional relationship. The patterns described by Noble capture the common practices of implementing relationships in object-oriented programming languages. In fact, the examples given in Section 3.3.2 follow some of these patterns. Likewise, either these patterns or some of their variations are typically used to implement the higher level constructs provided by other approaches. For instance, in the implementation of a DML domain specification I use some of these patterns, as do other authors. One of the problems of using most of the patterns described by Noble is that we lose track of a relationship after it is implemented: Implementing a relationship, spreads the code among several different classes, making it difficult to recover the relationship again later [Guéhéneuc and Albin-Amiot, 2004; Gueheneuc and Albin-Amiot, 2003]. The pattern that resists better to this problem is the Relationship Object (or one of its specializations, the Active Value, or the Collection Object), which uses a distinct object to

5.8 Related Work

137

represent the relationship, thereby encapsulating all the code in a single place. Several authors have proposed solutions that are, in their essence, variations of this pattern—for instance, [Noble and Grundy, 1995; Suscheck and Sandén, 2003]. A more recent approach by Pearce and Noble [2006] uses aspect-oriented programming (see [Kiczales, Lamping, Menhdhekar, Maeda, Lopes, Loingtier, and Irwin, 1997]) to implement relationships as a crosscutting concern. By leveraging on the facilities provided by the Aspect/J language, the authors propose a Relationship Aspect Library that allows programmers to represent relationships explicitly, separate from their participating classes. Although promising, this approach has still several shortcomings. For instance, to have more than one relationship for a given class, the authors resorted to the workaround of having several almost identical copies of the aspects that implement a relationship. Obviously, this solution does not scale for any medium-sized application. Moreover, the syntax for adding links to a relationship departs significantly from the common practice on the object-oriented programming area. For instance, using the example given by Pearce and Noble in their paper, programmers must write code such as Attends.aspectOf().add(student, course) to create a new relationship link between two objects, instead of writing the typical course.enroll(student) method call (unless, of course, they write by hand the enroll method to call this code, which defeats the purpose of the proposal). This last issue is, in fact, a problem shared with many other approaches that make the relationship a first-class object in the language. For instance, both the DSM language and the RelJ language use this same approach to change a relationship. The DML language, on the other hand, provides both ways of adding links to relationships. We may, thus, use either the code Course.Attends.add(course, student) to add a new link to the

Attends association, or use the more standard call course.addStudent(student), instead.

5.8.3

Generating the Code for Associations

Given that we have well-defined patterns for implementing an association, rather than applying those patterns ourselves, we may automate their implementation instead. In fact, this approach has been increasing steadily in popularity.

The advances in the

areas of generative programming [Czarnecki and Eisenecker, 2000] and model-driven development [Selic, 2003] certainly contributed to this increase. There is a wealth of tools, both free and commercial, that generate automatically code for fragments of the UML language.3 With code generation, however, comes the problem of avoiding round-tripping, which many tools fail to address. 3

For a comparison of 10 such tools, see [Akehurst, Howells, and McDonald-Maier, 2007].

138

Domain Modeling Language

Avoiding round-tripping is one of the ten objectives set up by Harrison et al. [2000], in their work on mapping high-level UML designs to Java. Harrison and his colleagues propose a new method for generating Java code from a high-level design model expressed in a UML class diagram. To avoid round-tripping problems, they split the implementation of a class into two classes plus an interface, so that one of the classes contains the code generated and the other is for the programmer make changes. The semantics of entity type declarations in the DML language borrow from this, even though it uses a simpler (and better) approach as it dispenses the interface. A significant part of [Harrison et al., 2000] is dedicated to the implementation of associations. Unlike in the DML, which strives for using common programming idioms, in this work the authors propose the use of cursors to manipulate an association. Cursors are similar in spirit to the association-aware set described in the semantics of associations declarations (see Section 5.6.2.4). Yet, whereas the association-aware set of DML implements the Java’s standard Set interface, cursors provide an entirely new and rather more limited interface. Moreover, the only way to change an association is through a cursor, which, again, clashes with the common practice in the area. Génova et al. [2003], on the other hand, propose also a mapping for a fragment of the UML class diagrams, but instead of cursors, they follow a more traditional approach of adding methods to the classes in each end of an association. Yet, it is not clear from their paper how they distinguish among different associations for the same class. Also, they describe the interface of the code generated for an association, but do not specify the semantics precisely; for instance, they do not say whether the collection returned by the getter of an association end is mutable or not. No implementation is described either. Finally, the most complete treatment of the UML language that I am aware of was published recently by Akehurst et al. [2007]. In this work, Akehurst and his colleagues describe thoroughly code generation patterns for each one of the possible variations on a UML 2.0 association. They stumbled upon some difficulties, however. For instance, how to enforce the minimum multiplicities of a bidirectional association, for which they have no good answer. As we shall see in Section 6.5, the DML compiler will be able to solve this problem. To conclude this discussion of related work, I point out that in none of these proposals is it clear whether the programmer may customize the behavior of the code generated for an association.

5.9

Summary

This chapter describes a new language—the DML language—that purports to reduce the gap that currently exists between a domain model and that domain model’s implementa-

5.9 Summary

139

tion. Specifically, in what regards the implementation of a domain model’s structure. Unlike current object-oriented programming languages, the DML language provides constructs for representing both entities and associations, thereby allowing a more direct implementation of a domain model that uses a modeling language with similar constructs—for instance, UML. The syntax and the semantics of the DML language are described thoroughly, giving, thus, a precise specification to the language and making it fully operational. Within its scope, the DML language is arguably a higher-level language than the currently available object-oriented programming languages. The DML, however, is not a general-purpose programming language, given that it allows only the representation of a domain model’s structure. In fact, it was designed to integrate seamlessly with the Java programming language; this design decision is visible both in the syntax and in the semantics of the language. The integration of DML with the Java language is accomplished through a process of code-generation that transforms a DML’s domain specification into Java code. Some of the requirements for the generated code are embedded into the DML semantics, but the exact code to generate is not part of the language specification, thereby allowing different strategies for implementing a domain model. Nevertheless, this chapter describes how to transform a domain specification into a transactional domain model implementation, by describing which Java code is generated for each of the language’s constructs.

This implementation leverages on the JVSTM

described in the previous chapter, and is sufficiently simple and readable to constitute a good design pattern for implementing associations in Java, even when the DML is not used. Finally, the DML language, when compared to the alternatives, provides some distinctive characteristics:

• The code generated from a domain specification has a programmer-friendly interface that adheres to the current practices used in object-oriented programming. • Even though the implementation of an association is entirely generated, the programmers may still customize the behavior of the generated code without hampering the round-tripping of the system. • The implementation of an association integrates well with the existing Java Collections Library, which gives to the programmers the freedom to use the interfaces that they are already used to. • By implementing a domain model in a DML domain specification, programmers

140

Domain Modeling Language

create a valuable intermediate artifact with a precise semantics that may be used for other purposes in the application. • Given its similarity with Java and its simplicity, the DML language is easy to learn and to use by current Java programmers.

Chapter 6

Consistency Predicates In the previous chapter, I proposed to separate the implementation of a rich domain model into the implementation of the domain structure and the implementation of the domain behavior. But, whereas to implement the structure of a domain model I proposed a new language, to implement the domain’s behavior I propose that programmers continue to use Java; the DML language proposed in the previous chapter was specially designed to allow this separation. Java is a widely used general purpose object-oriented programming language, with a reasonable set of programming constructs, and, more importantly, with an immense set of libraries, tools, and documentation that greatly simplify the task of developing new applications. Therefore, by using Java as an implementation language, we may take advantage of all the facilities available for it. Unfortunately, the implementation of a rich domain model in Java is not exempt of problems, as the examples discussed in Chapter 3 illustrate. One of the difficulties found in the implementation of a domain model was in implementing domain constraints, as the example of implementing the limit imposed on the total balance of a bank’s client that was discussed in Section 3.4.2. Yet, even though this is a single example, the problems that we found in that example are not uncommon. Quite on the contrary, that example is representative of a frequent problem in the implementation of rich domain models: How to ensure that the domain constraints are maintained during the execution of the program? The common approach to this problem is to implement the methods that may change the state of domain objects with great care, to ensure that the domain remains in a consistent state.

In this chapter, I propose a new approach—the use of consistency

predicates. Consistency predicates are a powerful construct that allows programmers to express the conditions for domain consistency separately from the methods that change the state of domain objects. Then, at commit time, these consistency predicates are used

142

Consistency Predicates

to validate the execution of atomic actions. By leveraging on the mechanisms needed to support STMs in a programming language, I show that consistency predicates may be introduced in Java with little additional effort. In the following, I start with a discussion on the difficulty of ensuring the consistency of a domain model. Then I introduce the notion of consistency predicates and show how they may be used to simplify the implementation of domain consistency. Finally, I describe how consistency predicates are implemented in the JVSTM.

6.1

Domain Consistency

One of the basic rules of good object-oriented programming, taught in most books on the subject, is that each object is responsible for maintaining the consistency of its own state. To enforce this rule, the state of an object should not be accessible to other objects. Rather than changing an object’s state directly, other objects call the methods provided by that object to perform the changes they need. Each of the methods provided by an object, in turn, must ensure that it leaves the object in a consistent state after its execution. Making an object the sole responsible for its own state is a good design rule because it limits the places in the code that need to have knowledge about the constraints for that object, independently of the context where the object is used. Therefore, by following this design rule, we increase the modularity of an object-oriented program. Yet, whereas following this rule may be possible for isolated objects—which do not have relationships with other objects or that have relationships only with owned objects—it is not possible, or at least convenient, in general, for rich domain models. Before I explain the reasons why rich domain models are special in this regard I discuss first the case of maintaining the consistency of a single object’s state.

6.1.1

Consistency of Single Objects

Consider the typical example of a buffer with a maximum capacity. A common implementation for such a buffer object is to maintain information in the object about its maximum capacity, the number of elements that it contains, and the contained elements. Given this implementation, the constraints that the state of every buffer object must verify are: (1) that the number of elements in the buffer is between zero and the buffer’s maximum capacity, inclusive; and (2) that the element count is equal to the number of elements actually contained in the buffer. Note that the constraints for a buffer object depend only on the buffer object’s state. Thus, we can be sure that these constraints remain valid throughout the entire lifetime

6.1 Domain Consistency

of a buffer object, if we enforce two things. First, that whenever a new buffer object is created, it is created in a consistent state. For instance, a constructor for a buffer object may receive a non-negative integer value as argument and return a new buffer object with that value as its maximum capacity, with an element count of zero, and with no elements in it. A buffer object with such a state is consistent, as it satisfies all the constraints stated above. Second, that each of the methods that changes the state of a buffer object leaves the object in a consistent state after the method’s execution, provided, of course, that the object was in a consistent state before the method was called. An example of a method that may change a buffer object is the method to add an element to the buffer. This method should add the element to the buffer and increment the element count by one. Failing to do one of the two things would leave the buffer object in an inconsistent state and is a programming error. Yet, sometimes doing both things leaves the object in an inconsistent state, also: When the number of elements in the buffer is equal to the buffer’s maximum capacity, adding another element violates the first constraint presented above. So, the method to add an element to a buffer should check for this case and return to the caller an indication that the operation could not be performed if the buffer is full. Because in Java it is usual to throw an exception in such cases to indicate a failure, in the following I shall assume that a method throws an exception whenever it cannot perform the requested operation. This simple example illustrates a common pattern for the methods that change the state of an object. These methods must check whether they may perform the operation requested, throwing an exception if not. Otherwise, they perform the operation, updating the state of the object such that the object remains in a consistent state in the end. Nevertheless, even though the object must be consistent in the end of an operation, it may be temporarily inconsistent during the execution of the operation. For example, the method to add an element to the buffer, either increments first the element count and then adds the element to the buffer, or it does these two things in the reverse order. In either case, the state of the object becomes inconsistent between the two operations. That inconsistency, however, must be eliminated before the method returns to its caller.1 Finally, note that not all of the object’s methods need to leave the object in a consistent state. Only the public methods—that is, methods that other objects may call—need to ensure that. The internal methods of the object, which may be used only as helper functions by the remaining methods of the object, do not need to satisfy this requirement.

1 The problem, of course, is worse when we have concurrent accesses to the object that may view that intermediate state. Possible solutions to this problem include using locks or atomic actions, and were already discussed in Section 3.1 and in Chapter 4—my preference, obviously, going to the use of atomic actions.

143

144

Consistency Predicates

6.1.2

Consistency of Rich Domain Models

A distinctive aspect of rich domain models is that many of the constraints for a rich domain model involve the state of more than one entity, rather than the state of a single entity as in the example of the buffer object presented above. For instance, the constraint implied by a bidirectional association between two different entity types, A and B, is a trivial (and common) example of a constraint that involves more than one entity: If an instance of A has a reference to an instance of B, then that instance of B must have a reference to that same instance of A. Moreover, depending on the multiplicities of the association, there may be further constraints for the relationship between instances of both types. The problem of having constraints that depend on the state of more than one object, however, is that it is no longer easy to localize the implementation of those constraints in one single object. If a constraint refers to the state of more than one object, then we must ensure that, whenever a change occurs in the state of any of those objects, the new state after the change still satisfies the constraint. Unfortunately, the approach of checking the constraint at each method that may affect the constraint’s validity brings with it several problems. First, it means that, to implement the constraints for a rich domain model, programmers must reason about several classes simultaneously, rather than concentrating on only one. One of the advantages of object-oriented programming is that it reduces the complexity of the programming task by allowing programmers to concentrate on the implementation of each class in turn, ignoring the implementation details of other classes. But, when the implementation of a constraint for a class depends on the state of objects from other classes, programmers must take into consideration the implementation of all those other classes, also. In particular, they may have to change the other classes’ methods to ensure that the constraint is not violated. Yet, having several methods to worry about is error-prone, because programmers may easily forget about one of those methods, thereby opening the door to domain inconsistencies. Often, such errors are difficult to detect, either because the constraint is not explicitly stated anywhere in the code, or simply because the constraint is so complex that it is not clear which are the objects that it depends on. Furthermore, the probability of occurring such a programming error increases when the code needs to be changed as a result of some requirements change. Second, this approach leads rapidly to modularity problems such as code scattering, code tangling, and strong coupling among the domain classes. Code scattering occurs because the implementation of each constraint is spread over several methods of possibly different classes. Code tangling, on the other hand, occurs when different constraints refer to the state of the same entity type—in this case, the methods of the shared entity type must have code to check each of the constraints, thereby mixing several concerns in

6.2 Examples of Constraints

a single method. The strong coupling among the domain classes occurs because each of the entities referred to by a constraint needs to be aware of the other entities, so that it can check the constraint when its state changes. A possible solution to these modularity problems is the systematic use of the observer pattern [Gamma et al., 1995] to register dependencies between entities. Yet, that solution complicates the domain implementation enormously. Thus, in this dissertation I strive for a better solution. Finally, there is a more fundamental problem—a problem of composition—with the approach of checking the constraints for a rich domain model at each of the domain entity types’ methods: In many cases, those methods are used as part of larger operations, and, thus, they may need to break the domain’s constraints temporarily during the operation. Consider, for example, the case of establishing a new bidirectional link between two objects. After one of the objects is changed to point to the other, the domain is not in a consistent state until the latter object is changed to point to the former object. Therefore, the method that changes the state of the former object cannot check whether the constraint affected by that change is violated or not; it must assume that the method that calls it ensures the final consistency of the domain. In fact, these methods are like the private methods of the single object case, which may break the constraints for an object. But, whereas in the case of single objects, the callers of the private methods are necessarily from the same class, in the case of a rich domain model, the callers of the methods that are part of larger operations must be, often, from other classes. So, it is much harder to ensure that all such callers ensure the domain consistency, because they are not confined to a single class; in fact, in many cases, the callers are not limited in any way. Moreover, to ensure that, in the end, they leave the domain in a consistent state, the higher-level operations that call the lower-level operations must have knowledge about the details of the objects they affect, thereby breaking the object modularity. Therefore, the composition of two or more objects into new collaborations is difficult with current object-oriented practices; specially when the constraints of the composed objects may need to be violated temporarily during the execution of a composing operation.

6.2

Examples of Constraints

The problem associated with the difficulty of composing domain objects was already discussed in Section 3.4.2. To exemplify the remaining problems with concrete examples, I shall discuss, in the following, the implementation, using a traditional approach, of several constraints taken from the same banking domain which was introduced in Section 2.2.2. Each of these examples illustrates some of the problems with the traditional approach of implementing constraints in a rich domain model. Then, in the next section, I present the consistency predicates, which allow a much simpler implementation of the same constraints.

145

146

Consistency Predicates

Clients with an active checking account As a first example, consider the functionality requirement that all the bank’s clients must have at least one active checking account. Note that this constraint is not represented in the structure of the domain model because the design that I chose as a solution for the banking domain has an association between the classes Client and ClientAccount, which includes both checking and savings accounts. So, having a multiplicity value of

1..* in the role declaration for the ClientAccount is not enough to guarantee that each client has a checking account. Starting with the implementation of the constructor for the class Client, must a new client, as returned by the constructor, have already a checking account? If the returned object must have a consistent state, the answer to the previous question is yes, and we may implement it in one of two ways:

• The constructor receives a non-closed checking account as an argument and adds that checking account to the client’s set of accounts. In this case, however, the checking account must already exist before the call to the Client’s constructor. So, it must have been created without an owner, which violates the constraint that all the client accounts must have exactly one owner. • The constructor for the class Client creates a new checking account by calling the CheckingAccount constructor with the instance of the client as the owner of the account to create. This solution allows that the constructors for both classes return consistent instances. Yet, it is a little bit awkward that the responsibility of creating a checking account is in the client, rather than in the bank, because, most probably, there are restrictions for the creation of an account that should be verified by the bank.

This difficulty of ensuring the consistency of a newly constructed object is amplified if we consider cases where we need to create several objects that must form a complex graph. Neither of the two previous solutions scales well for such cases. In general, it is easier to have some method that knows how to create all the objects and how to link them together. So, if we take that route and we decide that the constructor for the class Client does not need to return a client with a checking account, where else is the responsibility of ensuring the consistency of the new client object? Most probably, the creation of a new client, the creation of a new checking account, and the linking of the two objects, should be the responsibility of some method in the class Bank. Unfortunately, even though such a method in the class Bank may ensure the consistency of the clients it creates, it is not easy in Java to ensure that clients are created only by this method. Thus, by allowing the creation of inconsistent clients, we open up the door for having at another part of the

6.2 Examples of Constraints

application code the creation of an instance of the class Client without ensuring the proper constraints for the client objects. Either way, regardless of the solution chosen for the construction of the client object, then we have the problem of avoiding that future changes to the state of the objects violate the consistency of the client object. Which changes may do that? Either the removal of an account from the set of accounts of a client, or the closing of a checking account. Therefore, we must specialize both of these operations to ensure that the client remains in a consistent state when each of the operations is performed. If the normal execution of an operation would cause an inconsistency, then the operation must fail by throwing an exception. In Listing 6.1 on the following page, I show a possible implementation of this functionality. Note that, even though I created a method in the class

Client to check the constraint, that method must be called from several locations in the code, both in the class Client and in the class CheckingAccount. This implementation suffers from the problem of code scattering, because the code that implements the constraint is scattered over various methods of several different classes. Furthermore, given that the method close of the class CheckingAccount needs to call the method checkActiveCheckingAccountExists in the class Client, it creates a stronger coupling between these two classes.

Closed accounts The second example is about the constraints associated with closed accounts. In the description of the example given in Section 2.2.2, there are several functionalities related to closed accounts: only classes with a balance of zero may be closed; closed accounts cannot have further deposits or withdrawals; and once the balance of a savings account reaches zero, the account must be closed. In this case, because the constraints depend only on the state of one object (a client account), the implementation of these requirements is relatively straightforward:

• We must check in the method close of the class ClientAccount that the balance of the account is zero, throwing an exception if not. • We must override, in the class ClientAccount, both of the methods deposit and

withdraw to throw an exception when the account is closed. Or, in alternative, we may override only the method setBalance to throw an exception in the same condition. • We must override the method withdraw (or the method setBalance) in the class

SavingsAccount to close the account when the balance reaches zero.

147

148

Consistency Predicates

class Client extends ClientState { void checkActiveCheckingAccountExists() { for (ClientAccount acc : getAccountSet()) { if (acc.isChecking() && (! acc.isClosed())) { return; } } throw new ClientWithoutCheckingAccountException(); } static { AccountOwnership.addListener(new CheckListener()); } static class CheckListener extends AssocAdapter { void afterRemove(Client client, ClientAccount acc) { client.checkActiveCheckingAccountExists(); } } } class CheckingAccount extends CheckingAccountState { @Atomic void close() { super.close(); getOwner().checkActiveCheckingAccountExists(); } } Listing 6.1: Ensuring that a client has always at least an active checking account. The class AssocAdapter is a generic class that implements the interface AssocListener with all the methods empty. It is used to simplify the creation of listeners that need to specialize only one of the methods.

6.3 Consistency Predicates for Atomic Actions

class ClientAccount extends ClientAccountState { @Atomic void close() { if (! getBalance().isZero()) { throw new CloseAccountWithMoneyException(); } setClosed(true); } void setBalance(Money newBalance) { if (isClosed()) { throw new ClosedAccountException(); } super.setBalance(newBalance); } } class SavingsAccount extends SavingsAccountState { @Atomic void setBalance(Money newBalance) { super.setBalance(newBalance); if (newBalance.isZero()) { close(); } } } Listing 6.2: Implementation of the constraints for closed accounts. In Listing 6.2, I show the methods of the class ClientAccount and of the class

SavingsAccount that implement these three changes. Unlike in the previous example, the code that implements each constraint is localized in the appropriate class, rather than being spread over different classes. Yet, note that each of the methods shown in this example mixes code from different concerns, rather than having each concern implemented separately. It is, thus, an example of code tangling.

6.3

Consistency Predicates for Atomic Actions

To simplify the implementation of constraints for a rich domain model, I propose the use of consistency predicates. The two key ideas underlying consistency predicates are the following: • A consistency predicate is a predicate that checks whether a domain object satisfies some particular constraint. • Consistency predicates are checked only at the end of an atomic action, rather than

149

150

Consistency Predicates

A value:int

0..1 a

0..* b

B value:int

Figure 6.1: UML class diagram for two classes with a bidirectional association between them. being checked whenever a method of a domain class is executed; the atomic action is valid only if all the consistency predicates evaluate to true. Even though the constraints for rich domain models may involve the state of more than one object, I assume that there is always one object through which we can access all the objects needed to check the validity of a constraint. Thus, we may check the validity of all the constraints for a rich domain model with predicates for single objects—these predicates are the consistency predicates. Note that, even though consistency predicates are predicates for single objects, they may refer to other objects that are accessible through the object that is being checked by the predicate. In fact, I propose that consistency predicates be implemented as domain classes’ methods, with no restrictions other than that they must return a boolean value and that they should have no side-effects. To show an example of methods that may implement consistency predicates, consider the case of two classes, A and B, with a bidirectional association between them, as depicted in Figure 6.1. We may implement these classes in DML as follows:

class A { int value; } class B { int value; } association Rel { A playsRole a { multiplicity 1..1; } B playsRole b { multiplicity 0..*; } } Consider, also, that there are two constraints involving these classes: (1) the value of each instance of class B must be less than half the value of the instance of class A with which the instance of B is related to; and (2) the value of an instance of class A must be greater than or equal to the sum of the values of all the instances of class B that are related to that instance of A. In Listing 6.3 on the next page, I show how the consistency predicates corresponding to these two constraints may be implemented as methods of the classes A and B. Consistency predicates are meant to ensure that the state of a domain model is kept consistent. The domain is in a consistent state if all the domain’s objects satisfy all the consistency predicates—for example, in the case just presented, all the instances of classes A and B must satisfy the two consistency predicates implemented by the methods

6.3 Consistency Predicates for Atomic Actions

class A extends AState { boolean sumOfBsNotGreaterThanValue() { int sum = 0; for (B b : getBSet()) { sum += b.getValue(); } return getValue() >= sum; } } class B extends BState { Boolean valueIsLessThanHalfTheValueOfA() { return getValue() < (getA().getValue() / 2); } } Listing 6.3: Methods implementing the consistency predicates for the classes A and B. Each method returns the value true if the receiving object satisfies the constraint and return the value false otherwise.

shown in Listing 6.3. So, we may check whether the domain is in a consistent state by calling each of the methods implementing consistency predicates for all the applicable domain objects. But, when should we do this check? If we assume that we start with a domain in a consistent state—that is, where all the consistency predicates evaluate to true—then the only way to break the domain consistency is by changing the state of any domain object. So, we just need to check the consistency of the domain state when something changes in that state. Yet, as we discussed in the previous section, it is not possible to have the domain in a consistent state all the time because, to perform certain operations, we may need to break the consistency temporarily, during the execution of the operation. So, I argue that the boundary for enforcing the domain’s consistency should not be at the end of each method that may change the state of a domain object. Rather, it should be at the end of each atomic action, because they correspond to the smallest unit of work semantically meaningful in a domain model. In fact, when using a language with atomic actions, the state of the domain changes only as the result of some successful atomic action. Atomic actions, as implemented, for instance, in the JVSTM, give us already the properties of atomicity and isolation. By requiring that at the end of each atomic action all the consistency predicates be true, we enforce also the property of consistency for atomic actions, thereby ensuring that the domain will always remain in a consistent state. Thus, if at the end of an atomic action, any consistency predicate evaluates to false, the atomic action cannot commit successfully. Instead, the atomic action must abort, because it is not consistent. Note that consistency predicates cannot transform an incon-

151

152

Consistency Predicates

sistent action into a consistent one, because consistency predicates cannot change the state of the domain. Consistency predicates only check the consistency of the domain’s state, preventing inconsistencies by vetoing the commit of the actions that would cause an inconsistency. Having a complete and correct set of consistency predicates helps in the implementation of a rich domain model because any programming error in the update of the domain state that would cause an inconsistency is detected by the consistency predicates. Detecting programming errors, however, is not the sole purpose of consistency predicates. On the contrary, consistency predicates implement also the domain logic that is typically found in the methods that change the state of a domain object—the domain logic that checks whether the operation can be performed in the current state of the object and with the given arguments. By using consistency predicates much of that code may disappear from those methods. Instead, the methods perform the operations as usual and if a constraint is violated, consistency predicates detect that violation and abort the atomic action, thereby undoing all the changes that caused the inconsistency. This combination of atomic actions with consistency predicates represents a significant change in how programmers may develop a rich domain model: Rather than using a defensive style of programming that checks first whether the program’s execution may proceed with the changes requested, they may, instead, specify separately the constraints in consistency predicates and perform the actions without checking first, knowing, however, that if something goes wrong all the changes will be undone.

6.4

Consistency Predicates in Java

To express consistency predicates in Java, I propose the use of a new annotation type,

@ConsistencyPredicate, that may be used to annotate the methods that check the consistency predicates’ conditions. These methods must be non-static, have a return type of boolean, and have no arguments. Also, because these methods are meant to check that an object of their class satisfies the consistency predicate, they should be made final so that they are not overridden in a subclass. The code of the methods should return the boolean value true if the condition of the consistency predicate is verified, and false otherwise. To determine its return value, the method may execute whatever it wants, provided that it has no side-effects. If the execution of a consistency predicate returns false, or fails in any way, the transaction executing that consistency predicate is aborted, throwing an exception of type ConsistencyException. The annotation ConsistencyPredicate, however, may specify a different exception type to throw. In that case, the exception thrown by the transaction is of the type indicated in the annotation.

6.4 Consistency Predicates in Java

class Client extends ClientState { @ConsistencyPredicate(NegativeClientBalanceException.class) boolean totalBalanceNonNegative() { return (! totalBalance().isNegative()); } } Listing 6.4: Consistency predicate for checking that the client total balance is not negative. The exception to throw if the consistency predicate fails is of the type NegativeClientBalanceException.

class Client extends ClientState { @ConsistencyPredicate boolean atLeastOneCheckingAccount() { for (ClientAccount acc : getAccountSet()) { if (acc.isChecking() && (! acc.isClosed())) { return true; } } return false; } } Listing 6.5: Consistency predicate for checking that a client always have a nonclosed checking account. The method getAccountSet returns the set with all the client’s accounts. The method isChecking returns true if the account is a checking account and false otherwise.

In Listing 6.4, Listing 6.5, Listing 6.6, Listing 6.7, and Listing 6.8 on the next page, I show the implementation of the constraints discussed above for the banking domain. Contrary to the implementations discussed before, the use of consistency predicates allows us to have all the constraints implemented in the appropriate class, in a single code unit (a method) that deals with one and only one consistency predicate. So, we no longer have code tangling or code scattering for the implementation of constraints that refer to several domain entities. The coupling among the classes is reduced also, given that only the class that wants to enforce a constraint needs to know about that

class ClientAccount extends ClientAccountState { @ConsistencyPredicate boolean closedAccountHasNoMoney() { return (! isClosed()) || getBalance().isZero(); } } Listing 6.6: Consistency predicate for checking that a closed ClientAccount has no money.

153

154

Consistency Predicates

class SavingsAccount extends SavingsAccountState { @ConsistencyPredicate boolean accountWithNoMoneyMustBeClosed() { return (! getBalance().isZero()) || isClosed(); } } Listing 6.7: Consistency predicate for checking that a SavingsAccount with no money must be closed.

class CheckingAccount extends CheckingAccountState { @ConsistencyPredicate boolean accountWithNegativeBalanceMustBeInBank() { return getBalance().isNegative() == hasBank(); } } Listing 6.8: Consistency predicate for checking that a CheckingAccount is in the list of accounts to process by the bank if and only if it has a negative balance.

constraint. Finally, given that the execution of the consistency predicates is not statically determined in certain points of the code, it is now possible to compose several classes into more coarse-grained objects with operations that orchestrate several of the finer-grained classes’ operations without the need to tweak the methods being composed.

6.5

Enforcement of Multiplicities with Consistency Predicates

Besides all the remaining advantages described above, consistency predicates allows us, also, an elegant solution to the much debated problem of implementing the constraints imposed by the multiplicities of an association between two classes. In fact, none of the work that I am aware of that tackles the problem of implementing associations—either by extending the programming language, or by presenting patterns on how to implement them—addresses conveniently the problem of enforcing the association’s multiplicities. In may cases, this problem is simply ignored, whereas in other cases the problem is acknowledged but no convincing solution is presented to solve it. Likewise, when I described the implementation of a DML specification in Section 5.7, I skipped over the implementation of multiplicities, but referred to this section as presenting a solution for it. The difficulty in implementing the constraints imposed by the multiplicity property on an association’s role is not in implementing the code that checks whether the constraint is verified; that is trivial, in fact. Rather, the problem is in knowing when to do that check.

6.6 Implementation of Consistency Predicates in the JVSTM

class ClientAccountState extends Account { @ConsistencyPredicate final boolean checkMultiplicityOfOwner() { return hasOwner(); } } class ClientState { @ConsistencyPredicate final boolean checkMultiplicityOfAccount() { return (getAccountSet().size() >= 1); } } Listing 6.9: Consistency predicates generated by the DML compiler for checking the multiplicity of the AccountOwnership association.

Consider, for example, that we want to enforce the multiplicities for the following association (shown previously in Figure 5.8):

association AccountOwnership { Client playsRole owner { multiplicity 1; } ClientAccount playsRole account { multiplicity 1..*; } }

Checking that the multiplicities are correct for an object of any of the classes is just a simple matter of checking that the client account has an owner, or that the client has at least one account. Yet, where and when should that check be made? The answer to this question proves difficult when no notion of atomic action exists. With consistency predicates, however, we may now use the multiplicity information specified in the DML language to enforce, at the end of each atomic action, that domain objects have the correct number of links as prescribed by an association declaration. In Listing 6.9, I show the consistency predicates generated by the DML compiler for checking the multiplicity of the AccountOwnership association.

6.6

Implementation of Consistency Predicates in the JVSTM

The implementation of consistency predicates in the JVSTM is relatively straightforward.

155

156

Consistency Predicates

We saw already that to use consistency predicates in a Java program we use the method-level annotation @ConsistencyPredicate. Thus, finding all the consistency predicates for a particular set of classes is just a matter of using the reflection capabilities of the Java language to iterate over all the methods of all those classes and collect all the methods found with this annotation. This, obviously, is needed only once for each program execution, provided that we cache the result somewhere.2 Having the set of consistency predicates for a program, we need now to know when to call them to check the consistency of the domain. Given their semantics, we know that they must be called at transaction’s commit-time. Yet, not all transactions need to check the consistency. Because only write-transactions can change the state of the domain, only these transactions need to check the consistency predicates. So, a first solution to the problem is to check all the consistency predicates for all the objects belonging to classes with consistency predicates whenever a write transaction is about to commit. Even though this solution is simple to implement, it is unacceptably inefficient in terms of time. We may do better in terms of execution time if we spend some memory to keep a dependence network between the objects. The fundamental idea is that, at the end of a transaction, we should check only the following: • The consistency predicates of all the objects created during the transaction. • The consistency predicates that may have become invalid because of a change made in some existing transactional location—that is, a change in a VBox. To be able to do this, transactions need to know when a new object is created and which consistency predicates depend on each box. The first part is easily implemented by adding a new method to the Transaction interface that allows new objects to notify the transaction about their creation, which they should do in their constructors (if we are using the DML to generate the domain classes, this is easily automated). To implement the second part, however, we need to keep a record of which consistency predicates depend on which boxes.

This is the purpose of both the class

DependenceRecord and the association shown in Figure 6.2. When a consistency predicate is executed for the first time for a new object, a new instance of a DependenceRecord is created and is initialized with the new object being 2

There are many other ways to implement this; for some we may even do all the work at compile time. Yet, as this is relatively straightforward, I do not discuss all the alternatives here, so that I may concentrate on the most relevant aspects of the implementation.

6.6 Implementation of Consistency Predicates in the JVSTM

VBox

0..* depended

0..* dependence

DependenceRecord dependent:Object predicate:Method

Figure 6.2: UML class diagram showing the central elements for the implementation of consistency predicates in the JVSTM. checked (which is kept in the dependent attribute) and with the Java’s method that implements the consistency predicate (which is kept in the predicate attribute). Then, to execute the consistency predicate for the new object, a new special kind of nested transaction that knows about this dependence record is created. This special transaction differs from a normal nested transaction in the following: • If a write to a box is attempted, the transaction throws an exception. This prevents that consistency predicates have side-effects (at least on the transactional state). • Whenever a box is read during the execution of the consistency predicate, the transaction should record that the consistency predicate depends on the value of that box. So, it creates a link between the dependence record and the box. Therefore, at the end of the execution of the consistency predicate, the dependence record is related to the set of all the boxes on which the consistency predicate validity depends. As the association between the class DependenceRecord and the class VBox is bidirectional, each vbox is related, also, to the set of all the dependence records that depend on the value of the box. When a top-level write transaction commits, it just needs to go through each of the boxes in its write set and recheck all the dependence records that are registered with each box. Rechecking a dependence record means reexecuting the method stored in the slot

predicate on the dependent object stored in the slot dependent. This reexecution is similar to the first one, when the dependence record is initially created, except that we need to clear all the existing depended values for the dependence record (which has the side-effect of removing the dependence record from the boxes on which it depended upon, also) before the reexecution of the consistency predicate. To conclude this outline of the implementation of the consistency predicates in the JVSTM, there is one final question to be answered: At which phase during the commit of a write transaction is the consistency check performed? To answer this question it is worth noting that the process of checking the consistency predicates for one particular transaction may occur concurrently with a similar process for another transaction. Moreover, during the execution of this process, some shared data structures (namely, the links between boxes and dependence records) are changed. So, we need to be careful in implementing this process to avoid problems with data races.

157

158

Consistency Predicates

Consider, for instance, that we have a single dependence record, DR, that depends on two boxes, B1 and B2. Imagine now that two concurrent transactions, T1 and T2, are committing at the same time, and that T1 changed box B1 and T2 changed box B2. Then, both transactions should recheck the dependence record DR. If both are doing the consistency check concurrently, however, the following series of events may happen: transaction T1 identifies DR within the dependencies of B1 and prepares to recheck it, clearing all the DR depended values—as a consequence of this, box B2 is no longer related to DR; then, transaction T2 checks box B2 to see which dependencies exist for this box and finds no dependence; finally, transaction T1 rechecks DR and, while doing it, adds again the dependence to box B2. This example indicates that we may fail to reexecute some dependence records if these data races are not dealt with. Unfortunately, if that happens, an inconsistency may easily slip through into the domain. A safe and easy way to deal with this problem is to leverage on the support for concurrency given by the JVSTM. If we implement the association between the class

DependenceRecord and the class VBox transactionally—that is, using versioned sets to keep the links—then we may add and remove links to the association within a transaction without fear of those changes being made visible to other transactions or that concurrent accesses to those links cause an inconsistency. The idea is that the commit of a top-level write transaction checks the consistency of its changes by executing all the needed consistency predicates before its validation phase. That is, the consistency check is part of the normal transaction execution, except that it is performed immediately before the commit, after all the user code has executed (so that we know already which boxes have changed). During this consistency checking, the transaction may read new boxes to evaluate the consistency predicates, but it may write, also, to the boxes that are used to keep the dependence records and the vboxes connected to each other. Naturally, all these new reads and writes are recorded in the transaction read set and write set, respectively. So, if some consistency predicate fails, the transaction simply aborts and all the changes made to the dependence records are discarded, as usual. If, on the other hand, all the consistency predicates succeed, then the commit of the top-level transaction proceeds with the validation phase, where it may find that a dependence record was concurrently updated by another transaction, in which case it detects a conflict and restarts the whole transaction. Otherwise, if the transaction is valid, the commit of the transaction will make the changes to the dependence records permanent. Even though it is possible to find alternative solutions to this problem that avoid some of the conflicts caused by changes in the dependencies, this implementation has the advantage of simplicity, given that it relies on the existing support given by the JVSTM. So, this was the solution that I adopted to implement the consistency predicates in the JVSTM.

6.7 Related Work

6.7

Related Work

The idea that the state of an object must satisfy a set of predicates, often called invariants, traces back to the work of Hoare [1972] on data-representation correctness. Later, Meyer built on this idea, proposing the use of class invariants as one of the fundamental elements of his design by contract methodology for object-oriented design [Meyer, 1988, 1992a]. The design by contract approach is a design methodology in which programmers must specify a contract for each class. A contract is specified by a set of pre-conditions and post-conditions for each of the public class’s methods, as well as a set of class invariants. Class invariants specified for a class C are predicates that must be true for all instances of C that are publicly exposed to other parts of the program. They provide, thus, a set of guarantees about the state of an instance of C. As part of his work on the design by contract approach, Meyer built into the programming language Eiffel [Meyer, 1992b] a set of constructs to allow the specification of contracts, including, therefore, the specification of class invariants. Having class invariants explicitly represented in a program allows the Eiffel runtime to check whether the invariants are true during the execution of the program, thereby facilitating the detection of errors. Meanwhile, this design by contract approach at the programming language level has been applied to many other languages, including, naturally, the Java programming language [Kramer, 1998; Karaorman, Hölzle, and Bruno, 1999; Bartetzko, 1999; Leavens, Ruby, Rustan, Leino, Poll, and Jacobs, 2000; Flanagan, Leino, Lillibridge, Nelson, Saxe, and Stata, 2002; Lackner, Krall, and Puntigam, 2002; Leavens, Cheon, Clifton, Ruby, and Cok, 2005]. All of these extensions to Java follow, more or less closely, the semantics of the constructs provided by Eiffel. Even though consistency predicates resemble very much class invariants, there are some fundamental differences between the two approaches. First and foremost, there is a conceptual difference between the two approaches. One of the basic tenets of all the design by contract approaches is that class invariants are used in a program as a debugging tool only. That is, the idea is that class invariants’ checking is performed during program execution only to detect programming errors that may cause the invariants to be broken. In particular, a correct program should never obtain a false result from the evaluation of class invariants. So, once a program has been tested, class invariants may be disabled. In fact, Eiffel and most of the other approaches that implement class invariants allow programs to run with all the checking disabled. Consistency predicates, on the other hand, are not for debugging purposes, even though they may be used like that. Rather, they implement a significant part of a domain’s logic and, therefore, cannot be disabled, if the program is to work as expected. That is,

159

160

Consistency Predicates

the semantics of a program depends on the execution of consistency predicates. For this, however, consistency predicates depend on the semantics of failure recovery given by atomic actions, which are not generally available in the languages with support for class invariants. A second difference is on the expressiveness of the predicates. Class invariants as proposed by Eiffel (and followed by all the design by contract extensions to Java) cannot access arbitrary data nor call arbitrary methods to verify their conditions. Instead, they must restrict themselves to access the private state of the object being verified. In particular, class invariants cannot access the state of an object referenced by the object being checked, nor can they call a method on such an object. These restrictions impose severe limitations on the expressiveness of class invariants that preclude many interesting invariants that depend on several objects. In fact, several authors acknowledge this problem and propose to extend class invariants so that they may access the state of other objects [Barnett, DeLine, Fähndrich, Rustan, Leino, and Schulte, 2004; Dietl and Müller, 2005; Müller, Poetzsch-Heffter, and Leavens, 2006]. To accomplish that, they build on work on ownership models to limit and to control the aliasing of objects to a set of well known places within an aggregate [Clarke, Potter, and Noble, 1998]. Class invariants in such approaches are able to extend their visibility to all the objects owned by the object being checked. Even though these approaches represent a step further, they are still too limited for the needs of a rich domain model. In such a domain model, it is not practical to limit the aliasing between entity types, given that they are often part of very intricate object graphs where there is no object that is a natural owner of the others. A third and final difference between the two approaches is on when the predicates are evaluated. In Eiffel, class invariants are checked at the beginning and at the end of the execution of each of the class’s methods (except for helper methods), whereas consistency predicates are checked only at the end of a transaction. I discussed already in the beginning of this chapter, the differences between these two approaches. Having atomic actions in a programming language is, actually, the key enabling element for having consistency predicates as I have proposed them in this dissertation: Once we have atomic actions, the idea underlying consistency predicates is relatively straightforward. Therefore, it is no surprise that there are many things in common between the work described in this chapter and the proposal made recently by Harris and Peyton-Jones [2006] to add data invariants to transactional memory. Even though Harris and Peyton-Jones are working in a different setting, with a different STM implementation made for the Haskell programming language, their implementation strategy is quite similar to mine: To reduce the number of predicates that need to

6.8 Summary

be evaluated at the end of a transaction, both implementations use a dependency record between transactional locations and the predicates that depend on them. Also, they use a nested transaction to evaluate each invariant (as they call them), but, unlike in my implementation, they allow the invariant to write to transactional locations and then abort the nested transaction in the end to avoid side-effects. Furthermore, to make the changes in the dependencies after all the invariants’ checking, they modify the commit algorithm of their STM implementation to deal specifically with the dependencies that were changed, whereas I rely on the already existing mechanisms in the STM to accomplish that. In fact, my approach is sufficiently generic to be applicable almost without changes to any other STM implementation that supports nested transactions. Another difference, now at the programming interface level, is that invariants are dynamically specified during the execution of the program, rather than being a static element of the program as consistency predicates are. Also unlike consistency predicates, in their proposal invariants are checked whenever they are created, even though they are checked again at the end of the transaction. Finally, Harris and Peyton-Jones follow with the tradition of design by contract that invariants are meant for debugging only. Yet, I argue that this conceptual difference has a significant pragmatical influence on how consistency predicates (or invariants) are used in the implementation of a rich domain model.

6.8

Summary

This chapter proposes to separate the implementation of domain constraints from the implementation of the remaining aspects of a domain model. To allow this separation, it proposes the use of consistency predicates, which are automatically executed when a toplevel write transaction commits to check whether the domain objects are in a consistent state. The chapter uses several of the domain constraints from the banking domain model to compare their implementation using the current best-practices with the implementation using consistency predicates. It describes, also, how consistency predicates may be used to implement the enforcement of an association’s multiplicities. Finally, it describes the implementation of consistency predicates in the JVSTM and discusses related work.

161

162

Consistency Predicates

Chapter 7

Validation This dissertation’s thesis is that it is possible to simplify significantly the task of implementing an object-oriented domain model by making a small, non-disruptive, and easy to implement set of additions to the current object-oriented programming languages. More specifically, that we may achieve that goal by adding to an object-oriented programming language support for atomic actions, a declarative language for specifying the structural aspects of a domain model, and consistency predicates that validate atomic actions at commit-time. In the last three chapters, from Chapter 4 through Chapter 6, I made concrete proposals for each of these different additions. For each case, I identified the problems, proposed a solution, and exemplified how that solution helps in solving the problems identified. Moreover, I described how to implement each of the proposals, showing not only that their implementation is feasible, but also that it is possible to do it with little effort in a practical way—that is, without introducing major changes in the languages and the tools used by software developers. In fact, all the work described thus far tries to respect the guiding principles put forth in the introduction of this dissertation. The purpose of this chapter is to complement the thesis’ validation that was given along each proposal, by describing two applications of the proposals made in this dissertation. First, I describe the application of all my proposals in the development of a large realworld web application: the Fénix system. The work described in this dissertation and its application to the Fénix system are, actually, quite intertwined, as they have occurred in parallel, influencing each other. Second, I evaluate the performance of the JVSTM implementation for a more STM-like traditional setting. Even though the JVSTM was developed with the goal of simplifying the implementation of a domain-intensive application, in this chapter I report on how it performs in two existing benchmarks for STMs.

164

Validation

7.1

The Fénix Case Study

The Fénix project is an open-source university management system that aims to incorporate all on-line campus activities and related management services [FenixEDU]. From its initial deployment with an initial limited set of functionalities in 2001 until now (mid 2007) it has been continuously growing both in functionality and use. In this section, I describe how the proposals made in this dissertation were employed in the more recent development of this system.

7.1.1

Fénix History

The project has been in development at the Instituto Superior Técnico (IST), from the Technical University of Lisbon, since 2001. The project started with a limited set of functionalities to support the creation and the management of web pages for some of the IST’s courses, but has ever since expanded its functionality to encompass most of the activities and management of the IST. Over time, it replaced most of the school’s legacy information systems that were becoming obsolete and hard to maintain. Therefore, it went through an enormous pressure for increasing its scope. The Fénix system is now an indispensable element for all the IST’s activities and users, which comprise over 12,000 students, 1,000 faculty, and 700 administrative staff. In particular, most of the administrative staff depends on the system for their daily work. Even though the IST continues to be the system’s lead development organization, other universities and companies joined the project and are now using it and further developing it for deployment in other universities. Presently (mid 2007), the system is deployed in three other universities and is being deployed in at least another three. The initial development team at IST included only a handful of programmers but has since expanded to include more programmers, as well as a Users Support and Requirements Team and a Web Design Team. The programmers team comprises a set of senior members, all of them with a degree in software engineering or a related area, and a set of graduation students that are, typically, in the last year of their degree in software engineering at IST. The senior members work full-time on the project, whereas the graduation students work only half-time and during approximately one year as part of their degree final project. I show in Table 7.1 the evolution in the composition of the Fénix’s programmers team over time. I joined the Fénix team in 2004 as an external collaborator and became partly responsible for the system’s architecture since 2005. Since that time, I have introduced

7.1 The Fénix Case Study

165

Number of members in the team by year Type of Member Senior Graduation Students

2001/02

2002/03

2003/04

2004/05

2005/06

2006/07

5 2

5 12

7 19

7 16

14 10

12 2

Table 7.1: Composition of the Fénix project’s programmers team at IST.

into the system the JVSTM, the DML, and the consistency predicates described in this dissertation. Yet, my work was not to use them to implement the Fénix domain model. Rather, the bulk of my work for the Fénix project has been in integrating my proposals into the system’s framework and architecture, so that the rest of the team may use them to implement the domain model.

7.1.2

The Original Fénix Software Architecture

The core of the Fénix system is developed in the Java programming language and consists of a web application that was initially developed by following the best-practices for that software development area, as of 2001. The system used the standard three-layered architecture of an enterprise application that I discussed in Chapter 2 (see Figure 2.1 on page 16). Therefore, it separates the code that implements the domain model from both the code that accesses the database and the code that deals with the user interaction. The implementation of the domain model, however, was separated in two distinct elements: (1) the classes implementing the domain entities, which consisted mostly of the entities’ properties, and getters and setters to access those properties; and (2) the classes that implemented the application’s services, which contained most of the domain logic related to the behavior of the entities. This solution, in fact, is akin to the pattern Transaction Script described by Fowler [2002]. Each service, implemented typically as a single method, used the services of the data-source layer to access the objects representing the domain entities and operated on those objects according to the domain model’s requirements. To control the concurrent access to the domain entities that is inherent to this kind of applications, the Fénix code resorted to the interface provided by the implementation of the ODMG 3.0 Object Persistence API [Cattell, Barry, Berler, Eastman, Jordan, Russell, Schadow, Stanienda, and Velez, 2000] available in the Object/Relational mapping tool that was used in the project: the Apache DB Project’s OJB [OJB]. The ODMG API includes operations for locking objects either for read or for write before they are accessed. Unfortunately, this lock-based approach to concurrency was

166

Validation

highly error-prone, as programmers often forgot or misplaced the acquisition of locks, causing frequent consistency problems into the domain data. Moreover, with the increased usage of the system appeared also the first performance problems, which were attributed, after some performance profiling, to the overheads incurred in the acquisition and the management of locks by the operations that accessed many thousands of objects. In fact, these performance problems led to the development, in 2004, of a preliminary version of the versioned STM described in this dissertation.

7.1.3

The Use of this Dissertation’s Work in the Development of the Fénix System

Even though the first proposal that I made specifically for the Fénix system was the versioned STM, the first of this dissertation’s proposals to be effectively employed in the development of the system was the DML language proposed in Chapter 5. The Fénix team started to use the DML language in April 2005 and the JVSTM was deployed later, in September 2005. The consistency predicates, on the other hand, were deployed only during 2007 with a limited implementation. Thus, given that the experience of using consistency predicates in the Fénix system is still very limited, in the following I shall describe only the use of the DML and the JVSTM in the Fénix project.

7.1.3.1

Implementation of the Fénix Domain Model with the DML

The goal of introducing the DML language in the development of the Fénix domain model was initially twofold. First, to eliminate the errors in the management of bidirectional associations. Second, to simplify the introduction of the STM in the system. The classes that implemented the domain entities in the Fénix system followed a common pattern: each class contained a set of attributes with a pair of getter and setter methods for each of the attributes. Likewise, the implementation of associations consisted, in most cases, in one attribute in one (for unidirectional associations) or in both (for bidirectional associations) of the participating classes, again with the corresponding getter and setter methods; when the multiplicity of an association’s end admitted more than one element, the attribute used to implement that end of the association was of a collection type. This simple approach of implementing associations, however, had the problem that the setters of each of the attributes that corresponded to an association did not enforce the consistency of the association when the association was bidirectional. Thus, the responsibility of ensuring the consistency of such associations was on the programmer

7.1 The Fénix Case Study

that wrote the code that needed to create or remove a link; the programmer would have to invoke the two necessary methods to ensure that the link was consistently created or removed. Unfortunately, this is too much error-prone, and this was one of the problems that was causing significant troubles to the Fénix team and to the project. On the other hand, to use an STM to control the concurrent access to the domain entities, all the classes that implemented domain entities needed to be changed to become transactional. Given that the number of domain classes was slightly over two hundred, that was not an easy task to do. Moreover, back then, it was not clear whether the use of an STM would be feasible for a system with the dimension of the Fénix system. So, committing to that change was a risk too high for the project. The use of the DML solved both of these problems. Having the domain model’s structure represented in a declarative way allows us (1) to generate the methods that implement correctly the associations between the domain classes, and (2) to generate the domain classes as transactional classes or as plain java classes, as we may see fit—it is just a matter of changing the DML compiler’s backend. In fact, when the DML was first introduced in the development of the Fénix domain model, in April 2005, the DML compiler generated the classes exactly as they were implemented in the system at that time. In particular, neither it generated the classes as transactional, nor it generated the methods that implemented bidirectional associations correctly. The goal was to allow a gradual conversion from the Java classes to their DML substitutes, given that the large number of classes in the system would not allow a rapid conversion. Indeed, this conversion process took place during the second quarter of 2005, from April to June, and was performed incrementally by all the team, while they were developing new functionalities. When finally all the classes and associations became represented in DML, we could change the DML compiler’s backend to generate the code that ensured the consistency of bidirectional associations; this occurred in July 2005. Later, with the introduction of the JVSTM, in September 2005, the compiler was once again changed to generate transactional classes. Given that the interface of the classes generated was always the same, no further changes in the existing code were necessary in either case. This use of the DML language illustrates one of its advantages that were already mentioned in Chapter 5—that we may change the code generation strategy without changing the remaining code of the system.

167

168

Validation

Lines of code for each date (×1000) Source Code Artifact

Apr 2005

Sep 2005

Jul 2007

64 121 112 28 18

37 5 92 107 115 33 21

136 12 195 89 238 8 1

Domain classes DML code (generated from the DML) Remaining domain code (services) Presentation layer Persistence layer OJB mappings

Table 7.2: Lines of code for different parts of the Fénix system. 7.1.3.2

Other Benefits of Using the DML

Even though the benefits described above are sufficient to justify the use of the DML for implementing a domain model’s structure, there are other more far reaching benefits as well. Another advantage of the DML, of course, is that it reduces significantly the amount of work required to implement a domain model. This is shown in Table 7.2, where I present the approximate number of lines of code (LOC) of several parts of the system at three different instants of its development: (1) in the beginning of April 2005, before starting to use the DML language; (2) in the beginning of September 2005, when all the domain classes were already in the DML and the JVSTM was introduced; and (3) as of this writing, after almost two years of use of the DML and the JVSTM in the system. From April 2005 to September 2005, the most significant change in the values presented in the table was in the implementation of the domain classes: From an initial value of 64 thousand LOC for implementing the domain model’s classes, remained only 37 thousand LOC in the end. This value, however, does not reflect the actual reduction in size of the code that implements the domain model’s structure for two reasons: • During that period the Fénix team continued to develop new functionalities and to create new domain classes; the slight increase in the number of LOC in the presentation layer indicates this, also. • In parallel with the conversion to a DML representation of the domain model’s structure, the Fénix team started a refactoring process to move part of the code that was in the classes implementing the services to the domain classes, instead. The decrease of 15 thousand of LOC in the code of the services, despite the increase in the system’s functionality, confirm this. If we consider both of this factors, it becomes clear that the real reduction of code obtained with the DML should be much higher than what is shown in the table. In fact,

7.1 The Fénix Case Study

the code generated by the DML compiler in September 2005 for the 5 thousand lines of DML code was about 92 thousand LOC of Java code. This value is much higher that the initial size of the domain classes (64 thousand LOC) for several reasons: First, because the code that is generated by the DML compiler implements correctly the bidirectionality of the associations, where the original code did not; second, because all the associations are now bidirectional; and third, because the DML compiler generates also the set of optional methods that, in most cases, did not exist in the original domain model implementation. Furthermore, one of the differences in the numbers from September 2005 to July 2007, reveal another benefit of using the DML that was realized only later: The almost elimination of the OJB mappings. The OJB mappings are a description in XML of how the domain classes and associations map into the tables of a relational database. As we have the information about the domain classes and associations in a DML domain specification, those mappings may be dynamically created from that domain specification, thereby eliminating the need to write those mappings manually. Finally, this possibility of using the information about the domain model structure is being explored in several other places in the Fénix system as well:

• To automate the generation of interfaces for presenting or accepting information about a domain entity. • To generate documentation in the form of UML class diagrams to help in the visualization and the understanding of the domain model.

7.1.3.3

The JVSTM in the Fénix System

Unlike the introduction of the DML, which forced the refactoring of much of the domain model’s code, the introduction of the JVSTM in the Fénix system was performed without changes in the domain model’s code. The changes were all performed in the infrastructural code that supports the system’s software architecture.1 To use the JVSTM in an application we need to do two things: (1) to implement the mutable shared objects as transactional objects (by using the class VBox), and (2) that we identify and annotate the atomic actions. The first task is performed by the DML compiler, that generates the domain classes using versioned boxes to represent their attributes. The second task is easier to accomplish in a web application, given that in most of the cases, they process each user request as 1

Actually, after the introduction of the JVSTM, the code that acquired the locks of the ODMG API was no longer necessary, but removing that code was a simple matter of a global find and replace in the project’s source code.

169

170

Validation

Date 09/2005 12/2005 03/2006 06/2006 09/2006 12/2006 03/2007 06/2007

Classes

Associations

227 255 334 346 455 557 613 704

324 349 422 679 851 931 1011 1098

Table 7.3: Evolution of the number of classes and associations in the Fénix domain model.

a single atomic action. So, we may wrap the entire request processing code with a transaction at the web-application-framework level, thereby removing from the programmer’s hand the responsibility of doing it. Even though I do not have data to quantify it, the perceived benefits for the Fénix application that resulted from the combined use of the JVSTM with the DML, was an increase in the applications’ performance, stability, and robustness. The increase in performance results, on one hand, from the elimination of the overheads associated with lock acquisition and management, and, on the other hand, with the improved utilization of the database cache, given that it is no longer necessary to pay the penalty of a round-trip to database to ensure the consistency of the data. The improved stability and robustness of the system results from the elimination of inconsistencies caused by faulty lock acquisition and lack of bidirectional associations’ management. From the point of view of the software development task, the major benefit that results from the use of the JVSTM is the simplification of the programming model. Not only as a consequence of the simpler concurrency control approach, but also because some architectural changes were made possible by the use of the JVSTM and the DML. For instance, in Table 7.2 on page 168 we see a significant reduction in the LOC needed for the persistence layer. This reduction results from an architectural change enabled both by the JVSTM and the DML that eliminates the dependency that the domain layer had of the persistence layer: For example, the code in the domain layer no longer needs to explicitly store the objects in the database when they are changed; the JVSTM automatically does that in the commit of a memory transaction, invoking the methods of the persistence layer to persistently store the objects that were changed (provided that the memory transaction is valid). The simplification of the programming model allows for a faster evolution of the system, given that it is simpler to develop new code and that programmers spend less time in

7.1 The Fénix Case Study

171

# associations

1098

# classes

227

0 Sep 2005

Dec 2005

Mar 2006

Jun 2006

Sep 2006

Dec 2006

Mar 2007

Jun 2007

Figure 7.1: Evolution of the number of classes and associations in the Fénix domain model.

debugging their code. The evolution of the number of classes and associations in the Fénix domain model, shown in Table 7.3 on the facing page and graphically depicted in Figure 7.1, shows that the system has experienced a fast growth since the JVSTM and the DML were introduced in the development of the system. In fact, the domain model more than tripled its size in less than two years, even though the team has remained more or less the same throughout that period.

7.1.4

The Fénix Transactional Workload

In Section 4.2, I presented the rationale underlying the development of the JVSTM. In particular, I made a set of assumptions regarding the typical workload of a domainintensive application:

• That the number of write transactions is low when compared with the total number of transactions; probably, less than 10%. • That most of the transactions are medium-sized; that is, that the average size of their read sets and write sets have hundreds to thousands of objects. • That occasionally there are long-running transactions that need to access thousands to millions of objects. • That an updating transaction may read many objects, but typically changes just a few.

172

Validation

27878965

20909223

13939482

6969741

0 Oct 2006

Nov 2006

Dec 2006

Jan 2007

Feb 2007

Mar 2007

Apr 2007

May 2007

Jun 2007

Figure 7.2: Total transactions successfully processed by the Fénix web application from October 2006 to June 2007.

To validate these assumptions, I instrumented the Fénix system to collect statistics about the transactional system during its execution. In its normal execution, the Fénix web application is deployed in a cluster with four servers, which are Intel-based machines with 4Gb of memory and two 32-bit processors. Each of these machines runs an instance of the Fénix web application within the Tomcat application server. From mid September 2006 until now, each application server writes periodically2 to the database the values of a series of counters that each server maintains internally. The values written by each server are the following: a server identification, a timestamp with the current time, the number of read-only transactions processed, the number of write transactions that successfully committed, and the number of conflicts. Given these data, it is now possible to investigate what is the transactional workload of the Fénix system. In the results shown here, I consider all the data collected between the 1st of October 2006 and the 30th of June 2007, inclusive. This particular period has the particularity that it spans an integral number of months (9), weeks (39), and days (273).

7.1.4.1

Total Number of Transactions Over Time

In Figure 7.2, I show the monthly total of transactions successfully committed during the period under examination. The decrease in the number of transactions during Novem2

The interval between each write is 5 minutes.

7.1 The Fénix Case Study

2665893

1999419

1332946

666473

0 1 2 3 4 5 6 7 8 9 10111213141516171819202122232425262728 Figure 7.3: Total daily transactions successfully processed by the Fénix web application on February 2007.

ber and December coincide with the end of the first semester, and with the Christmas holidays. On the other hand, the period from mid February to mid March corresponds to one of the two periods of highest activity for the Fénix application that occur when all the IST’s students (more than 12,000) enroll for all the courses and shifts for their next semester. In fact, this year the enrollments for the second semester started on the 17th of February 2007, at 15:00, as can be seen in Figure 7.3 and in Figure 7.4. Contrast the values shown for the enrollments’ day with the values shown in Figure 7.5 and in Figure 7.6, which present the daily and the hourly average of transactions processed, respectively, over the period of nine months from October 2006 to June 2007. Whereas on average the Fénix application processes less than one million transactions per day (even though that value has been increasing over time), on the enrollments’ day the total number of transactions exceeds two and a half millions. More dramatically, the total number of transactions processed on the first hour after the start of the enrollments is more than ten times the maximum number of transactions processed on average for each hour of the day. These values show that the load of the system under normal conditions is well below what it is able to process. Furthermore, the values depicted in Figure 7.5 and in Figure 7.6 show that the workload of the system varies with the working patterns of the people using the system: There are more transactions processed during the week than at weekends and the highest rate of transactions per hour occur during the working hours, with a slight decrease around the lunch and the dinner hours.

173

174

Validation

524157

393117

262078

131039

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Figure 7.4: Total hourly transactions successfully processed by the Fénix web application on the first day of enrollments, the 17th of February 2007.

825131

618848

412565

206282

0 Mon

Tue

Wed

Thu

Fri

Sat

Sun

Figure 7.5: Average daily number of transactions successfully processed by the Fénix web application for each day of the week from October 2006 to June 2007.

7.1 The Fénix Case Study

50168

37626

25084

12542

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Figure 7.6: Average hourly number of transactions successfully processed by the Fénix web application from October 2006 to June 2007.

7.1.4.2

The Read/Write and Write/Conflicts Ratios

The numbers shown so far show the total number of transactions successfully processed by the system, without distinguishing between read-only and write transactions. Yet, one of the assumptions that influenced significantly the design of the JVSTM was that read-only transactions largely outnumber write transactions. In the Fénix web application, that proves to be the case, as we may see in Figure 7.7. Given that this plot is in a logarithmic scale, the monthly totals show that the number of read-only transactions is two orders of magnitude higher than the number of write transactions. Likewise, the number of successful write transactions is two orders of magnitude higher than the number of conflicts. Looking at these values monthly, however, may hide differences in these ratios, given that the write transactions (and, therefore, the higher probability of conflicts) are more concentrated during the working hours of the administrative staff. Thus, in Figure 7.8, I show the hourly average for each of the three values. This plot confirms that even though the number of write transactions and conflicts increases during the working hours, they maintain the ratios shown before. Finally, the stress test for these ratios is the enrollments’ day, where thousands of students are rushing to get the best timetables and, therefore, contending for the same domain objects. The hourly totals for the enrollments’ day are depicted in Figure 7.9, which show a significant reduction in both ratios on the first hour after the start of the enrollments’ period. Nevertheless, even in that pathological case, there is still an order of magnitude difference between each value.

175

176

Validation

reads

100000000 10000000 1000000

writes

100000 conflicts 10000 1000 100 10 1 Oct 2006

Nov 2006

Dec 2006

Jan 2007

Feb 2007

Mar 2007

Apr 2007

May 2007

Jun 2007

Figure 7.7: Monthly total of read transactions, write transactions, and conflicts in the Fénix web application from October 2006 to June 2007. The yy axis is in a logarithmic scale.

100000

reads

10000 1000 writes 100 10 conflicts 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Figure 7.8: Hourly average of read transactions, write transactions, and conflicts in the Fénix web application from October 2006 to June 2007. The yy axis is in a logarithmic scale.

7.1 The Fénix Case Study

177

1000000 100000

reads

10000 writes

1000 100 10

conflicts

1

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Figure 7.9: Hourly total of read transactions, write transactions, and conflicts in the Fénix web application on the first day of enrollments, the 17th of February 2007. The yy axis is in a logarithmic scale.

7.1.4.3

The Dimension of the Transactions

The previous results confirm that the Fénix web application exhibits the transactional workload, in terms of read/write ratio, for which the JVSTM was designed. Namely, that there are many more reads that writes. The remaining assumptions underlying the design of the JVSTM are related to the dimension of the transactions—that is, how many transactional reads and writes are performed by each transaction. To assess the dimension of the Fénix transactions, I instrumented the JVSTM used in the Fénix servers to count the number of boxes read and written during each transaction. These values are then accumulated and written to the database along the previously written statistics. The new statistical values collected for each period of 5 minutes are the accumulated number of boxes read (or written) by all the transactions, and the maximum number of boxes read (or written) by a single transaction. Moreover, these values are collected for each of the two types of transaction: read-only transactions and write transactions. The values that I report here were collected by the Fénix servers for a continuous period of almost 14 days in July 2007. The total number of statistical records collected during that period was 18847, each one corresponding to a 5 minutes period of one server. In Table 7.4, I show both the average and the maximum number of boxes read or written for each type of transaction during this period. The number of boxes written on average by each transaction is, as expected, very low, even though there are some

178

Validation

Number of boxes accessed Operation/type of transaction

Average

Maximum

Reads/read-only transaction Reads/write transaction Writes/write transaction

5,844 47,226 35

63,746,562 2,292,625 32,340

Table 7.4: Number of boxes accessed by each type of transaction by the Fénix web application. Number of periods by type of transaction Maximum number of boxes read Max Max Max Max

> > > >

10,000,000 boxes 1,000,000 boxes 500,000 boxes 100,000 boxes

Read-Only transactions

Write transactions

15 461 1,121 9,763

0 36 60 8,377

Table 7.5: Number of large transactions, for each type of transaction, in the Fénix web application. The count in each row includes the value of the row above.

transactions that write a more significant amount of boxes. Nevertheless, compared to both the average and the maximum number of boxes read by a write transaction, the number of boxes written is much lower. This suggests that it is a good design decision to concentrate on reducing the overheads associated with the read of a transactional location for this type of applications. More so, if we look into the maximum number of boxes read by read-only transactions. During the period under analysis, the largest read-only transaction performed more than 63 millions of reads. Such a large number, however, may have been a sporadic transaction not occurring often. So, to see how frequent large transactions are in the Fénix web application, I counted how many records exist with a maximum number of reads larger than a certain threshold. The results of this counting are shown in Table 7.5. Considering that this is a small period of less than 14 days, we may see that, on average, the Fénix application processes, per day, at least one transaction that performs more than 10 millions of reads. Lowering the number of boxes read to one million, we have more than 32 such transactions per day, on average. Finally, on average, each 10 minutes we have at least one transaction that reads more than 100 thousand boxes. Note, however, that these numbers are lower bounds for the number of transactions reading such number of boxes, given that we may have many transactions in a 5 minute period. Also, I would like to draw your attention to the comparison between the numbers for read-only and for write transactions. Even though for higher counts of boxes the number of write transactions is significantly lower than the number of read-only transactions, we

7.2 JVSTM Performance

see, surprisingly, that there is almost a similar number of write transactions reading more than 100 thousand boxes. On one hand, this result is surprising, because the overall number of read-only transactions is much higher than the number of write transactions. On the other hand, it is in accordance with the fact that on average, write transactions read almost 50 thousand boxes, ten times the average of read-only transactions. Finally, these high number of reads performed by some read-only transactions, together with the fact that most of the transactions processed by the system are read-only transactions, justify the use of speculative read-only transactions in the Fénix application. In fact, the use of speculative transactions in the Fénix application, eliminated some of the out-of-memory errors that were caused by the read sets of such very large transactions. Unfortunately, for write transactions, no similar optimization exists.

7.2

JVSTM Performance

The literature on Software Transactional Memory uses, typically, a set of simple benchmarks to evaluate and to compare different STM implementations. In most cases, those benchmarks consist of a couple of operations operating on a data structure such as a redblack tree, or a linked list. The usual measure of performance for such benchmarks is the number of transactions per second that each STM implementation delivers when running the benchmark for a fixed amount of time. One such set of benchmarks is publicly available with the implementation of the DSTM2 proposed by Herlihy et al. [2006]. More recently, Guerraoui, Kapalka, and Vitek [2007] proposed the STMBench7 benchmark as a more realistic example of what should an STM implementation be prepared to find in a real-world application. Even though the STMBench7 is still very limited in size when compared with the Fénix system, it is, however, much more realistic than the small examples typically used in this area. Therefore, in this section I use these two benchmarks to evaluate the performance of the JVSTM. Even though the goal for the JVSTM was not to provide the best-performing STM implementation, it is a stated goal of this dissertation that the implementation of the STM proposed be suitable for practical use. This brief evaluation shall confirm that this goal is attained.

7.2.1

Benchmark Running Environment

All the results presented in this section were obtained in a single machine with two dual-core AMD Opteron processors, which gives us the total of four available cores. The machine runs Linux and all the tests were performed without any other significant process running in the machine.

179

180

Validation

All the tests were compiled and executed with version 1.5.0_11-b03 of the Sun’s Java Runtime Environment using the default options.

7.2.2

Results for the DSTM2 Benchmarks

The publicly available implementation of the DSTM2 [DSTM2] includes three variations of a benchmark that consists in the repeated insertion, removal, and search of a randomly chosen integer in a set of integers. The set of integers is implemented either as a sorted single linked list, a skip list, or a red-black tree. The benchmark allows the specification of the number of threads to spawn, the percentage of updates that each thread should perform, how long should the benchmark run, and which STM implementation should be used. The choice of the STM implementation is made though the specification of a transactional factory that implements the interface specified by the DSTM2 framework [Herlihy et al., 2006]. I created two factories to run these tests: one using the JVSTM, and another that uses a single global lock to ensure exclusive access to the data structure in each operation. Moreover, I made some minor changes to the benchmark to ensure that the sets have a bounded size. More specifically, I limited the integers that are randomly chosen by the benchmark to be within the range 0 to 1023. Thus, each set will have a maximum size of 1024 elements. Also, all the benchmarks start with a set that was previously initialized to contain half of its maximum size (512 elements in this case) of randomly chosen integers. Given that the argument of each of the three operations performed on the set is randomly chosen with a uniform distribution, and that the inserts and the removals alternate with each other, the set will have, on average, 512 elements. So, the probability of a search operation returning true is 0.5. Likewise, both the insertion and the removal of an integer have a probability of 0.5 of not changing the set. One consequence of this randomness is that the effective number of write transactions for a requested workload of n % of updates is in reality only half of that. I ran each of the three benchmarks varying the following parameters:

• The number of threads to use (one of: 1, 2, 4, 8, 16, or 32). • The percentage of updates (one of: 100%, 50%, 10%, or 0%). • The transactional factory to use, which was one of the following: – ofree: This factory is provided with the DSTM2 and is one of the factories described in the paper that introduces the DSTM2. It implements the obstructionfree DSTM described previously in [Herlihy et al., 2003], but using a visible reads approach.

7.2 JVSTM Performance

– Shadow: This is another factory provided with the DSTM2 code and described in the paper. – JVSTM: This is the factory that uses the JVSTM, but where all the transactions start as possible read-write transactions. – Speculative: This is a variation of the previous factory, which starts all transactions speculatively as read-only transactions and restarts them when a write is attempted. – JVSTM-ROH: This is a variation of the JVSTM factory that follows the eventual read-only hints given when a transaction starts. The code of the DSTM2 benchmark was changed to allow the specification, by the programmer of an atomic action, that a given transaction is read-only. In particular, all the executions of the search operation are made as read-only. This factory follows these hints by starting the transactions marked as read-only in that mode. – Lock: This factory uses a single global exclusive lock to prevent that two concurrent threads access the same data structure. The two STM implementations that came with the DSTM2 need a contention manager to resolve the conflicts. Even though there are many contention management policies proposed in the literature,3 in these tests I did not explore variations of contention management. Instead, in both cases, I used the default value configured in the DSTM2 code: the backoff contention manager. For each combination of the parameters, I ran 8 different executions of the benchmark for 20 seconds. The value of transactions per second that I use in the following discussion of the results is the average of the 8 values obtained with these runs. In the following, I shall present the results for each of the 3 benchmarks. I present in tables the results obtained for all the six factories. To facilitate the comparison of the various STMs, I present the results graphically also in form of plots in a logarithmic scale. In these plots, however, I do not show the results for all the variants of the JVSTM, to avoid cluttering the plot. Instead, in the plots I present only the results for the Speculative factory.

7.2.2.1

Results for the List Benchmark

I show, in Table 7.6 through Table 7.9 the results for the List Benchmark. The plots corresponding to these values are depicted in Figure 7.10 through Figure 7.13, respectively. In this benchmark, the JVSTM performs much better than either of the two DSTM2 implementations, but it is at least an order of magnitude worse than the simple lock approach for write dominated loads. 3

See, for instance [Guerraoui, Herlihy, and Pochon, 2005; Scherer III and Scott, 2004, 2005].

181

182

Validation

Transactions/second by number of threads Factory Lock JVSTM JVSTM-ROH Speculative Shadow ofree

1

2

4

8

16

32

422938 23916 23080 18074 640 552

214365 28946 31496 26164 523 379

374421 33203 34473 39042 557 372

375223 13074 14344 12218 494 342

359732 11162 12563 10061 603 390

365472 6197 5677 4451 585 380

Table 7.6: The results for the List Benchmark with 100% of updates.

Transactions/second by number of threads Factory Lock JVSTM JVSTM-ROH Speculative Shadow ofree

1

2

4

8

16

32

414173 25577 30229 25620 834 662

207505 40761 46798 42070 712 483

337147 51786 66985 73418 741 436

351625 29286 22859 27900 765 490

336868 16180 16433 18147 875 601

342779 11146 9724 9438 930 691

Table 7.7: The results for the List Benchmark with 50% of updates.

Transactions/second by number of threads Factory Lock JVSTM JVSTM-ROH Speculative Shadow ofree

1

2

4

8

16

32

433732 28364 39469 37935 1311 855

249314 48178 73042 73024 1362 1000

373933 72879 145393 142326 1342 1071

353379 36034 51281 47071 1402 1187

359725 38481 47992 43897 1591 1423

334260 19024 33455 30018 1706 1617

Table 7.8: The results for the List Benchmark with 10% of updates.

Transactions/second by number of threads Factory Lock JVSTM JVSTM-ROH Speculative Shadow ofree

1

2

4

8

16

32

800430 34535 57948 57291 2649 991

374063 53970 114281 115751 3728 1910

702136 79273 236709 225987 5564 2998

705199 80951 223788 227925 5617 3020

714766 81278 219088 235643 5749 2864

688335 81497 230003 226171 5512 2743

Table 7.9: The results for the List Benchmark with 0% of updates.

7.2 JVSTM Performance

183

List Benchmark (100% updates)

1000000

Transactions/second

100000 Lock 10000 1000

Speculative Shadow

100

ofree

10 1 1

2

4

8

16

32

Number of threads

Figure 7.10: Transactions per second processed by each method for the List Benchmark with 100% of updates.

List Benchmark (50% updates)

1000000

Transactions/second

100000 Lock 10000 Speculative

1000

Shadow

100

ofree 10 1 1

2

4

8

16

32

Number of threads

Figure 7.11: Transactions per second processed by each method for the List Benchmark with 50% of updates.

184

Validation

List Benchmark (10% updates)

1000000

Transactions/second

100000 Lock 10000 Speculative 1000 Shadow 100

ofree

10 1 1

2

4

8

16

32

Number of threads

Figure 7.12: Transactions per second processed by each method for the List Benchmark with 10% of updates.

List Benchmark (0% updates)

1000000

Transactions/second

100000

Lock Speculative

10000 1000

Shadow ofree

100 10 1 1

2

4

8

16

32

Number of threads

Figure 7.13: Transactions per second processed by each method for the List Benchmark with 0% of updates.

7.2 JVSTM Performance

185

Transactions/second by number of threads Factory Lock JVSTM JVSTM-ROH Speculative Shadow ofree

1

2

4

8

16

32

1283214 132044 131348 58185 1186 1102

553313 160152 162268 98303 1375 211

1164354 88145 90303 111385 1367 17

1210855 92856 94012 73634 1384 14

1206412 87676 87099 64784 1369 13

1166084 83972 82766 63891 1364 45

Table 7.10: The results for the Red-Black Tree Benchmark with 100% of updates.

The exclusive lock approach, as expected, performs better with a single thread than with more threads. The JVSTM, on the other hand, scales well, regardless of the workload used, up to 4 threads. After that number, the performance degrades rapidly, except for the case of 0% of updates. This sudden fall of the performance when the number of threads is higher than the number of processors results from an increasing number of conflicts, and, therefore, restarts, of the transactions. Given that all the transactions must traverse the list nodes from the beginning up to the point of insertion or removal, they have a high probability of traversing a node that is concurrently updated by another thread. This effect is more visible when the rate of updates is highest. The results for the DSTM2 implementations place the Shadow factory above the ofree, which is consistent with the results presented in [Herlihy et al., 2006]. These results show, also, the overheads of generic transactions when compared to read-only transactions. Even though the JVSTM with the speculative transactions has to restart all the transactions erroneously assumed to be read-only, that is largely compensated by the gains obtained in all the tests, except when we have only 1 thread with 100% updates. Finally, an interesting result is that, in many cases, the JVSTM with speculative transactions performs even better than the JVSTM-ROH because, in reality, only half of the update operations are updating transactions. So, whereas the JVSTM-ROH uses a generic read-write transaction for all the update operations, the JVSTM with speculative transactions tries always a read-only transaction first, which proves to be true in at least half of the cases (for a workload of 100%).

7.2.2.2

Results for the Red-Black Tree Benchmark

I show, in Table 7.10 through Table 7.13 the results for the Red-Black Tree Benchmark. These values are depicted graphically, also, in Figure 7.14 through Figure 7.17. Like in the previous case, the JVSTM performs much better than both of the DSTM2 implementations, but is much worse than the simple lock approach. In this case, however,

186

Validation

Transactions/second by number of threads Factory Lock JVSTM JVSTM-ROH Speculative Shadow ofree

1

2

4

8

16

32

1440942 182191 200083 98344 2146 1986

685616 255390 278694 170429 2321 326

1323005 181625 182988 229593 2363 42

1320038 163095 163429 135732 2468 37

1322938 147826 151694 126116 2470 55

1323574 140170 148615 113520 2377 155

Table 7.11: The results for the Red-Black Tree Benchmark with 50% of updates. Transactions/second by number of threads Factory Lock JVSTM JVSTM-ROH Speculative Shadow ofree

1

2

4

8

16

32

1596168 304063 405617 283056 7619 6450

898924 451420 616440 469520 3005 1147

1360411 607360 898527 751179 3391 77

1465382 403839 519580 440848 5719 184

1456302 366217 465813 400654 5451 335

1470091 354499 450957 383148 5886 947

Table 7.12: The results for the Red-Black Tree Benchmark with 10% of updates. Transactions/second by number of threads Factory Lock JVSTM JVSTM-ROH Speculative Shadow ofree

1

2

4

8

16

32

1752592 400369 601922 597068 70686 29887

1180457 626557 1004470 1024964 86704 48232

1557793 860508 1576576 1563086 105488 68262

1680455 842556 1605674 1530231 83349 65963

1661551 886790 1614081 1600315 86818 62677

1627463 908179 1608275 1595123 85619 65050

Table 7.13: The results for the Red-Black Tree Benchmark with 0% of updates. Red-Black Tree Benchmark (100% updates)

1000000

Lock

Transactions/second

100000 10000

Speculative

1000 Shadow 100 10 ofree 1 1

2

4

8

16

32

Number of threads

Figure 7.14: Transactions per second processed by each method for the Red-Black Tree Benchmark with 100% of updates.

7.2 JVSTM Performance

187

Red-Black Tree Benchmark (50% updates)

1000000

Lock

Transactions/second

100000

Speculative

10000 1000

Shadow 100 10

ofree

1 1

2

4

8

16

32

Number of threads

Figure 7.15: Transactions per second processed by each method for the Red-Black Tree Benchmark with 50% of updates.

Red-Black Tree Benchmark (10% updates)

1000000

Lock

Transactions/second

100000

Speculative 10000 Shadow

1000 100

ofree

10 1 1

2

4

8

16

32

Number of threads

Figure 7.16: Transactions per second processed by each method for the Red-Black Tree Benchmark with 10% of updates.

188

Validation

Red-Black Tree Benchmark (0% updates)

1000000

Lock Speculative

Transactions/second

100000

Shadow

10000

ofree 1000 100 10 1 1

2

4

8

16

32

Number of threads

Figure 7.17: Transactions per second processed by each method for the Red-Black Tree Benchmark with 0% of updates.

there is a large difference between different workloads. For write-dominated workloads the JVSTM starts to degrade its performance with 4 threads and the overall throughput is much lower than for read-dominated workloads. The problem with this benchmark is that most of the changes in the set cause a rebalancing of the tree that may propagate often to the root of the tree, thereby creating a contention point in the data structure. This single point of contention is, also, the reason for the disastrous performance of the ofree: As this implementation uses visible reads and all the transactions read the root of the tree, a transaction that is trying to change that node either backs off or it aborts the readers; in either case, no progress is made. This particular problem does not affect the JVSTM, given that reads do not perturb nor are perturbed by the remaining of the system. In fact, the results obtained by the JVSTM for this benchmark when we have only 10% of updates are the best among all the three benchmarks. So, for read-dominated workloads, a red-black tree may be a good fit for the JVSTM.

7.2.2.3

Results for the Skip List Benchmark

Finally, I show, in Table 7.14 through Table 7.17 the results for the Skip List Benchmark. These values are depicted graphically, also, in Figure 7.18 through Figure 7.21. This is the first benchmark where the JVSTM performs better than the lock-based approach. In particular, for all the workloads except the one with 100% of updates, the JVSTM has a higher transaction rate with 4 threads than the lock-based approach.

7.2 JVSTM Performance

189

Transactions/second by number of threads Factory Lock JVSTM JVSTM-ROH Speculative Shadow ofree

1

2

4

8

16

32

305121 68128 68905 47857 1965 1768

158993 102747 102593 80641 396 520

163357 103485 103825 128983 68 61

170373 75967 72128 68326 49 36

166545 64742 66043 55606 48 37

198497 59258 62111 53011 46 37

Table 7.14: The results for the Skip List Benchmark with 100% of updates.

Transactions/second by number of threads Factory Lock JVSTM JVSTM-ROH Speculative Shadow ofree

1

2

4

8

16

32

318502 77654 92628 68632 3269 2896

172220 128398 158112 129393 692 1017

175241 180405 212225 215560 156 102

182282 108499 123948 114377 94 83

212797 94706 103584 94131 77 64

225285 89580 97521 85970 76 106

Table 7.15: The results for the Skip List Benchmark with 50% of updates.

Transactions/second by number of threads Factory Lock JVSTM JVSTM-ROH Speculative Shadow ofree

1

2

4

8

16

32

359652 99094 144467 132659 9876 6975

214196 160091 271900 236497 3079 3421

202393 235781 466776 440954 635 388

215050 158337 247383 230325 347 298

229224 139398 227163 211212 384 456

255033 133906 220600 203021 500 553

Table 7.16: The results for the Skip List Benchmark with 10% of updates.

Transactions/second by number of threads Factory Lock JVSTM JVSTM-ROH Speculative Shadow ofree

1

2

4

8

16

32

444574 138673 239763 224471 48369 18092

240094 227342 421094 438824 39520 25335

277495 307955 805865 782900 45951 35360

276243 321226 825336 773491 34333 32569

296080 319811 822072 806197 35713 31752

312701 328089 842003 798779 32099 31589

Table 7.17: The results for the Skip List Benchmark with 0% of updates.

190

Validation

Skip List Benchmark (100% updates)

1000000

Transactions/second

100000 Lock

10000

Speculative

1000 100 10

Shadow ofree

1 1

2

4

8

16

32

Number of threads

Figure 7.18: Transactions per second processed by each method for the Skip List Benchmark with 100% of updates.

Skip List Benchmark (50% updates)

1000000

Transactions/second

100000 Lock Speculative

10000 1000 100

Shadow ofree

10 1 1

2

4

8

16

32

Number of threads

Figure 7.19: Transactions per second processed by each method for the Skip List Benchmark with 50% of updates.

7.2 JVSTM Performance

191

Skip List Benchmark (10% updates)

1000000 100000 Transactions/second

Lock Speculative

10000 1000 100

Shadow ofree

10 1 1

2

4

8

16

32

Number of threads

Figure 7.20: Transactions per second processed by each method for the Skip List Benchmark with 10% of updates.

Skip List Benchmark (0% updates)

1000000

Transactions/second

100000

Speculative Lock

10000

Shadow ofree

1000 100 10 1 1

2

4

8

16

32

Number of threads

Figure 7.21: Transactions per second processed by each method for the Skip List Benchmark with 0% of updates.

192

Validation

7.2.3

Results for the STMBench7 Benchmark

The STMBench7 benchmark was proposed recently by Guerraoui et al. [2007] as a more realistic benchmark for evaluating STM implementations. This benchmark is an adaptation of the OO7 benchmark [Carey, DeWitt, and Naughton, 1993]: It maintained the underlying data structure of the original benchmark, but removed the database-related parts and added a set of new operations that allow for a more adequate evaluation of STMs.

7.2.3.1

The STMBench7 Data Structure

The data structure of the STMBench7 benchmark consists in a large number of objects of several different types—modules, assemblies, composite parts, atomic parts, connections, documents, and manuals—which are organized as follows: There is a single module that contains a seven-level-deep tree of assemblies, where each level of the tree has tree children; each of the leaves of this tree contains several composite parts, which, in turn, have a document and a graph of atomic parts which are connected via connection objects. All these objects, which are called generically design-library objects, are related with one another either by using simple references between them, or by using collections of objects. The benchmark starts with the creation of this data structure, using a pair of factories: (1) a factory for creating each of the design-library objects, and (2) a factory for creating the sets, bags, and indexes needed to relate the objects with one another. The factories are used by the STMBench7 benchmark to allow that different STM implementations provide their own version of the elements that compose the data structure. Therefore, I created one implementation of each of these factories using the JVSTM. For implementing the design-library objects, I used a vbox for each of their fields and maintained all the remaining code, except that where a field access was made, there is now a vbox operation. For implementing the collections, I used a simple approach: I implemented two purely functional data structures, which are, thus, thread-safe, and used a single vbox to hold an instance of the collection. So, any operation that changes a collection creates a new instance of the collection and changes the vbox to that new value. The two purely functional data structures that I used to implement the collections of the STMBench7 benchmark were: (1) a single linked list, and (2) a red-black tree that follows the implementation described in [Okasaki, 1998].

7.2 JVSTM Performance

7.2.3.2

The STMBench7 Operations

To operate on its data structure, the STMBench7 benchmark implements 45 distinct operations, which are organized across two different dimensions: the operation category, and whether the operations are read-only or read-write. The four categories for the operations are: • Long traversals. These operations traverse most of the objects in the data structure, starting either at the module and going top-down, or starting at an atomic part and going bottom-up. These are the lengthier operations in the benchmark. • Short traversals. These operations are similar to the previous ones, except that they traverse a smaller number of objects. • Short operations. These operations access only a few objects and perform some operation on those objects. • Structure modification operations. These operations make changes in the data structure, by creating or deleting elements. In this category there are no read-only operations.

7.2.3.3

The STMBench7 Execution Parameters and Results

The STMBench7 benchmark allows us to specify, for each run, the number of threads to use, the type of workload pretended, which synchronization strategy to use, and for how long should each thread run; additionally, it allows us to disable certain categories of operations. There are three predefined workload values in the STMBench7, corresponding to different splits between read-only and read-write operations: read-dominated (90% reads/10% writes), read-write (60% reads/40% writes), and write-dominated (10% reads/90% writes). The STMBench7 assigns a probability for each of its operations by taking into account the workload type chosen and a predefined ratio assigned to each category type. This probability is then used by each thread to choose randomly the sequence of operations to execute. Regarding the choice of synchronization strategy, the STMBench7 benchmark comes already with two locking strategies implemented: • A coarse-grained locking strategy that uses a single read-write lock for the entire data structure. • A medium-grained locking strategy that uses one read-write lock for each level of the data structure.

193

194

Validation

The intended semantics for the synchronization strategies is that each operation executes atomically. Thus, to test the JVSTM, I implemented another synchronization strategy that wraps each operation with a JVSTM transaction. Moreover, as in the STMBench7 benchmark we know whether each operation is read-only or not, the JVSTM strategy uses that information to create a transaction of the appropriate type. As for the operations to use, the original version of the STMBench7 allows us to disable, independently, all the long traversals, and all the structure modification operations. I extended it to allow us to disable, also, the read-write long traversals (but leaving the read-only long traversals active). Finally, there are two types of results produced at the end of a run: (1) the total throughput of the benchmark, measured in number of operations per second; and (2) the maximum latency for each type of operation. According to the authors of the benchmark, there are two typical uses for it: either we run the benchmark with all the operations enabled to measure the latency of the operations, or we run the benchmark with no long traversals active to measure the throughput.

7.2.3.4

Experimental Setup

I ran a series of tests varying the synchronization strategy, the number of threads, the workload type, and the mix of operations. Given that the latency results of the benchmark may be significantly influenced by the random execution of the JIT compiler of the Java virtual machine, I changed the benchmark so that it runs for 60 seconds in the beginning to warm up the virtual machine; after that initial warm up, the benchmark runs for the specified amount of time and, obviously, the results are measured only for that part of the run. The results shown below were obtained by running each test for 60 seconds. I show results for each of the three workload types and synchronization strategies, varying the number of threads through the values 1, 2, 4, 8, and 16. The throughput results were obtained for two mixes of operations: one with all the long traversals disabled, and another with only the read-write long traversals disabled. The latency results were obtained with all the operations enabled.

7.2.3.5

Throughput Results

I show the results obtained for the throughput tests with all the long traversals disabled in Table 7.18 through Table 7.20, where each table is for a particular workload type. Also, I show these values graphically in Figure 7.22 through Figure 7.24.

7.2 JVSTM Performance

195

Operations/second by number of threads Synchronization strategy Coarse-grained locking Medium-grained locking JVSTM

1

2

4

8

16

32

2362 2447 1632

2948 3269 2585

3302 4189 4260

2838 4080 2297

2673 3988 1835

2637 3876 1495

Table 7.18: The results of the STMBench7 benchmark with all the long traversals disabled and a read-dominated workload. Operations/second by number of threads Synchronization strategy Coarse-grained locking Medium-grained locking JVSTM

1

2

4

8

16

32

1569 1528 947

1418 1698 1362

1498 1887 1453

1415 1769 1007

1459 1846 667

1489 1692 441

Table 7.19: The results of the STMBench7 benchmark with all the long traversals disabled and a read-write workload. Operations/second by number of threads Synchronization strategy Coarse-grained locking Medium-grained locking JVSTM

1

2

4

8

16

32

936 908 578

925 945 652

875 972 620

863 970 427

821 909 316

844 901 234

Table 7.20: The results of the STMBench7 benchmark with all the long traversals disabled and a write-dominated workload. No long traversals / Read-dominated

4260

Operations/second

medium 3195 coarse 2130 JVSTM 1065

0 1

2

4

8

16

32

Number of threads

Figure 7.22: Operations per second processed by each synchronization strategy for the STMBench7 benchmark with all the long traversals disabled and a readdominated workload.

196

Validation

No long traversals / Read-write

1887

Operations/second

medium coarse

1415

944

472

JVSTM

0 1

2

4

8

16

32

Number of threads

Figure 7.23: Operations per second processed by each synchronization strategy for the STMBench7 benchmark with all the long traversals disabled and a read-write workload.

No long traversals / Write-dominated

972

Operations/second

medium coarse 729

486

243

JVSTM

0 1

2

4

8

16

32

Number of threads

Figure 7.24: Operations per second processed by each synchronization strategy for the STMBench7 benchmark with all the long traversals disabled and a writedominated workload.

7.2 JVSTM Performance

197

Operations/second by number of threads Synchronization strategy Coarse-grained locking Medium-grained locking JVSTM

1

2

4

8

16

32

131 135 101

163 172 198

139 193 265

131 185 305

140 191 306

172 237 359

Table 7.21: The results of the STMBench7 benchmark with all the read-write long traversals disabled and a read-dominated workload.

Operations/second by number of threads Synchronization strategy Coarse-grained locking Medium-grained locking JVSTM

1

2

4

8

16

32

398 427 130

275 320 253

308 631 991

372 558 514

586 524 413

430 385 359

Table 7.22: The results of the STMBench7 benchmark with all the read-write long traversals disabled and a read-write workload.

These results show that, even though the JVSTM performs worse than either of the locking strategies with one thread, it scales much better than the locking strategies. In fact, for a read-dominated workload and four threads the JVSTM is clearly better than the coarse-grained locking and slightly better than the medium-grained locking. Also, from the shape of the plot, it is reasonable to expect that this difference would become more significant with more processors. Unfortunately, the performance of the JVSTM decreases abruptly when we have more threads than processors. This result is caused in part by the fact that the STMBench7 is not concurrency friendly—that is, there is lots of contention in the benchmark. Thus, as the number of threads (and consequently the number of transactions) increases, the probability of a conflict increases dramatically. Not only because it increases the probability that a concurrent write transaction exists, but also because the duration of each transaction increases. Nevertheless, these results are very encouraging for the JVSTM. Specially, when compared with the results reported by the authors of the STMBench7 benchmark in their paper for an implementation of another STM—the ASTM. According to those results, the ASTM cannot achieve the 10 operations per second in this benchmark, whereas the results presented for the locking strategy are very similar to those obtained by me. So, it is reasonable to say that the JVSTM performs at least one to two orders of magnitude better than the implementation used by the authors of the STMBench7. In fact, whereas Guerraoui et al. report that the ASTM takes roughly half an hour to execute one of the long traversals (namely, T1), when running with only one thread, the JVSTM executes that operation in less than 1.4 seconds in my experiments.

198

Validation

Operations/second by number of threads Synchronization strategy Coarse-grained locking Medium-grained locking JVSTM

1

2

4

8

16

32

822 803 470

844 949 543

873 923 603

859 893 427

845 837 315

839 863 229

Table 7.23: The results of the STMBench7 benchmark with all the read-write long traversals disabled and a write-dominated workload.

No read-write long traversals / Read-dominated

JVSTM

Operations/second

359

269 medium 180

coarse

90

0 1

2

4

8

16

32

Number of threads

Figure 7.25: Operations per second processed by each synchronization strategy for the STMBench7 benchmark with all the read-write long traversals disabled and a read-dominated workload.

7.2 JVSTM Performance

199

No read-write long traversals / Read-write

Operations/second

991

743

495 coarse medium JVSTM 248

0 1

2

4

8

16

32

Number of threads

Figure 7.26: Operations per second processed by each synchronization strategy for the STMBench7 benchmark with all the read-write long traversals disabled and a read-write workload.

No read-write long traversals / Write-dominated

949

Operations/second

medium coarse 712

475

237

JVSTM

0 1

2

4

8

16

32

Number of threads

Figure 7.27: Operations per second processed by each synchronization strategy for the STMBench7 benchmark with all the read-write long traversals disabled and a write-dominated workload.

200

Validation

Max latency by number of threads (ms) Synchronization strategy

Read-only oper.

Coarse-grained locking

Short Short Short Short Short Short

Medium-grained locking JVSTM

traversals operations traversals operations traversals operations

1

2

4

8

16

11 3 9 3 2 4

3564 4011 5891 5720 2 5

1702 2427 2500 4011 7 3

4256 4383 4256 4256 222 303

4515 4515 10960 10330 393 6068

Table 7.24: Maximum latency results for read-only short traversals and short operations with all the operations enabled and a read-dominated workload.

Max latency by number of threads (ms) Synchronization strategy

Read-only oper.

1

2

4

8

16

Coarse-grained locking

Short Short Short Short Short Short

1 3 8 3 4 7

4650 3167 1194 3359 2 3

5082 5082 1557 3564 11 4

4515 4650 6068 6068 186 436

7463 7463 7463 7687 34 115

Medium-grained locking JVSTM

traversals operations traversals operations traversals operations

Table 7.25: Maximum latency results for read-only short traversals and short operations with all the operations enabled and a read-write workload.

Finally, I show both in Table 7.21 through Table 7.23, and in Figure 7.25 through Figure 7.27, the results obtained when only the read-write long traversals are disabled. In this case, the results for the JVSTM are even better, as it performs significantly better than both the locking strategies except in the case of a write-dominated workload. These results validate the suitability of the JVSTM for workloads with long read-only operations.

7.2.3.6

Latency Results

To conclude the evaluation of the JVSTM performance, I show, in Table 7.24 through Table 7.26, the results of the maximum latency for all the read-only short traversals and short operations, for each of the various workloads, when all the operations are enabled. These results show that, in the JVSTM, the execution of read-only transactions is not affected by the remaining of the system. Whereas for the locking strategies there is a significant increase of the maximum latency when the number of threads increases, that phenomenon does not occur with the JVSTM. The problem with the locking strategies is that the reads must wait that a lengthy write transaction (a long write traversal, typically) finishes before they may proceed. In the JVSTM, however, the reads never need to wait.

7.3 Summary

201

Max latency by number of threads (ms) Synchronization strategy

Read-only oper.

1

2

4

8

16

Coarse-grained locking

Short Short Short Short Short Short

2 3 1 3 4 2

1652 569 489 474 0 3

5391 5720 1343 1030 1 34

5391 4011 3895 3564 6 6

4650 4515 6437 6437 4790 15

Medium-grained locking JVSTM

traversals operations traversals operations traversals operations

Table 7.26: Maximum latency results for read-only short traversals and short operations with all the operations enabled and a write-dominated workload.

7.3

Summary

This chapter presented the application of some of the proposals made in this dissertation to a real-world large application—the Fénix system—which is developed by a team of more than a dozen of programmers. The results obtained with this application show the effectiveness of the proposals made in this dissertation, both in the reduction of problems related to the implementation of a rich domain model, and the adequacy of the proposals for real-world usage. It presents, also, a brief performance evaluation of the JVSTM using more standard benchmarks in the area of STMs. The results of this performance evaluation show that, even though the JVSTM was not specifically designed as a high-performance STM implementation, it performs very well either in comparison with some of the best-known STM implementations for Java, or in comparison with a baseline traditional locking strategy. More specifically, the JVSTM is one to two orders of magnitude faster than the DSTM2 and an implementation of the ASTM, as reported in [Guerraoui et al., 2007].

202

Validation

Chapter 8

Conclusions In this concluding chapter, I present the main contributions of this dissertation and discuss some of the new research avenues that this work opens.

8.1

Main Contributions

The primary goal of the work presented in this dissertation was to simplify the implementation of rich object-oriented domain models. Moreover, that this should be accomplished in a programmers’ friendly way—that is, in such a way that it is not only easily comprehensible by an average programmer, but also that it is practical to use in current software development. I achieved this goal, because throughout this dissertation I identified some of the problems with the existing approaches, I proposed solutions to those problems, and I validated those solutions. So, the achievement of this goal is the primary contribution of my work. Yet, for achieving this goal, I made more specific contributions in several of the areas addressed by this work. In the area of Software Transactional Memory, I distinguish the following main contributions:

• I proposed a new model of Software Transactional Memory that keeps multiple versions for each transactional location. Even though the idea of multi-version concurrency control is not new, this was the first application of this idea to the area of Software Transactional Memory. This new multi-version model allows read operations to execute both with less overheads and with no synchronization with the remaining threads of a system. Moreover, this model provides the guarantee that read-only transactions never fail.

204

Conclusions

• I described in detail an implementation of the versioned STM model that is simple to use, simple to implement without extra support from the execution runtime, and, that, at the same time, performs one to two orders of magnitude better than some of the best-known implementations of STM in Java. • I presented a lock-free algorithm for garbage-collecting the old values made unreachable during the execution of a versioned STM. This algorithm frees the old values without having to perform costly sweepings of the memory, by leveraging on the intimate knowledge about the workings of the STM that creates and make inaccessible the transactional values. Even though at present this algorithm is implemented as part of a Java library and, therefore, cannot intervene directly in the garbage collection process of the underlying Java runtime, it is reasonable to assume that this algorithm would be easily integrated into the Java runtime garbage collection process with significant advantages for that process, if the JVSTM would be implemented at the level of the Java virtual machine.

In the area of object-oriented programming, I distinguish the following main contributions:

• I proposed a practical approach for integrating new programming constructs into existing programming languages and practices. This approach is based on a set of software-engineering-motivated guiding principles introduced in Section 1.2.2. I argue that newly proposed programming constructs should integrate seamlessly not only with existing programming languages, but also with the tools and practices that programmers use, so that the new constructs may be readily adopted by programmers already trained in the existing programming languages. • I introduced the DML language as a language that allows the specification, in a declarative way, of a domain model’s structure, and that integrates seamlessly with the Java programming language. This language is simple to learn, not only because its syntax is purposefully quite similar to the Java’s syntax, but also because its constructs correspond to well-established concepts used at the modeling level. In particular, the DML supports the specification of entities and associations between entities, both as first class constructs in the language. Even though this has been done already over the years for various languages, the novelty of the DML is that it is neither an extension of an existing language, nor an entirely new general-purpose language meant to replace the existing languages. Rather, it is limited in expressiveness by design, and built to integrate with a mainstream programming language. Its use in the development of a large real-world system—the Fénix system—and the readiness of its adoption by the programmers of that system demonstrates the effectiveness of this approach. • I proposed, also, as part of the implementation of a domain specification described

8.2 Future Research

in the DML language, a new pattern for implementing associations in an objectoriented programming language. Compared to the existing patterns for implementing associations, my proposal has the advantage of being simpler to implement (once we have all the supporting classes implemented). Furthermore, it provides a friendlier interface to the programmer that is now able to create or remove association links in several different ways, including through the use of the familiar interfaces of the Java Collection Framework. This new pattern allows, also, that third-party programmers—that is, programmers that do not have access to the code implementing the association—customize the behavior of the associations operations. • I introduced the idea of using consistency predicates to separate the implementation of a domain model’s constraints from the code that updates the state of the domain entities. By doing so, we reduce the code scattering, the code tangling, and the strong coupling that results from the usual approach of implementing these two concerns together. Moreover, this approach facilitates the composition of objects in a truly object-oriented spirit, by allowing the composition of existing objects without having to adapt them specifically to the new composition. This is made possible by having the verification of the consistency predicates separate from the remaining of the code, so that the composing object may control when will the consistency predicates be evaluated, without knowledge or consent from the composed object. • I described an implementation of consistency predicates in the JVSTM that is sufficiently generic to be used without significant changes by most, if not all, other STM implementations. This implementation does not require any significant change to the underlying STM implementation. Instead, it leverages on the support for atomicity given by STMs to implement the evaluation of consistency predicates and the necessary recording of dependencies.

Finally, in the area of software engineering, I contributed with a study of the typical transactional workloads of a large real-world web application—the Fénix system—which is representative of a large class of applications. This study was performed for an extended period of time and gives us valuable insights about the needs of such kind of applications, so that better solutions, suited to their specificities, may be developed.

8.2

Future Research

Even though the work described in this dissertation is self-contained, in the sense that it may be readily used without further developments, it does not constitute, quite naturally, the ultimate solution for any of the problems that it addresses. Rather on the contrary, the decision to make proposals that are simple to learn and to implement makes these

205

206

Conclusions

proposals particularly susceptible to being extensible in many various ways. In fact, I believe that it should be a goal of any research work to open up new research directions. Therefore, I hope that others, as I intend myself, will follow-up on the work that I describe in this dissertation. In particular, I envision that the following topics, presented in no particular order, will be pursued further in the future: • The implementation of the versioned STM at the virtual machine level. Implementing the versioned STM at the virtual machine level, rather than at the higher-level of the Java programming language, will allow for an entirely different set of design decisions regarding its implementation. In particular, it should be possible to reduce significantly the overhead incurred both in memory and in execution time by the use of vboxes. Furthermore, related to this approach, it would make sense to integrate the garbage collection of old unreachable values into the algorithms of garbage collection of the existing runtimes. • The development of a lock-free version of the commit of a versioned STM transaction. The commit operation is, in fact, embarrassingly parallel; so, a simple solution to remove the current locking implementation would be to make all the threads that want to commit to cooperate (or help) in the commit of other transactions. This simple idea, however, is not trivial to put in practice with good results, because other concerns, such as the amount of cache-coherency traffic generated by such an approach may drive the performance of the system down. Thus, it is necessary to investigate all the conflicting forces that may influence such a solution. • The development of data structures adequate for a versioned transactional memory. There is much work and accumulated knowledge in data structures for parallel computing, but the area of Transactional Memory introduces a significant amount of changes that may render most of that knowledge useless in determining which data structures are more adequate for a program that uses transactional memory. More specifically, to make STMs more widely used in programming at large, we must give programmers libraries of transactional classes such as collections, together with a set of recommendations on which collections are best for each task. For a versioned STM, in particular, a promising avenue of research is to study the applicability of all the work made in purely functional persistent data structures. • The reduction of conflicts for write transactions in the versioned STM. Even though the versioned STM is very good at eliminating conflicts for read-only transactions, its performance degrades abruptly when the amount of conflicts for write transactions increases. Thus, a much needed line of future research is to develop new alternatives to either reduce the conflicts, or to help in solving them. One possible direction to follow is the integration of contention management into the versioned

8.2 Future Research

STM, which many other STMs use; the specifics of a versioned model, however, may bring different design decisions into this integration. Another line of research worthy of consideration is the use of an arbitrarily fine-grained structure of nested transactions to allow the partial reexecution of a conflicting transaction. The idea, in this case, is to avoid the whole reexecution of a conflicting transaction by reexecuting (eventually, with help from other threads) only the parts that may have changed because of a conflict. • The extension of the DML language to support the specification of more aspects of a domain model’s structure. The DML, as presented in this dissertation, allows the representation of only a limited set of domain modeling constructs. Therefore, it is to be expected that in the future other constructs be added to the language. For instance, constructs to specify other kinds of associations, such as qualified associations, or constructs to specify the refinement of entities or associations. • The extension of the design pattern for implementing associations. The pattern introduced in this dissertation does not address other common requirements of associations, such as that they should be ordered, or sorted, or even indexed by a given attribute of one of the participating objects. Thus, an interesting and useful line of research work would be the extension of this pattern to support such things. • The interaction between consistency predicates and other programming language constructs. The integration of consistency predicates in a programming language interferes with several other constructs that may exist in the language. For instance, an obvious candidate is the system of exceptions of a language, given that consistency predicates may signal their failure through exceptions. Thus, it would be interesting to see whether the exceptions thrown by consistency predicates should deserve some kind of special treatment by the language. Or, even if not by the language, whether the distinction between exceptions thrown by consistency predicates and other types of exceptions would be useful for a programmer, from a pragmatical point of view. • More elaborate studies of the transactional workloads of real-world domain-intensive applications. The study performed in this dissertation was rather simple, and served mostly to validate the assumptions underlying the development of the versioned STM. Yet, knowing in more detail how do real-world large applications use transactions may provide valuable feedback into the development of transactional systems.

These topics are, by no means, an exhaustive list of all the research work that may follow from what is described in this dissertation. Rather, it is a list of some of the topics on which I had already some thoughts, and with which I would like to finish my dissertation.

207

208

Conclusions

Bibliography Akehurst, D., Howells, G., and McDonald-Maier, K. Implementing associations: UML 2.0 to Java 5. Software and Systems Modeling, volume 6(1):pages 3–35, 2007. Albano, A., Ghelli, G., and Orsini, R. A relationship mechanism for a strongly typed object-oriented database programming language. In Proceedings of the 17th International Conference on Very Large Data Bases, pages 565–575. 1991. Alur, D., Crupi, J., and Malks, D. Core J2EE Patterns: Best Practices and Design Strategies. Prentice-Hall, Inc., New Jersey, USA, 2001. ANSI and ITIC. American National Standard for information technology: programming language — Common LISP: ANSI X3.226-1994. American National Standards Institute, 1430 Broadway, New York, NY 10018, USA, 1996. Arnold, K., Gosling, J., and Holmes, D. The Java Programming Language. Addison-Wesley, Reading, Massachusetts, USA, third edition, 2000. Barnett, M., DeLine, R., Fähndrich, M., Rustan, K., Leino, M., and Schulte, W. Verification of object-oriented programs with invariants. Journal of Object Technology, volume 3(6):pages 27–56, 2004. Special issue: ECOOP 2003 workshop on FTfJP. Bartetzko, D. Parallelität und Vererbung beim "Programmieren mit Vertrag" — Weiterentwicklung von JaWA. Master’s thesis, Universität Oldenburg, 1999. Bass, L., Clements, P., and Kazman, R. Software Architecture in Practice. SEI Series in Software Engineering. Addison-Wesley, Reading, Massachusetts, USA, second edition, 2003. Bernstein, P. A. and Goodman, N. Multiversion concurrency control — theory and algorithms. ACM Transactions on Database Systems, volume 8(4):pages 465–483, 1983. Bierman, G. and Wren, A. First-class relationships in an object-oriented language. In Proceedings of the 19th European Conference on Object-Oriented Programming, volume 3586 of Lecture Notes in Computer Science, pages 262–286. Springer-Verlag, 2005. Booch, G. Object-Oriented Analysis and Design with Applications. Addison-Wesley, Reading, Massachusetts, USA, second edition, 1994.

210

BIBLIOGRAPHY

Booch, G., Rumbaugh, J., and Jacobson, I. The Unified Modeling Language User Guide. Addison-Wesley, 1999. Cachopo, J. and Rito-Silva, A.

Versioned boxes as the basis for memory transac-

tions. In Workshop on Synchronization and Concurrency in Object-Oriented Languages (SCOOL05). 2005. Available at http://hdl.handle.net/1802/2101. Cachopo, J. and Rito-Silva, A. Versioned boxes as the basis for memory transactions. Science of Computer Programming, volume 63(2):pages 172–185, 2006. Carey, M. J., DeWitt, D. J., and Naughton, J. F. The OO7 benchmark. SIGMOD Record (ACM Special Interest Group on Management of Data), volume 22(2):pages 12–21, 1993. Cattell, R. G. G., Barry, D. K., Berler, M., Eastman, J., Jordan, D., Russell, C., Schadow, O., Stanienda, T., and Velez, F., (Editors) The Object Data Standard – ODMG 3.0. Morgan Kaufmann Publishers, Inc., Los Altos, USA, 2000. Chen, P. P.-S. The entity-relationship model—toward a unified view of data. ACM Transactions on Database Systems, volume 1(1):pages 9–36, 1976. Clarke, D. G., Potter, J. M., and Noble, J. Ownership types for flexible alias protection. In Proceedings of the 13th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, SIGPLAN Notices, pages 48–64. ACM Press, 1998. Clements, P., Bachmann, F., Bass, L., Garlan, D., Ivers, J., Little, R., Nord, R., and Stafford, J. Documenting Software Architectures: Views and Beyond. SEI Series in Software Engineering. Addison-Wesley, Reading, Massachusetts, USA, 2003. Coplien, J. O. Advanced C++ Programming Styles and Idioms. Addison-Wesley, Reading, Massachusetts, USA, 1992. Czarnecki, K. and Eisenecker, U. W. Generative Programming: Methods, Tools, and Applications. Addison-Wesley, Reading, Massachusetts, USA, 2000. Dietl, W. and Müller, P. Universes: Lightweight ownership for JML. Journal of Object Technology, volume 4(8):pages 5–32, 2005. DSTM2. Dynamic Software Transactional Memory Library 2.0. Visited in 2007. Home page at http://www.sun.com/download/products.xml?id=453fb28e. Evans, E. Domain-Driven Design: Tackling Complexity in the Heart of Software. AddisonWesley, Reading, Massachusetts, USA, 2003. Evermann, J. and Wand, Y. Toward formalizing domain modeling semantics in language syntax. IEEE Transactions on Software Engineering, volume 31(1):pages 21–37, 2005. FenixEDU. FénixEDU. 2005. Home page at http://fenixedu.sourceforge.net.

BIBLIOGRAPHY

Flanagan, C., Leino, K. R. M., Lillibridge, M., Nelson, G., Saxe, J. B., and Stata, R. Extended static checking for java. SIGPLAN Not., volume 37(5):pages 234–245, 2002. Fowler, M. Patterns of Enterprise Application Architecture. Addison-Wesley, Reading, Massachusetts, USA, 2002. Fowler, M., Beck, K., Brant, J., Opdyke, W., and Roberts, D. Refactoring: Improving the Design of Existing Code. Addison-Wesley, 1999. France, R. B., Ghosh, S., Dinh-Trong, T., and Solberg, A. Model-driven development using uml 2.0: Promises and pitfalls. Computer, volume 39(2):pages 59–66, 2006. Gabriel, R. P., Northrop, L., Schmidt, D. C., and Sullivan, K. Ultra-large-scale systems. In Companion to the 21st ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, SIGPLAN Notices, pages 632–634. ACM Press, 2006. Gamma, E., Helm, R., Johnson, R., and Vlissides, J. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, Reading, Massachusetts, USA, 1995. Génova, G., del Castillo, C. R., and Llorens, J. Mapping UML associations into java code. Journal of Object Technology, volume 2(5):pages 135–162, 2003. Génova, G., Llorens, J., and Martínez, P. The meaning of multiplicity of n-ary associations in UML. Software and Systems Modeling, volume 1(2):pages 86–97, 2002. Gosling, J., Joy, B., Steele, G., and Bracha, G. The Java Language Specification. AddisonWesley, Reading, Massachusetts, USA, third edition, 2005. Graham, P. and Barker, K. Effective optimistic concurrency control in multiversion object bases. In Proceedings of the International Symposium on Object-Oriented Methodologies and Systems, volume 858, pages 313–328. Springer-Verlag, 1994. Gueheneuc, Y.-G. and Albin-Amiot, H. A pragmatic study of binary class relationships. In Proceedings of the 18th IEEE International Conference on Automated Software Engineering, pages 277–280. 2003. Guéhéneuc, Y.-G. and Albin-Amiot, H. Recovering binary class relationships: putting icing on the UML cake. In Proceedings of the 19th ACM SIGPLAN Conference on ObjectOriented Programming, Systems, Languages, and Applications, volume 39 of SIGPLAN Notices, pages 301–314. ACM Press, 2004. Guerraoui, R., Herlihy, M., and Pochon, S. Toward a theory of transactional contention management. In Proceedings of the 24nd Annual ACM Symposium on Principles of Distributed Computing. ACM Press, 2005.

211

212

BIBLIOGRAPHY

Guerraoui, R., Kapalka, M., and Vitek, J. STMBench7: A benchmark for software transactional memory. In Proceedings of the Second European Systems Conference. 2007. Hailpern, B. and Tarr, P. Model-driven development: the good, the bad, and the ugly. IBM Systems Journal, volume 45(3):pages 451–461, 2006. Harris, T. and Fraser, K. Language support for lightweight transactions. In Proceedings of the 18th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, volume 36 of SIGPLAN Notices, pages 388–402. ACM Press, 2003. Harris, T., Marlowe, S., Peyton-Jones, S., and Herlihy, M. Composable memory transactions. In Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. ACM Press, 2005. Harris, T. and Peyton-Jones, S. Transactional memory with data invariants. In First ACM SIGPLAN Workshop on Languages, Compilers, and Hardware Support for Transactional Computing. 2006. Harrison, W., Barton, C., and Raghavachari, M. Mapping UML designs to Java. In Proceedings of the 15th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, volume 35 of SIGPLAN Notices, pages 178–187. ACM Press, 2000. Herlihy, M., Luchangco, V., and Moir, M. A flexible framework for implementing software transactional memory. In Proceedings of the 21st Annual ACM SIGPLAN Conference on Object-Oriented Programing, Systems, Languages, and Applications, pages 253–262. ACM Press, 2006. Herlihy, M., Luchangco, V., Moir, M., and Scherer, III., W. N. Software transactional memory for dynamic-sized data structures. In Proceedings of the 22nd Annual ACM Symposium on Principles of Distributed Computing, pages 92–101. ACM Press, 2003. Herlihy, M. and Moss, J. E. B. Transactional memory: Architectural support for lockfree data structures. In Proceedings of the 20th Annual International Symposium on Computer Architecture. 1993. Herlihy, M. P. Wait-free synchronization. ACM Transactions on Programming Languages and Systems, volume 13(1):pages 124–149, 1991. Herlihy, M. P. and Wing, J. M. Linearizability: a correctness condition for concurrent objects. ACM Transactions on Programming Languages and Systems, volume 12(3):pages 463–492, 1990. Hoare, C. A. R. Proof of correctness of data representations. Acta Informatica, volume 1:pages 271–281, 1972.

BIBLIOGRAPHY

Iscoe, N., Williams, G. B., and Arango, G. Domain modeling for software engineering. In Proceedings of the 13th International Conference on Software Engineering, pages 340– 343. IEEE Computer Society, 1991. Joy, B., Guy L. Steele, J., Gosling, J., and Bracha, G. The Java Language Specification. Addison-Wesley, Reading, Massachusetts, USA, second edition, 2000. JVSTM. JVSTM. 2005. Home page at http://www.esw.inesc-id.pt/~jcachopo/

jvstm. Karaorman, M., Hölzle, U., and Bruno, J. jContractor: A reflective java library to support Design by Contract. In Proceedings of the 2nd International Conference on Meta-Level Architectures and Reflection, number 1616 in Lecture Notes in Computer Science, pages 175–196. Springer-Verlag, 1999. Kiczales, G., Lamping, J., Menhdhekar, A., Maeda, C., Lopes, C., Loingtier, J.-M., and Irwin, J. Aspect-oriented programming. In M. Aksit and S. Matsuoka, (Editors) Proceedings of the 11th European Conference on Object-Oriented Programming, volume 1241 of Lecture Notes in Computer Science, pages 220–242. Springer-Verlag, 1997. Kramer, R. iContract — the Java Design by Contract tool. In Proceedings of the TOOLS’98: Technology of Object-Oriented Languages and Systems, pages 295–307. IEEE Computer Society, 1998. Lackner, M., Krall, A., and Puntigam, F. Supporting design by contract in java. Journal of Object Technology, volume 1(3):pages 57–76, 2002. Special issue: TOOLS USA 2002 proceedings. Leavens, G. T., Cheon, Y., Clifton, C., Ruby, C., and Cok, D. R. How the design of JML accommodates both runtime assertion checking and formal verification. Science of Computer Programming, volume 55:pages 185–208, 2005. Leavens, G. T., Ruby, C., Rustan, K., Leino, M., Poll, E., and Jacobs, B. Jml (poster session): notations and tools supporting detailed design in java. In OOPSLA ’00: Addendum to the 2000 proceedings of the conference on Object-oriented programming, systems, languages, and applications (Addendum), pages 105–106. ACM Press, 2000. Manson, J., Pugh, W., and Adve, S. V. The java memory model. In Conference Record of the 32th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages, pages 378–391. ACM Press, 2005. Marathe, V. J., Scherer, W. N., and Scott, M. L. Design tradeoffs in modern software transactional memory systems. In Proceedings of the 7th Workshop on Languages, Compilers, and Run-Time Support for Scalable Systems, pages 1–7. ACM Press, 2004. Marathe, V. J. and Scott, M. L. A qualitative survey of modern software transactional memory systems. Technical Report UR CSD;TR 839, 2004.

213

214

BIBLIOGRAPHY

Mellor, S. J., Scott, K., Uhl, A., and Weise, D. MDA Distilled. Addison-Wesley, Reading, Massachusetts, USA, 2004. Meyer, B. Object-Oriented Software Construction. Prentice-Hall, Inc., New Jersey, USA, 1988. Meyer, B. Applying design by contract. Computer, volume 25(10):pages 40–51, 1992a. Meyer, B. Eiffel: The Language. Prentice-Hall, Inc., New Jersey, USA, 1992b. Moss, J. E. B. and Hosking, A. L. Nested transactional memory: Model and preliminary architecture sketches.

In Workshop on Synchronization and Concurrency in

Object-Oriented Languages (SCOOL05). 2005.

Available at http://hdl.handle.

net/1802/2099. Müller, P., Poetzsch-Heffter, A., and Leavens, G. T. Modular invariants for layered object structures. Science of Computer Programming, volume 62(3):pages 253–286, 2006. Noble, J. Basic relationship patterns. volume 4, chapter 6, pages 73–94. Addison-Wesley, Reading, Massachusetts, USA, 2000. Noble, J. and Grundy, J. Explicit relationships in object-oriented development. 1995. Object Management Group. Unified modeling language: Superstructure (version 2.0). Available at http://www.omg.org/cgi-bin/doc?formal/05-07-04, Visited in 2007. OJB. Object/Relational Bridge – OJB. Visited in 2007. Home page at http://db.

apache.org/ojb. Okasaki, C. Purely Functional Data Structures. Cambridge University Press, Cambridge, MA, USA, 1998. OMG. OMG Model Driven Architecture. 2007. Home page at http://www.omg.org/

mda. Parnas, D. L. On the criteria to be used in decomposing systems into modules. Communications of the ACM, volume 15(12):pages 1053–1058, 1972. Pearce, D. J. and Noble, J. Relationship aspects. In Proceedings of the 5th International Conference on Aspect-Oriented Software Development, pages 75–86. ACM Press, 2006. Polak, B., (Editor) Ultra-Large-Scale Systems: The Software Challenge of the Future. Software Engineering Institute, Carnegie Mellon, Pittsburgh, USA, 2006.

Available at

http://www.sei.cmu.edu/uls/. Pugh, W. Fixing the java memory model. In Proceedings of the ACM 1999 Conference on Java Grande, pages 89–98. ACM Press, 1999.

BIBLIOGRAPHY

215

Raistrick, C., Francis, P., and Wright, J. Model Driven Architecture with Executable UML. Cambridge University Press, Cambridge, MA, USA, 2004. Reed, D. P. Naming and Synchronization in a Decentralized Computer System. Ph.D. thesis, MIT, Cambridge, MA, USA, 1978. Reed, D. P. Implementing atomic actions on decentralized data. ACM Transactions on Computer Systems, volume 1(1):pages 3–23, 1983. Riegel, T., Felber, P., and Fetzer, C. A lazy snapshot algorithm with eager validation. In 20th International Symposium on Distributed Computing (DISC). 2006a. Riegel, T., Fetzer, C., and Felber, P. Snapshot isolation for software transactional memory. In First ACM SIGPLAN Workshop on Languages, Compilers, and Hardware Support for Transactional Computing. 2006b. Riehle, D. Framework Design: A Role Modeling Approach. Ph.D. thesis, Swiss Federal Institute of Technology Zurich, 2000. Royce, W. W. Managing the development of large software systems: concepts and techniques. In Proceedings of the 9th International Conference on Software Engineering, pages 328–338. IEEE Computer Society, 1987. Rumbaugh, J. Relations as semantic constructs in an object-oriented language. In Proceedings of the 2nd ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, volume 22 of SIGPLAN Notices, pages 466–481. ACM Press, 1987. Scherer III, W. N. and Scott, M. L. Contention management in dynamic software transactional memory. In Proceedings of the ACM PODC Workshop on Concurrency and Synchronization in Java Programs. 2004. Scherer III, W. N. and Scott, M. L. Advanced contention management for dynamic software transactional memoryy. In Proceedings of the 24nd Annual ACM Symposium on Principles of Distributed Computing. ACM Press, 2005. Selic, B. The pragmatics of model-driven development. IEEE Software, volume 20(5):pages 19–25, 2003. Shah, A. V., Hamel, J. H., Borsari, R. A., and Rumbaugh, J. E.

DSM: an object-

relationship modeling language. In Proceedings of the 4th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, SIGPLAN Notices, pages 191–202. ACM Press, 1989. Shavit, N. and Touitou, D. Software transactional memory. In Proceedings of the 14th Annual ACM Symposium on Principles of Distributed Computing, pages 204–213. ACM Press, 1995.

216

BIBLIOGRAPHY

Suscheck, C. A. and Sandén, B. A construct for effectively implementing semantic associations. Journal of Object Technology, volume 2(3):pages 101–111, 2003. Thomas, D. UML - Unified or Universal Modeling Language? Journal of Object Technology, volume 2(1):pages 7–12, 2003. Thomas, D. and Barry, B. M. Model driven development: the case for domain oriented programming. In Companion to the 18th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, SIGPLAN Notices, pages 2–7. ACM Press, 2003.