Flexible integration of eventually consistent ...

Flexible integration of eventually consistent distributed storage with strongly consistent databases Olivier PARISOT, Antoine SCHLECHTER, Pascal BAULER, Fernand FELTZ

Centre de Recherche Public Gabriel Lippmann Belvaux – Grand Duchy of Luxembourg

www.lippmann.lu, [email protected]

Summary Introduction  Context  Related works  FlexibleDL  Use-case  Conclusion & perspectives 

2

Introduction

3

Introduction 

About CRP Gabriel Lippmann  www.lippmann.lu  Focus on visualization…  … but strong interest

for issues related to data storage  LuxDrops  cloud.lippmann.lu

4

Context

5

Context (1/3) 

Multi-tier architecture is the standard for distributed business applications & business services.



Traditionnally:  Relational DataBase

Management System  (Object Relational Mapping)



Most important requirement:  Strong consistency •

Assumed by RDBMS

6

Context (2/3) 

CAP theorem [*]





[*] Eric Brewer, “Towards Robust Distributed Systems”, PODC (Principles of Distributed Computing) Keynote (July 2000)

ACID vs BASE  Atomicity, (Strong) Consistency, Isolation, Durability •

Relational DataBase Management System

 Basically available, Soft state, Eventually consistent •

NoSQL data stores

7

Context (3/3) 

What about scalability in new/existing business applications?  No trade-off on availability!  Using NoSQL products could be a solution…  … but strong consistency is always/often needed



Question:  Is it possible to improve data access layer to benefit of strong

consistency OR scalability when needed? 

A solution:  Providing flexibility for consistency

8

Related works

9

Related works 

Providing different levels of consistency is an interesting theme for architecture design.



Two interesting approaches:  CQRS: Command Query Responsibility Segregation •

Greg Young • http://cqrs.files.wordpress.com/2010/11/cqrs_documents.pdf

 Lazy Primary Copy Replication •

M. Wiesmann, F. Pedone, A. Schiper, B. Kemme, G. Alonso, "Understanding Replication in Databases and Distributed Systems", Proceedings of the The 20th International Conference on Distributed Computing Systems, p.464, April 10-13, 2000

10

Related works: CQRS 

Read and write separation PRESENTATION LAYER

Commands: SC writes

COMMAND LAYER

Queries: EC reads

DISPATCHER

QUERY LAYER

SC = Strongly Consistent EC = Eventually Consistent

Impact on the whole architecture  No strongly consistent reads? 

11

Related works: Lazy Primary Copy Replication 

Provide a way to replicate data from a source data store to a target data store to guaranty that  The source data store is strongly consistent  The target data store is eventually consistent



Method:  Data changes have to be grouped before replication •

Set of data changes grouped by transaction

 Each group of data changes has to be kept into account only if the

related transaction is committed on the master data store.  During replication, ordering of data changes has to be preserved.

12

FlexibleD L

13

FlexibleDL 

A platform to provide when needed:  Strong consistency OR

 Eventual consistency (to allow scalability)



Data could be available in both modes.



Building blocks:  An architecture inspired by CQRS.  Usage of Lazy Primary Copy Replication.

14

FlexibleDL: concepts 

Glossary: two kinds of data  Relational Domain Model (RDM) • Strong consistency is required • Readable and writable  Data Transfer Object (DTO) • Eventual consistency is possible • Used to retrieve combined results • Mainly built using costly queries... • Mainly big objects… • … scalability is desired • De-normalized and duplicated • Read-only

15

FlexibleDL: positioning

PRESENTATION LAYER

SERVICE LAYER

DATA ACCESS LAYER: FlexibleDL

16

FlexibleDL: architecture SC reads

SC writes

SC = Strongly Consistent

FLEXIBLE DL

EC reads

EC = Eventually Consistent

Strongly Consistent Layer

RDBMS

Synchroniser

Eventually Consistent Layer

SCALABLE DATASTORE 17

FlexibleDL: strongly consistent layer (1/2) SC reads

FLEXIBLE DL

SC writes


RDBMS

EC reads

Synchroniser



FlexibleDL: strongly consistent layer (2/2) SC reads DTO

SC writes relational model

SC reads relational model

SC = Strongly Consistent

view

repository

view

repository view

STRONGLY CONSISTENT LAYER

repository

INTERCEPTOR data changes RDBMS 19

FlexibleDL: synchronizer (1/4) SC reads

FLEXIBLE DL

SC writes


RDBMS

EC reads

Synchroniser



FlexibleDL: synchronizer (2/4) EVENTUALLY CONSISTENT LAYER

STRONGLY CONSISTENT LAYER

SYNCHRONIZER INTERCEPTOR

APPLIER

PUBLISHER

LISTENER

ESB

21

FlexibleDL: synchronizer (3/4) 

Event-based TypeOfChange {ADD,UPD,DEL} DataDiff { Object modifiedObject String typeOfObject TypeOfChange typeOfChange } Event { List list; } 22

FlexibleDL: synchronizer (4/4) 

Usage of Lazy Primary Copy Replication mechanism  Interceptor •

Capture data changes by intercepting transactions. • An event = a set of data changes, grouped by transaction.

 Publisher •

Preserve ordering during events propagation. • Use a distributed message passing system (or an ESB).

 Listener •

Listen to events and pass them to Applyer.

 Applyer •

Apply events on the eventually consistent layer.

23

FlexibleDL: eventually consistent layer (1/2) SC reads

FLEXIBLE DL

SC writes


RDBMS

EC reads

Synchroniser



FlexibleDL: eventually consistent layer (2/2) 

Provide eventually consistent access to data. EC read DTO EC = Eventually Consistent

view

view view

EVENTUALLY CONSISTENT LAYER

ViewRefresher { onAdd(Object object); onUpd(Object object); onDel(Object object); }


FlexibleDL: query definition 

Several cases  The strongly consistent queries which are executed in strongly

consistent context only. •

SQL queries

 The eventually consistent queries which are executed in

eventually consistent context only. •

view refresher definition + target data store query

 The queries that can be executed in both consistent contexts. •

SQL queries AND

•

view refresher definition + target data store query • Both definitions must be equivalent!

26

FlexibleDL: system initialization strategy 

Prerequisite to exploit the architecture.



Two cases  The relational data base is empty, and the eventually consistent

layer is empty too. •

Start of an empty system with no data at all.

 The relational data base is already filled with data •

The eventually consistent layer has to be initially synchronized with the strongly consistent layer: ResyncEvent.

27

FlexibleDL: implementation 

Prototype implemented in JAVA.



Strongly consistent layer  Hibernate + HSQLDB



Eventually consistent layer  Hazelcast distributed & queryable hashmaps



Synchronizer  Interceptor: Hibernate listeners  Publisher & ESB & Listener: Hazelcast distributed topics



Easy to use it in any JAVA project: flexibleDL.jar 28

Use case

29

FlexibleDL: use case (1/3) 

A simple Order Management System (OMS)



Modeling  Persistent domain model definition @Entity class Product {…} • @Entity class PurchaseOrder {…} • @Entity class PurchaseOrderLine {…} •



Third party infrastructure: Running a RDBMS (hsqldb)  Running a data grid cluster (Hazelcast) 

30


Example of query: the X most purchased products.



Could be executed in two contexts:  Critical business feature: paying providers according to their

importance •

may be strongly consistent

 Not critical feature: public information on a web site for consumers •



may be eventually consistent

Using FlexibleDL, this request can be implemented in the two contexts.

31


Strongly consistent version: select new PurchasedProductCount(ol.product.id,ol.product.name,count(*)) from PurchaseOrderLine ol group by ol.product.id,ol.product.name order by count(*)

Eventually consistent version:



UPDATE

QUERY

ViewRefresherWithPurchaseOrderLine { PurchasedProductCountECView view

PurchasedProductCountECView { contentMap=Hazelcast.getMap( );

onAdd(PurchaseOrderLine ol) {...} onUpd(PurchaseOrderLine ol) {...} onDel(PurchaseOrderLine ol) {...}

executeQuery() { return contentMap.values() }

} }

32

Conclusion & perspectives

33

Conclusion and perspectives 

FlexibleDL  An elegant solution that supports both eventually and strongly

consistent.  Can be easily used to migrate existing business solutions to more scalable software architectures. 

Future works  Scalability measures in cloud environment.



Interesting perspectives  Eventually consistent write operations  Others consistency levels (for example: time-delayed consistency,

appointed consistency)

34

That’s all!

35