Flexible integration of eventually consistent distributed storage with strongly consistent databases Olivier PARISOT, Antoine SCHLECHTER, Pascal BAULER, Fernand FELTZ
Centre de Recherche Public Gabriel Lippmann Belvaux – Grand Duchy of Luxembourg
www.lippmann.lu,
[email protected]
Summary Introduction Context Related works FlexibleDL Use-case Conclusion & perspectives
2
Introduction
3
Introduction
About CRP Gabriel Lippmann www.lippmann.lu Focus on visualization… … but strong interest
for issues related to data storage LuxDrops cloud.lippmann.lu
4
Context
5
Context (1/3)
Multi-tier architecture is the standard for distributed business applications & business services.
Traditionnally: Relational DataBase
Management System (Object Relational Mapping)
Most important requirement: Strong consistency •
Assumed by RDBMS
6
Context (2/3)
CAP theorem [*]
[*] Eric Brewer, “Towards Robust Distributed Systems”, PODC (Principles of Distributed Computing) Keynote (July 2000)
ACID vs BASE Atomicity, (Strong) Consistency, Isolation, Durability •
Relational DataBase Management System
Basically available, Soft state, Eventually consistent •
NoSQL data stores
7
Context (3/3)
What about scalability in new/existing business applications? No trade-off on availability! Using NoSQL products could be a solution… … but strong consistency is always/often needed
Question: Is it possible to improve data access layer to benefit of strong
consistency OR scalability when needed?
A solution: Providing flexibility for consistency
8
Related works
9
Related works
Providing different levels of consistency is an interesting theme for architecture design.
Two interesting approaches: CQRS: Command Query Responsibility Segregation •
Greg Young • http://cqrs.files.wordpress.com/2010/11/cqrs_documents.pdf
Lazy Primary Copy Replication •
M. Wiesmann, F. Pedone, A. Schiper, B. Kemme, G. Alonso, "Understanding Replication in Databases and Distributed Systems", Proceedings of the The 20th International Conference on Distributed Computing Systems, p.464, April 10-13, 2000
10
Related works: CQRS
Read and write separation PRESENTATION LAYER
Commands: SC writes
COMMAND LAYER
Queries: EC reads
DISPATCHER
QUERY LAYER
SC = Strongly Consistent EC = Eventually Consistent
Impact on the whole architecture No strongly consistent reads?
11
Related works: Lazy Primary Copy Replication
Provide a way to replicate data from a source data store to a target data store to guaranty that The source data store is strongly consistent The target data store is eventually consistent
Method: Data changes have to be grouped before replication •
Set of data changes grouped by transaction
Each group of data changes has to be kept into account only if the
related transaction is committed on the master data store. During replication, ordering of data changes has to be preserved.
12
FlexibleD L
13
FlexibleDL
A platform to provide when needed: Strong consistency OR
Eventual consistency (to allow scalability)
Data could be available in both modes.
Building blocks: An architecture inspired by CQRS. Usage of Lazy Primary Copy Replication.
14
FlexibleDL: concepts
Glossary: two kinds of data Relational Domain Model (RDM) • Strong consistency is required • Readable and writable Data Transfer Object (DTO) • Eventual consistency is possible • Used to retrieve combined results • Mainly built using costly queries... • Mainly big objects… • … scalability is desired • De-normalized and duplicated • Read-only
15
FlexibleDL: positioning
PRESENTATION LAYER
SERVICE LAYER
DATA ACCESS LAYER: FlexibleDL
16
FlexibleDL: architecture SC reads
SC writes
SC = Strongly Consistent
FLEXIBLE DL
EC reads
EC = Eventually Consistent
Strongly Consistent Layer
RDBMS
Synchroniser
Eventually Consistent Layer
SCALABLE DATASTORE 17
FlexibleDL: strongly consistent layer (1/2) SC reads
FLEXIBLE DL
SC writes
Strongly Consistent Layer
RDBMS
EC reads
Synchroniser
Eventually Consistent Layer
SCALABLE DATASTORE 18
FlexibleDL: strongly consistent layer (2/2) SC reads DTO
SC writes relational model
SC reads relational model
SC = Strongly Consistent
view
repository
view
repository view
STRONGLY CONSISTENT LAYER
repository
INTERCEPTOR data changes RDBMS 19
FlexibleDL: synchronizer (1/4) SC reads
FLEXIBLE DL
SC writes
Strongly Consistent Layer
RDBMS
EC reads
Synchroniser
Eventually Consistent Layer
SCALABLE DATASTORE 20
FlexibleDL: synchronizer (2/4) EVENTUALLY CONSISTENT LAYER
STRONGLY CONSISTENT LAYER
SYNCHRONIZER INTERCEPTOR
APPLIER
PUBLISHER
LISTENER
ESB
21
FlexibleDL: synchronizer (3/4)
Event-based TypeOfChange {ADD,UPD,DEL} DataDiff { Object modifiedObject String typeOfObject TypeOfChange typeOfChange } Event { List list; } 22
FlexibleDL: synchronizer (4/4)
Usage of Lazy Primary Copy Replication mechanism Interceptor •
Capture data changes by intercepting transactions. • An event = a set of data changes, grouped by transaction.
Publisher •
Preserve ordering during events propagation. • Use a distributed message passing system (or an ESB).
Listener •
Listen to events and pass them to Applyer.
Applyer •
Apply events on the eventually consistent layer.
23
FlexibleDL: eventually consistent layer (1/2) SC reads
FLEXIBLE DL
SC writes
Strongly Consistent Layer
RDBMS
EC reads
Synchroniser
Eventually Consistent Layer
SCALABLE DATASTORE 24
FlexibleDL: eventually consistent layer (2/2)
Provide eventually consistent access to data. EC read DTO EC = Eventually Consistent
view
view view
EVENTUALLY CONSISTENT LAYER
ViewRefresher { onAdd(Object object); onUpd(Object object); onDel(Object object); }
SCALABLE DATASTORE 25
FlexibleDL: query definition
Several cases The strongly consistent queries which are executed in strongly
consistent context only. •
SQL queries
The eventually consistent queries which are executed in
eventually consistent context only. •
view refresher definition + target data store query
The queries that can be executed in both consistent contexts. •
SQL queries AND
•
view refresher definition + target data store query • Both definitions must be equivalent!
26
FlexibleDL: system initialization strategy
Prerequisite to exploit the architecture.
Two cases The relational data base is empty, and the eventually consistent
layer is empty too. •
Start of an empty system with no data at all.
The relational data base is already filled with data •
The eventually consistent layer has to be initially synchronized with the strongly consistent layer: ResyncEvent.
27
FlexibleDL: implementation
Prototype implemented in JAVA.
Strongly consistent layer Hibernate + HSQLDB
Eventually consistent layer Hazelcast distributed & queryable hashmaps
Synchronizer Interceptor: Hibernate listeners Publisher & ESB & Listener: Hazelcast distributed topics
Easy to use it in any JAVA project: flexibleDL.jar 28
Use case
29
FlexibleDL: use case (1/3)
A simple Order Management System (OMS)
Modeling Persistent domain model definition @Entity class Product {…} • @Entity class PurchaseOrder {…} • @Entity class PurchaseOrderLine {…} •
Third party infrastructure: Running a RDBMS (hsqldb) Running a data grid cluster (Hazelcast)
30
FlexibleDL: use case (2/3)
Example of query: the X most purchased products.
Could be executed in two contexts: Critical business feature: paying providers according to their
importance •
may be strongly consistent
Not critical feature: public information on a web site for consumers •
may be eventually consistent
Using FlexibleDL, this request can be implemented in the two contexts.
31
FlexibleDL: use case (3/3)
Strongly consistent version: select new PurchasedProductCount(ol.product.id,ol.product.name,count(*)) from PurchaseOrderLine ol group by ol.product.id,ol.product.name order by count(*)
Eventually consistent version:
UPDATE
QUERY
ViewRefresherWithPurchaseOrderLine { PurchasedProductCountECView view
PurchasedProductCountECView { contentMap=Hazelcast.getMap( );
onAdd(PurchaseOrderLine ol) {...} onUpd(PurchaseOrderLine ol) {...} onDel(PurchaseOrderLine ol) {...}
executeQuery() { return contentMap.values() }
} }
32
Conclusion & perspectives
33
Conclusion and perspectives
FlexibleDL An elegant solution that supports both eventually and strongly
consistent. Can be easily used to migrate existing business solutions to more scalable software architectures.
Future works Scalability measures in cloud environment.
Interesting perspectives Eventually consistent write operations Others consistency levels (for example: time-delayed consistency,
appointed consistency)
34
That’s all!
35