Highly available web sites with Tomcat and Clustered JDBC - People

25 downloads 588 Views 6MB Size Report
http://c-jdbc.objectweb.org/ - [email protected]. 7 - 07/11/2003. Outline. ➡ RAIDb. ➡C-JDBC. ➡Tomcat and C-JDBC. ➡Scalability. ➡High availability ...
Highly available web sites with Tomcat and Clustered JDBC Emmanuel Cecchet

Motivations  Database tier should be  scalable  highly available  without modifying the client application  database vendor independent  on commodity hardware JDBC JDBC

JDBC

Internet

http://c-jdbc.objectweb.org/ - [email protected]

2 - 07/11/2003

Scaling the database tier – Alternative 1 (master-slave)

Cons  failover time on master failure  scalability Web frontend

App. server

Master

Internet

http://c-jdbc.objectweb.org/ - [email protected]

3 - 07/11/2003

Scaling the database tier – Alternative 2 (atomic broadcast)

Cons  atomic broadcast scalability  no client side load balancing  heavy modifications of the database engine

Internet

Atomic broadcast

http://c-jdbc.objectweb.org/ - [email protected]

4 - 07/11/2003

Scaling the database tier – Alternative 3 (SMP)

 Cons  Cost  Scalability limit Web frontend

App. server Database

Internet

Well-known database vendor here Well-known hardware + database vendors here http://c-jdbc.objectweb.org/ - [email protected]

5 - 07/11/2003

Scaling the database tier – Alternative 4 (shared disks)

Cons  still expensive hardware  availability Web frontend

App. server

Database

Disks

Internet

Another well-known database vendor http://c-jdbc.objectweb.org/ - [email protected]

6 - 07/11/2003

Outline

RAIDb C-JDBC Tomcat and C-JDBC Scalability High availability

http://c-jdbc.objectweb.org/ - [email protected]

7 - 07/11/2003

RAIDb - Definition

Redundant Array of Inexpensive Databases better performance and fault tolerance than a single database, at a low cost, by combining multiple database instances into an array of databases RAIDb levels offers various tradeoff of performance and fault tolerance

http://c-jdbc.objectweb.org/ - [email protected]

8 - 07/11/2003

Key ideas

RAIDb controller gives the view of a single database to the client balance the load on the database backends

RAIDb levels RAIDb-0: full partitioning RAIDb-1: full mirroring RAIDb-2: partial replication composition possible

http://c-jdbc.objectweb.org/ - [email protected]

9 - 07/11/2003

RAIDb levels

RAIDb-0 partitioning no duplication and no fault tolerance at least 2 nodes SQL requests RAIDb controller

table 1

table 2 & 3

http://c-jdbc.objectweb.org/ - [email protected]

table ...

table n-1

table n 10 - 07/11/2003

RAIDb levels

RAIDb-1 mirroring performance bounded by write broadcast at least 2 nodes SQL requests RAIDb controller

Full DB http://c-jdbc.objectweb.org/ - [email protected]

Full DB

Full DB

Full DB

Full DB 11 - 07/11/2003

RAIDb levels

RAIDb-2 partial replication at least 2 copies of each table at least 3 nodes SQL requests RAIDb controller

Full DB

table x

http://c-jdbc.objectweb.org/ - [email protected]

table y

table x & y

table z 12 - 07/11/2003

RAIDb levels composition SQL requests

RAIDb-1-0

RAIDb-1 controller

no limit to the composition deepness

RAIDb-0 controller

table w

table x & y

RAIDb-0 controller

table z

table w

table y

table x & z

RAIDb-0 controller

table w http://c-jdbc.objectweb.org/ - [email protected]

table x

table y

table z 13 - 07/11/2003

Outline

RAIDb C-JDBC Tomcat and C-JDBC Scalability High availability

http://c-jdbc.objectweb.org/ - [email protected]

14 - 07/11/2003

C-JDBC overview

Middleware implementing RAIDb 100% Java implementation open source (LGPL)

Two components generic JDBC driver (C-JDBC driver) C-JDBC Controller

Read-one, Write all approach Supports heterogeneous databases

http://c-jdbc.objectweb.org/ - [email protected]

15 - 07/11/2003

architectural overview

Application server C-JDBC MySQL JDBC driver JVM

http://c-jdbc.objectweb.org/ - [email protected]

C-JDBC controller MySQL database JVM

16 - 07/11/2003

Inside the C-JDBC Controller Java client program (Servlet, EJB, ...)

C-JDBC driver

RMI Sockets

RMI Sockets

C-JDBC Controller XML configuration file Administration console

JMX

Configuration & administration

Virtual database

Virtual database

Authentication Manager

Authentication Manager

Request Manager

Request Manager

XML engine

Scheduler Recovery Log

Request Cache

Scheduler Recovery Log

Request Cache

Load balancer

http://c-jdbc.objectweb.org/ - [email protected]

Load balancer

Database Backend

Database Backend

Database Backend

Database Backend

Database Backend

Connection Manager

Connection Manager

Connection Manager

Connection Manager

Connection Manager

MySQL JDBC driver

MySQL JDBC driver

MySQL JDBC driver

Oracle JDBC driver

Oracle JDBC driver

MySQL

MySQL

MySQL

Oracle

Oracle 17 - 07/11/2003

Virtual Database

gives the view of a single

Java client program (Servlet, EJB, ...)

C-JDBC driver

Virtual database Authentication Manager Request Manager Scheduler Recovery Log

Request Cache Load balancer

Database Backend

Database Backend

Database Backend

Connection Manager

Connection Manager

Connection Manager

MySQL JDBC driver

MySQL JDBC driver

MySQL JDBC driver

MySQL

MySQL

MySQL

database establishes the mapping between the database name used by the application and the backend specific settings backends can be added and removed dynamically configured using an XML configuration file

http://c-jdbc.objectweb.org/ - [email protected]

18 - 07/11/2003

Authentication Manager

Matches real login/password

Java client program (Servlet, EJB, ...)

C-JDBC driver

Virtual database Authentication Manager Request Manager Scheduler Recovery Log

Request Cache Load balancer

Database Backend

Database Backend

Database Backend

Connection Manager

Connection Manager

Connection Manager

MySQL JDBC driver

MySQL JDBC driver

MySQL JDBC driver

MySQL

MySQL

MySQL

used by the application with backend specific login/ password Administrator login to manage the virtual database

http://c-jdbc.objectweb.org/ - [email protected]

19 - 07/11/2003

Scheduler

Manages concurrency control Specific implementations for

Java client program (Servlet, EJB, ...)

C-JDBC driver

Virtual database Authentication Manager Request Manager Scheduler Recovery Log

Request Cache Load balancer

Database Backend

Database Backend

Database Backend

Connection Manager

Connection Manager

Connection Manager

MySQL JDBC driver

MySQL JDBC driver

MySQL JDBC driver

MySQL

MySQL

MySQL

Single DB, RAIDb 0, 1 and 2 Query-level Optimistic and pessimistic transaction level

http://c-jdbc.objectweb.org/ - [email protected]

uses the database schema that is automatically fetched from backends

20 - 07/11/2003

Request cache caches results from SQL requests improved SQL statement analysis to

Java client program (Servlet, EJB, ...)

C-JDBC driver

limit cache invalidations

Virtual database

table based invalidations column based invalidations single-row SELECT optimization

Authentication Manager Request Manager Scheduler Recovery Log

Request Cache Load balancer

Database Backend

Database Backend

Database Backend

Connection Manager

Connection Manager

Connection Manager

MySQL JDBC driver

MySQL JDBC driver

MySQL JDBC driver

MySQL

MySQL

MySQL

request parsing possible in the

http://c-jdbc.objectweb.org/ - [email protected]

C-JDBC driver offload the controller parsing caching in the driver

21 - 07/11/2003

Load balancer 1/2 RAIDb-0

Java client program (Servlet, EJB, ...)

query directed to the backend having the

C-JDBC driver

needed tables Virtual database Authentication Manager

RAIDb-1 read executed by current thread write executed in parallel by a dedicated

Request Manager Scheduler Recovery Log

thread per backend result returned if one, majority or all commit if one node fails but others succeed, failing node is disabled

Request Cache Load balancer

Database Backend

Database Backend

Database Backend

Connection Manager

Connection Manager

Connection Manager

MySQL JDBC driver

MySQL JDBC driver

MySQL JDBC driver

RAIDb-2 same as RAIDb-1 except that writes are sent only to nodes owning the written table

MySQL

MySQL

MySQL

http://c-jdbc.objectweb.org/ - [email protected]

22 - 07/11/2003

Load balancer 2/2

Static load balancing policies

Java client program (Servlet, EJB, ...)

C-JDBC driver

Round-Robin (RR) Weighted Round-Robin (WRR)

Virtual database Authentication Manager Request Manager

Least Pending Requests First

Scheduler Recovery Log

Request Cache Load balancer

Database Backend

Database Backend

Database Backend

Connection Manager

Connection Manager

Connection Manager

MySQL JDBC driver

MySQL JDBC driver

MySQL JDBC driver

MySQL

MySQL

MySQL

http://c-jdbc.objectweb.org/ - [email protected]

(LPRF) request sent to the node that has the shortest pending request queue efficient if backends are homogeneous in terms of performance

23 - 07/11/2003

Connection Manager

Connection pooling for a

Java client program (Servlet, EJB, ...)

C-JDBC driver

backend Simple : no pooling RandomWait : blocking pool FailFast : non-blocking pool VariablePool : dynamic pool

Virtual database Authentication Manager Request Manager Scheduler Recovery Log

Request Cache Load balancer

Database Backend

Database Backend

Database Backend

Connection Manager

Connection Manager

Connection Manager

MySQL JDBC driver

MySQL JDBC driver

MySQL JDBC driver

MySQL

MySQL

MySQL

Connection pools defined on a

http://c-jdbc.objectweb.org/ - [email protected]

per login basis resource management per login dedicated connections for admin 24 - 07/11/2003

Recovery Log

Checkpoints are associated

Java client program (Servlet, EJB, ...)

C-JDBC driver

Virtual database Authentication Manager Request Manager Scheduler Recovery Log

Request Cache Load balancer

Database Backend

Database Backend

Database Backend

Connection Manager

Connection Manager

Connection Manager

MySQL JDBC driver

MySQL JDBC driver

MySQL JDBC driver

MySQL

MySQL

MySQL

with database dumps Record all updates and transaction markers since a checkpoint Used to resynchronize a database from a checkpoint JDBCRecoveryLog

http://c-jdbc.objectweb.org/ - [email protected]

store information in a database can be re-injected in a C-JDBC cluster for fault tolerance 25 - 07/11/2003

Functional overview Client application (Servlet, EJB, ...)

Client application (Servlet, EJB, ...)

connect login, password C-JDBC driver connect myDB Controller execute SELECT * FROM t C-JDBC Virtual database 1

C-JDBC driver

Authentication Manager Request Manager Scheduler Recovery Query result cache Log Load balancer

http://c-jdbc.objectweb.org/ - [email protected]

Database Backend

Database Backend

Database Backend

Connection Manager

Connection Manager

Connection Manager

MySQL JDBC driver

MySQL JDBC driver

MySQL JDBC driver

MySQL

MySQL

MySQL 26 - 07/11/2003

Functional overview Client application (Servlet, EJB, ...)

Client application (Servlet, EJB, ...)

C-JDBC driver

C-JDBC driver

execute INSERT INTO t …

C-JDBC Controller Virtual database 1 Authentication Manager Request Manager Scheduler Recovery Query result cache Log Load balancer

http://c-jdbc.objectweb.org/ - [email protected]

Database Backend

Database Backend

Database Backend

Connection Manager

Connection Manager

Connection Manager

MySQL JDBC driver

MySQL JDBC driver

MySQL JDBC driver

MySQL

MySQL

MySQL 27 - 07/11/2003

Failures execute INSERT INTO t …

Client application (Servlet, EJB, ...)

Client application (Servlet, EJB, ...)

C-JDBC driver

C-JDBC driver

C-JDBC Controller

No 2 phasecommit  parallel transactions  failed nodes are automatically disabled

http://c-jdbc.objectweb.org/ - [email protected]

Virtual database 1 Authentication Manager Request Manager Scheduler Recovery Query result cache Log Load balancer Database Backend

Database Backend

Database Backend

Connection Manager

Connection Manager

Connection Manager

MySQL JDBC driver

MySQL JDBC driver

MySQL JDBC driver

MySQL

MySQL

MySQL 28 - 07/11/2003

Outline

RAIDb C-JDBC Tomcat and C-JDBC Scalability High availability

http://c-jdbc.objectweb.org/ - [email protected]

29 - 07/11/2003

Highly available web site  Apache clustering  L4 switch, RR-DNS, One-IP techniques, LVS, …

 Tomcat clustering  mod-jk, session replication

 Database clustering  C-JDBC

RR-DNS

mod-jk

Parsing cache Result cache Metadata cache

Internet

http://c-jdbc.objectweb.org/ - [email protected]

30 - 07/11/2003

Result cache

Cache contains a list of SQL_ResultSet Policy defined by queryPattern_Policy 3 policies  EagerCaching: variable granularities for invalidations  RelaxedCaching: invalidations based on timeout  NoCaching: never cached RUBiS bidding mix with 450 clients Throughput (rq/min)

No cache

Coherent cache

Relaxed cache

3892

4184

4215

Avg response time

801 ms

284 ms

134 ms

Database CPU load

100%

85%

20%

C-JDBC CPU load

-

15%

7%

http://c-jdbc.objectweb.org/ - [email protected]

31 - 07/11/2003

Configuring Tomcat 5

Same setting for all Tomcat nodes Copy c-jdbc-driver.jar to $CATALINA_HOME/common/lib

Without connection pooling 1. load driver  Class.forName("org.objectweb.cjdbc.driver.Driver") .newInstance();

2. get a connection  Connection conn = DriverManager. getConnection("jdbc:cjdbc://host/myDB","user","password"); http://c-jdbc.objectweb.org/ - [email protected]

32 - 07/11/2003

Configuring Tomcat 5

With connection pooling factory org.apache.commons.dbcp.BasicDataSourceFactory driverClassName org.objectweb.cjdbc.driver.Driver url jdbc:cjdbc://localhost/myDB http://c-jdbc.objectweb.org/ - [email protected]

33 - 07/11/2003

Configuring C-JDBC  Virtual database configuration file

http://c-jdbc.objectweb.org/ - [email protected]

36 - 07/11/2003

C-JDBC vertical scalability

allows nested RAIDb levels allows tree architecture for scalable write broadcast necessary with large number of backends C-JDBC driver re-injected in C-JDBC controller

http://c-jdbc.objectweb.org/ - [email protected]

37 - 07/11/2003

C-JDBC vertical scalability

RAIDb-1-1

Client program

with C-JDBC no limit to composition deepness

C-JDBC driver JVM

Client program

Client program

C-JDBC driver JVM

C-JDBC driver JVM

C-JDBC controller RAIDb-1 C-JDBC driver

C-JDBC controller RAIDb-1

C-JDBC controller RAIDb-1

C-JDBC controller RAIDb-1

MySQL JDBC driver

MySQL JDBC driver

MySQL JDBC driver

MySQL

MySQL

MySQL

http://c-jdbc.objectweb.org/ - [email protected]

MySQL

MySQL

MySQL

MySQL

MySQL 38 - 07/11/2003

Vertical scalability  Addresses JVM scalability issues  Distributing large number of connections on many backends

Client program

Client program

C-JDBC driver JVM

C-JDBC driver JVM

Client program C-JDBC driver JVM

C-JDBC controller Partial replication C-JDBC driver

C-JDBC controller Partial replication

C-JDBC controller Partial replication

C-JDBC controller Partial replication

DB native JDBC driver

DB native JDBC driver

DB native JDBC driver

DB 1 http://c-jdbc.objectweb.org/ - [email protected]

DB 2

DB 3

DB 4

DB 5

DB 6

DB7 39 - 07/11/2003

TPC-W benchmark (Amazon.com)

Throughput in requests per minute

 Nearly linear speedups with the shopping mix 1600 1400 1200 1000

Single Database

800

Full replication

600

Partial replication

400 200 0 0

1

2

3

4

5

6

Number of nodes http://c-jdbc.objectweb.org/ - [email protected]

40 - 07/11/2003

Outline

RAIDb C-JDBC Tomcat and C-JDBC Scalability High availability

http://c-jdbc.objectweb.org/ - [email protected]

41 - 07/11/2003

Controller replication  Prevent the controller from being a single point of failure  Group communication for controller synchronization  C-JDBC driver supports multiple controllers with automatic failover

jdbc:c-jdbc://node1:25322,node2:12345/myDB Java client program

C-JDBC driver JVM

C-JDBC driver JVM

C-JDBC Controller MySQL PostgreSQL JDBC driver JDBC driver

MySQL

http://c-jdbc.objectweb.org/ - [email protected]

Java client program

PostgreSQL

JGroups

Java client program

Java client program

C-JDBC driver

C-JDBC driver

JVM

JVM

C-JDBC Controller PostgreSQL MySQL JDBC driver JDBC driver

PostgreSQL

MySQL

42 - 07/11/2003

Controller replication Client application (Servlet, EJB, ...)

Client application (Servlet, EJB, ...)

Client application (Servlet, EJB, ...)

C-JDBC driver

C-JDBC driver

C-JDBC driver

jdbc:cjdbc://node1,node2/myDB Total order reliable multicast

C-JDBC Controller

C-JDBC Controller

Distributed Request Manager

Distributed Request Manager

Virtual database 1

Virtual database 2

Authentication Manager

Authentication Manager

Request Manager

Request Manager

Scheduler

Scheduler

Query result cache

Query result cache

Load balancer

Load balancer

Database Backend

Database Backend

Database Backend

Database Backend

Connection Manager

Connection Manager

Connection Manager

Connection Manager

MySQL JDBC driver

MySQL JDBC driver

MySQL JDBC driver

Oracle JDBC driver

MySQL

MySQL

MySQL

Oracle

http://c-jdbc.objectweb.org/ - [email protected]

43 - 07/11/2003

Client program

Mixing horizontal & vertical scalability

C-JDBC driver JVM

Client program

Client program

C-JDBC driver JVM

C-JDBC driver

C-JDBC controller Full replication

C-JDBC controller Full replication DB native JDBC driver

JVM

C-JDBC driver

C-JDBC driver

C-JDBC controller Full replication DB 1

DB 2

DB native C-JDBC JDBC driver driver

C-JDBC controller Full replication DB native JDBC driver

DB 3

http://c-jdbc.objectweb.org/ - [email protected]

DB 4

DB 5

DB 6

DB 7

44 - 07/11/2003

Building initial checkpoint  Octopus is an ETL tool  Use Octopus to store a dump of the initial database state  Currently done by the user using the database specific dump tool Octopus dump for initial checkpoint

EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...

C-JDBC driver JVM

disabled

C-JDBC Controller Recovery Log PostgreSQL JDBC driver

PostgreSQL disabled http://c-jdbc.objectweb.org/ - [email protected]

45 - 07/11/2003

Logging  Backend is enabled  All database updates are logged (SQL statement, user, transaction, …)

EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...

C-JDBC driver JVM

JDBC Recovery Log

Octopus dump for initial checkpoint

enabled

C-JDBC Controller Recovery Log PostgreSQL JDBC driver

PostgreSQL enabled http://c-jdbc.objectweb.org/ - [email protected]

46 - 07/11/2003

Adding new backends 1/3  Add new backends

EJB Container

while system online  Restore dump corresponding to JDBC initial checkpoint Recovery Log with Octopus Octopus

JOnAS, WebLogic, JBoss, WebSphere, ...

C-JDBC driver JVM

enabled

C-JDBC Controller Recovery Log PostgreSQL JDBC driver

dump for initial checkpoint

PostgreSQL PostgreSQL disabled enabled http://c-jdbc.objectweb.org/ - [email protected]

PostgreSQL disabled 47 - 07/11/2003

Adding new backends 2/3 EJB Container

 Replay updates from the log

JOnAS, WebLogic, JBoss, WebSphere, ...

C-JDBC driver JVM

JDBC Recovery Log

enabled

C-JDBC Controller Recovery Log

Octopus

PostgreSQL JDBC driver

dump for initial checkpoint

PostgreSQL disabled http://c-jdbc.objectweb.org/ - [email protected]

PostgreSQL enabled

PostgreSQL disabled 48 - 07/11/2003

Adding new backends 3/3 EJB Container

 Enable backends

JOnAS, WebLogic, JBoss, WebSphere, ...

when done

C-JDBC driver JVM

JDBC Recovery Log

enabled

C-JDBC Controller Recovery Log

Octopus

PostgreSQL JDBC driver

dump for initial checkpoint

PostgreSQL enabled http://c-jdbc.objectweb.org/ - [email protected]

PostgreSQL enabled

PostgreSQL enabled 49 - 07/11/2003

Making new checkpoints  Disable one backend to have a coherent snapshot  Mark the new checkpoint entry in the log  Use Octopus to store JDBC the dump Recovery Log

EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...

C-JDBC driver JVM

enabled

C-JDBC Controller Recovery Log

dump for initial checkpoint

...

Octopus

PostgreSQL JDBC driver

dump for dump last for last checkpoint checkpoint

PostgreSQL disabled http://c-jdbc.objectweb.org/ - [email protected]

PostgreSQL enabled

PostgreSQL enabled 50 - 07/11/2003

Making new checkpoints  Replay missing updates from log

EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...

C-JDBC driver JVM

JDBC Recovery Log

enabled

C-JDBC Controller Recovery Log

dump for initial checkpoint

...

Octopus

PostgreSQL JDBC driver

dump for dump last forfor last checkpoint checkpoint

PostgreSQL disabled http://c-jdbc.objectweb.org/ - [email protected]

PostgreSQL enabled

PostgreSQL enabled 51 - 07/11/2003

Making new checkpoints  Re-enable backend

EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...

when done

C-JDBC driver JVM

JDBC Recovery Log

enabled

C-JDBC Controller Recovery Log

dump for initial checkpoint

...

Octopus

PostgreSQL JDBC driver

dump for dump last for for last checkpoint checkpoint

PostgreSQL enabled http://c-jdbc.objectweb.org/ - [email protected]

PostgreSQL enabled

PostgreSQL enabled 52 - 07/11/2003

Handling failures  A node fails!  Automatically

EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...

disabled but should be fixed or changed JDBC by administrator Recovery Log

C-JDBC driver JVM

enabled

C-JDBC Controller Recovery Log

dump for initial checkpoint

...

Octopus

PostgreSQL JDBC driver

dump for dump last for for last checkpoint checkpoint

PostgreSQL disabled http://c-jdbc.objectweb.org/ - [email protected]

PostgreSQL enabled

PostgreSQL enabled 53 - 07/11/2003

Recovery 1/3  Restore latest dump with Octopus

EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...

C-JDBC driver JVM

JDBC Recovery Log

enabled

C-JDBC Controller Recovery Log

dump for initial checkpoint

...

Octopus

PostgreSQL JDBC driver

dump for dump last for last checkpoint checkpoint

PostgreSQL disabled http://c-jdbc.objectweb.org/ - [email protected]

PostgreSQL enabled

PostgreSQL enabled 54 - 07/11/2003

Recovery 2/3  Replay missing updates from log

EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...

C-JDBC driver JVM

JDBC Recovery Log

enabled

C-JDBC Controller Recovery Log

dump for initial checkpoint

...

Octopus

PostgreSQL JDBC driver

dump for dump last forfor last checkpoint checkpoint

PostgreSQL disabled http://c-jdbc.objectweb.org/ - [email protected]

PostgreSQL enabled

PostgreSQL enabled 55 - 07/11/2003

Recovery 3/3  Re-enable backend

EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...

when done

C-JDBC driver JVM

JDBC Recovery Log

enabled

C-JDBC Controller Recovery Log

dump for initial checkpoint

...

Octopus

PostgreSQL JDBC driver

dump for dump last for for last checkpoint checkpoint

PostgreSQL enabled http://c-jdbc.objectweb.org/ - [email protected]

PostgreSQL enabled

PostgreSQL enabled 56 - 07/11/2003

Demo

Administration tools HypersonicSQL in-memory databases JMX

http://c-jdbc.objectweb.org/ - [email protected]

57 - 07/11/2003

Conclusion

RAIDb C-JDBC open source middleware for database replication RDBMS independent

Tomcat & C-JDBC no application modification required performance scalability high availability transparent failover

http://c-jdbc.objectweb.org/ - [email protected]

58 - 07/11/2003

Q&A _________ Thanks to all users and contributors ...

http://c-jdbc.objectweb.org http://c-jdbc.objectweb.org/ - [email protected]

59 - 07/11/2003

Bonus slides

HORIZONTAL SCALABILITY

Horizontal scalability

JGroups for controller synchronization Groups messages for writes only Client 1

Client...

Client n

DB2

DB3

http://c-jdbc.objectweb.org/ - [email protected]

Client...

Controller1

Controller1

DB1

Client 1

DB4

DB1

DB2

Client n

Controller2

DB3

DB4

62 - 07/11/2003

Horizontal scalability

Centralized write approach issues co

co

Con

D

D

Con

D

Con

D

D

D

co

Con

D

D

co

Con

D

D

?

D

D

? ?

?

Issues with transactions assigned to connections T1(Write)

T1(Read) Controller1

Controller2

Controller1

Controller2

Controller1

Controller2

DB1

DB2

DB1

DB2

DB1

DB2

http://c-jdbc.objectweb.org/ - [email protected]

T1(R)

T1(W)

T1(R) T2(W) 63 - 07/11/2003

Horizontal scalability

General case for a write query  3 multicast + 2n unicast DistributedVirtualDatabase@controller2

DistributedVirtualDatabase@controller1

DistributedRequestManager

execWriteRequest(AbstractWriteRequest) return result

Distributed scheduling handler

DistributedRequestManager

Scheduler

Write query handler

Scheduler

Load Balancer

Distributed scheduling handler

Send request to all controllers Wait for schedulers replies Send execute request order to all controllers

Distributed load balancer handler

Wait for load balancer replies Notify scheduler of completion

Notify controllers of completion success or not Return result

Distributed load balancer handler

Load Balancer

Notify scheduler of completion

DistributedVirtualDatabase@controller3 DistributedRequestManager Distributed scheduling handler Distributed load balancer handler

Scheduler

Load Balancer

Notify scheduler of completion

http://c-jdbc.objectweb.org/ - [email protected]

64 - 07/11/2003

Horizontal scalability

Solution: No backend sharing  1 multicast + n unicast [+ 1 multicast]

Controller1

DB1

DB2

Controller2

DB3

DB4

Controller3

DB5

DB6

DistributedVirtualDatabase@controller2 DistributedVirtualDatabase@controller1

DistributedRequestManager

execWriteRequest(AbstractWriteRequest) return result DistributedRequestManager

Distributed scheduling handler

Scheduler

Distributed load balancer handler

Load Balancer

Write query handler

Scheduler

Load Balancer

Send request to all controllers

Distributed scheduling handler

Wait if all backends failed

DistributedVirtualDatabase@controller3

Distributed load balancer handler Wait for load balancer replies

Wait if all backends failed

Notify controller(s) that have failed if other have succeeded Return result

DistributedRequestManager Distributed scheduling handler Distributed load balancer handler

Scheduler

Load Balancer

Wait if all backends failed

http://c-jdbc.objectweb.org/ - [email protected]

65 - 07/11/2003

Horizontal scalability

Issues with JGroups  resources needed by a channel  instability of throughput with UDP  performance scalability

TCP better than UDP but  unable to disable reliability on top of TCP  unable to disable garbage collection  ordering implementation is sub-optimal

Need for a new group communication layer optimized for cluster http://c-jdbc.objectweb.org/ - [email protected]

66 - 07/11/2003

Horizontal scalability  JGroups performance on UDP/FastEthernet

http://c-jdbc.objectweb.org/ - [email protected]

67 - 07/11/2003

USE CASES

Budget High Availability  High availability infrastructure “on a budget”  Typical eCommerce setup

 http://www.budget-ha.com

http://c-jdbc.objectweb.org/ - [email protected]

69 - 07/11/2003

OpenUSS: University Support System

 eLearning  High availability  Portability

Apache

Apache

Apache

Enhydra Director

Enhydra Director

Enhydra Director

 Linux, HP-UX, Windows  InterBase, Firebird, PostgreSQL, HypersonicSQL

OpenUSS

OpenUSS

OpenUSS

OpenUSS

Enhydra server

Enhydra server

Enhydra server

Enhydra server

JOnAS J2EE server

JOnAS J2EE server

JOnAS J2EE server

JOnAS J2EE server

C-JDBC driver

C-JDBC driver

C-JDBC driver

C-JDBC driver

JVM

JVM

JVM

JVM

 http://openuss.sourceforge.net

C-JDBC Controller Full replication

C-JDBC Controller Full replication

Firebird JDBC driver

Firebird JDBC driver

Firebird http://c-jdbc.objectweb.org/ - [email protected]

Firebird 70 - 07/11/2003

Flood alert system  Disaster recovery  Independent nodes synchronized with C-JDBC  VPN for security issues  http://floodalert.org

Flood alert system

Flood alert system

Flood alert system

JBoss J2EE server

JBoss J2EE server

JBoss J2EE server

C-JDBC driver

C-JDBC driver

C-JDBC driver

C-JDBC Controller Full replication

C-JDBC Controller Full replication

MySQL JDBC driver

MySQL JDBC driver

MySQL JDBC driver

MySQL

MySQL

MySQL

http://c-jdbc.objectweb.org/ - [email protected]

> 1000 miles

C-JDBC Controller Full replication

71 - 07/11/2003

J2EE benchmarking

Internet

 Large scale J2EE clusters

emulated users

 http://jmob.objectweb.org

http://c-jdbc.objectweb.org/ - [email protected]

72 - 07/11/2003

PERFORMANCE

TPC-W Browsing mix performance

http://c-jdbc.objectweb.org/ - [email protected]

74 - 07/11/2003

TPC-W Shopping mix performance

http://c-jdbc.objectweb.org/ - [email protected]

75 - 07/11/2003

TPC-W Ordering mix performance

http://c-jdbc.objectweb.org/ - [email protected]

76 - 07/11/2003