. 7 - 07/11/2003. Outline. ➡
RAIDb. ➡C-JDBC. ➡Tomcat and C-JDBC. ➡Scalability. ➡High availability ...
Highly available web sites with Tomcat and Clustered JDBC Emmanuel Cecchet
Motivations Database tier should be scalable highly available without modifying the client application database vendor independent on commodity hardware JDBC JDBC
JDBC
Internet
http://c-jdbc.objectweb.org/ -
[email protected]
2 - 07/11/2003
Scaling the database tier – Alternative 1 (master-slave)
Cons failover time on master failure scalability Web frontend
App. server
Master
Internet
http://c-jdbc.objectweb.org/ -
[email protected]
3 - 07/11/2003
Scaling the database tier – Alternative 2 (atomic broadcast)
Cons atomic broadcast scalability no client side load balancing heavy modifications of the database engine
Internet
Atomic broadcast
http://c-jdbc.objectweb.org/ -
[email protected]
4 - 07/11/2003
Scaling the database tier – Alternative 3 (SMP)
Cons Cost Scalability limit Web frontend
App. server Database
Internet
Well-known database vendor here Well-known hardware + database vendors here http://c-jdbc.objectweb.org/ -
[email protected]
5 - 07/11/2003
Scaling the database tier – Alternative 4 (shared disks)
Cons still expensive hardware availability Web frontend
App. server
Database
Disks
Internet
Another well-known database vendor http://c-jdbc.objectweb.org/ -
[email protected]
6 - 07/11/2003
Outline
RAIDb C-JDBC Tomcat and C-JDBC Scalability High availability
http://c-jdbc.objectweb.org/ -
[email protected]
7 - 07/11/2003
RAIDb - Definition
Redundant Array of Inexpensive Databases better performance and fault tolerance than a single database, at a low cost, by combining multiple database instances into an array of databases RAIDb levels offers various tradeoff of performance and fault tolerance
http://c-jdbc.objectweb.org/ -
[email protected]
8 - 07/11/2003
Key ideas
RAIDb controller gives the view of a single database to the client balance the load on the database backends
RAIDb levels RAIDb-0: full partitioning RAIDb-1: full mirroring RAIDb-2: partial replication composition possible
http://c-jdbc.objectweb.org/ -
[email protected]
9 - 07/11/2003
RAIDb levels
RAIDb-0 partitioning no duplication and no fault tolerance at least 2 nodes SQL requests RAIDb controller
table 1
table 2 & 3
http://c-jdbc.objectweb.org/ -
[email protected]
table ...
table n-1
table n 10 - 07/11/2003
RAIDb levels
RAIDb-1 mirroring performance bounded by write broadcast at least 2 nodes SQL requests RAIDb controller
Full DB http://c-jdbc.objectweb.org/ -
[email protected]
Full DB
Full DB
Full DB
Full DB 11 - 07/11/2003
RAIDb levels
RAIDb-2 partial replication at least 2 copies of each table at least 3 nodes SQL requests RAIDb controller
Full DB
table x
http://c-jdbc.objectweb.org/ -
[email protected]
table y
table x & y
table z 12 - 07/11/2003
RAIDb levels composition SQL requests
RAIDb-1-0
RAIDb-1 controller
no limit to the composition deepness
RAIDb-0 controller
table w
table x & y
RAIDb-0 controller
table z
table w
table y
table x & z
RAIDb-0 controller
table w http://c-jdbc.objectweb.org/ -
[email protected]
table x
table y
table z 13 - 07/11/2003
Outline
RAIDb C-JDBC Tomcat and C-JDBC Scalability High availability
http://c-jdbc.objectweb.org/ -
[email protected]
14 - 07/11/2003
C-JDBC overview
Middleware implementing RAIDb 100% Java implementation open source (LGPL)
Two components generic JDBC driver (C-JDBC driver) C-JDBC Controller
Read-one, Write all approach Supports heterogeneous databases
http://c-jdbc.objectweb.org/ -
[email protected]
15 - 07/11/2003
architectural overview
Application server C-JDBC MySQL JDBC driver JVM
http://c-jdbc.objectweb.org/ -
[email protected]
C-JDBC controller MySQL database JVM
16 - 07/11/2003
Inside the C-JDBC Controller Java client program (Servlet, EJB, ...)
C-JDBC driver
RMI Sockets
RMI Sockets
C-JDBC Controller XML configuration file Administration console
JMX
Configuration & administration
Virtual database
Virtual database
Authentication Manager
Authentication Manager
Request Manager
Request Manager
XML engine
Scheduler Recovery Log
Request Cache
Scheduler Recovery Log
Request Cache
Load balancer
http://c-jdbc.objectweb.org/ -
[email protected]
Load balancer
Database Backend
Database Backend
Database Backend
Database Backend
Database Backend
Connection Manager
Connection Manager
Connection Manager
Connection Manager
Connection Manager
MySQL JDBC driver
MySQL JDBC driver
MySQL JDBC driver
Oracle JDBC driver
Oracle JDBC driver
MySQL
MySQL
MySQL
Oracle
Oracle 17 - 07/11/2003
Virtual Database
gives the view of a single
Java client program (Servlet, EJB, ...)
C-JDBC driver
Virtual database Authentication Manager Request Manager Scheduler Recovery Log
Request Cache Load balancer
Database Backend
Database Backend
Database Backend
Connection Manager
Connection Manager
Connection Manager
MySQL JDBC driver
MySQL JDBC driver
MySQL JDBC driver
MySQL
MySQL
MySQL
database establishes the mapping between the database name used by the application and the backend specific settings backends can be added and removed dynamically configured using an XML configuration file
http://c-jdbc.objectweb.org/ -
[email protected]
18 - 07/11/2003
Authentication Manager
Matches real login/password
Java client program (Servlet, EJB, ...)
C-JDBC driver
Virtual database Authentication Manager Request Manager Scheduler Recovery Log
Request Cache Load balancer
Database Backend
Database Backend
Database Backend
Connection Manager
Connection Manager
Connection Manager
MySQL JDBC driver
MySQL JDBC driver
MySQL JDBC driver
MySQL
MySQL
MySQL
used by the application with backend specific login/ password Administrator login to manage the virtual database
http://c-jdbc.objectweb.org/ -
[email protected]
19 - 07/11/2003
Scheduler
Manages concurrency control Specific implementations for
Java client program (Servlet, EJB, ...)
C-JDBC driver
Virtual database Authentication Manager Request Manager Scheduler Recovery Log
Request Cache Load balancer
Database Backend
Database Backend
Database Backend
Connection Manager
Connection Manager
Connection Manager
MySQL JDBC driver
MySQL JDBC driver
MySQL JDBC driver
MySQL
MySQL
MySQL
Single DB, RAIDb 0, 1 and 2 Query-level Optimistic and pessimistic transaction level
http://c-jdbc.objectweb.org/ -
[email protected]
uses the database schema that is automatically fetched from backends
20 - 07/11/2003
Request cache caches results from SQL requests improved SQL statement analysis to
Java client program (Servlet, EJB, ...)
C-JDBC driver
limit cache invalidations
Virtual database
table based invalidations column based invalidations single-row SELECT optimization
Authentication Manager Request Manager Scheduler Recovery Log
Request Cache Load balancer
Database Backend
Database Backend
Database Backend
Connection Manager
Connection Manager
Connection Manager
MySQL JDBC driver
MySQL JDBC driver
MySQL JDBC driver
MySQL
MySQL
MySQL
request parsing possible in the
http://c-jdbc.objectweb.org/ -
[email protected]
C-JDBC driver offload the controller parsing caching in the driver
21 - 07/11/2003
Load balancer 1/2 RAIDb-0
Java client program (Servlet, EJB, ...)
query directed to the backend having the
C-JDBC driver
needed tables Virtual database Authentication Manager
RAIDb-1 read executed by current thread write executed in parallel by a dedicated
Request Manager Scheduler Recovery Log
thread per backend result returned if one, majority or all commit if one node fails but others succeed, failing node is disabled
Request Cache Load balancer
Database Backend
Database Backend
Database Backend
Connection Manager
Connection Manager
Connection Manager
MySQL JDBC driver
MySQL JDBC driver
MySQL JDBC driver
RAIDb-2 same as RAIDb-1 except that writes are sent only to nodes owning the written table
MySQL
MySQL
MySQL
http://c-jdbc.objectweb.org/ -
[email protected]
22 - 07/11/2003
Load balancer 2/2
Static load balancing policies
Java client program (Servlet, EJB, ...)
C-JDBC driver
Round-Robin (RR) Weighted Round-Robin (WRR)
Virtual database Authentication Manager Request Manager
Least Pending Requests First
Scheduler Recovery Log
Request Cache Load balancer
Database Backend
Database Backend
Database Backend
Connection Manager
Connection Manager
Connection Manager
MySQL JDBC driver
MySQL JDBC driver
MySQL JDBC driver
MySQL
MySQL
MySQL
http://c-jdbc.objectweb.org/ -
[email protected]
(LPRF) request sent to the node that has the shortest pending request queue efficient if backends are homogeneous in terms of performance
23 - 07/11/2003
Connection Manager
Connection pooling for a
Java client program (Servlet, EJB, ...)
C-JDBC driver
backend Simple : no pooling RandomWait : blocking pool FailFast : non-blocking pool VariablePool : dynamic pool
Virtual database Authentication Manager Request Manager Scheduler Recovery Log
Request Cache Load balancer
Database Backend
Database Backend
Database Backend
Connection Manager
Connection Manager
Connection Manager
MySQL JDBC driver
MySQL JDBC driver
MySQL JDBC driver
MySQL
MySQL
MySQL
Connection pools defined on a
http://c-jdbc.objectweb.org/ -
[email protected]
per login basis resource management per login dedicated connections for admin 24 - 07/11/2003
Recovery Log
Checkpoints are associated
Java client program (Servlet, EJB, ...)
C-JDBC driver
Virtual database Authentication Manager Request Manager Scheduler Recovery Log
Request Cache Load balancer
Database Backend
Database Backend
Database Backend
Connection Manager
Connection Manager
Connection Manager
MySQL JDBC driver
MySQL JDBC driver
MySQL JDBC driver
MySQL
MySQL
MySQL
with database dumps Record all updates and transaction markers since a checkpoint Used to resynchronize a database from a checkpoint JDBCRecoveryLog
http://c-jdbc.objectweb.org/ -
[email protected]
store information in a database can be re-injected in a C-JDBC cluster for fault tolerance 25 - 07/11/2003
Functional overview Client application (Servlet, EJB, ...)
Client application (Servlet, EJB, ...)
connect login, password C-JDBC driver connect myDB Controller execute SELECT * FROM t C-JDBC Virtual database 1
C-JDBC driver
Authentication Manager Request Manager Scheduler Recovery Query result cache Log Load balancer
http://c-jdbc.objectweb.org/ -
[email protected]
Database Backend
Database Backend
Database Backend
Connection Manager
Connection Manager
Connection Manager
MySQL JDBC driver
MySQL JDBC driver
MySQL JDBC driver
MySQL
MySQL
MySQL 26 - 07/11/2003
Functional overview Client application (Servlet, EJB, ...)
Client application (Servlet, EJB, ...)
C-JDBC driver
C-JDBC driver
execute INSERT INTO t …
C-JDBC Controller Virtual database 1 Authentication Manager Request Manager Scheduler Recovery Query result cache Log Load balancer
http://c-jdbc.objectweb.org/ -
[email protected]
Database Backend
Database Backend
Database Backend
Connection Manager
Connection Manager
Connection Manager
MySQL JDBC driver
MySQL JDBC driver
MySQL JDBC driver
MySQL
MySQL
MySQL 27 - 07/11/2003
Failures execute INSERT INTO t …
Client application (Servlet, EJB, ...)
Client application (Servlet, EJB, ...)
C-JDBC driver
C-JDBC driver
C-JDBC Controller
No 2 phasecommit parallel transactions failed nodes are automatically disabled
http://c-jdbc.objectweb.org/ -
[email protected]
Virtual database 1 Authentication Manager Request Manager Scheduler Recovery Query result cache Log Load balancer Database Backend
Database Backend
Database Backend
Connection Manager
Connection Manager
Connection Manager
MySQL JDBC driver
MySQL JDBC driver
MySQL JDBC driver
MySQL
MySQL
MySQL 28 - 07/11/2003
Outline
RAIDb C-JDBC Tomcat and C-JDBC Scalability High availability
http://c-jdbc.objectweb.org/ -
[email protected]
29 - 07/11/2003
Highly available web site Apache clustering L4 switch, RR-DNS, One-IP techniques, LVS, …
Tomcat clustering mod-jk, session replication
Database clustering C-JDBC
RR-DNS
mod-jk
Parsing cache Result cache Metadata cache
Internet
http://c-jdbc.objectweb.org/ -
[email protected]
30 - 07/11/2003
Result cache
Cache contains a list of SQL_ResultSet Policy defined by queryPattern_Policy 3 policies EagerCaching: variable granularities for invalidations RelaxedCaching: invalidations based on timeout NoCaching: never cached RUBiS bidding mix with 450 clients Throughput (rq/min)
No cache
Coherent cache
Relaxed cache
3892
4184
4215
Avg response time
801 ms
284 ms
134 ms
Database CPU load
100%
85%
20%
C-JDBC CPU load
-
15%
7%
http://c-jdbc.objectweb.org/ -
[email protected]
31 - 07/11/2003
Configuring Tomcat 5
Same setting for all Tomcat nodes Copy c-jdbc-driver.jar to $CATALINA_HOME/common/lib
Without connection pooling 1. load driver Class.forName("org.objectweb.cjdbc.driver.Driver") .newInstance();
2. get a connection Connection conn = DriverManager. getConnection("jdbc:cjdbc://host/myDB","user","password"); http://c-jdbc.objectweb.org/ -
[email protected]
32 - 07/11/2003
Configuring Tomcat 5
With connection pooling factory org.apache.commons.dbcp.BasicDataSourceFactory driverClassName org.objectweb.cjdbc.driver.Driver url jdbc:cjdbc://localhost/myDB http://c-jdbc.objectweb.org/ -
[email protected]
33 - 07/11/2003
Configuring C-JDBC Virtual database configuration file
http://c-jdbc.objectweb.org/ -
[email protected]
36 - 07/11/2003
C-JDBC vertical scalability
allows nested RAIDb levels allows tree architecture for scalable write broadcast necessary with large number of backends C-JDBC driver re-injected in C-JDBC controller
http://c-jdbc.objectweb.org/ -
[email protected]
37 - 07/11/2003
C-JDBC vertical scalability
RAIDb-1-1
Client program
with C-JDBC no limit to composition deepness
C-JDBC driver JVM
Client program
Client program
C-JDBC driver JVM
C-JDBC driver JVM
C-JDBC controller RAIDb-1 C-JDBC driver
C-JDBC controller RAIDb-1
C-JDBC controller RAIDb-1
C-JDBC controller RAIDb-1
MySQL JDBC driver
MySQL JDBC driver
MySQL JDBC driver
MySQL
MySQL
MySQL
http://c-jdbc.objectweb.org/ -
[email protected]
MySQL
MySQL
MySQL
MySQL
MySQL 38 - 07/11/2003
Vertical scalability Addresses JVM scalability issues Distributing large number of connections on many backends
Client program
Client program
C-JDBC driver JVM
C-JDBC driver JVM
Client program C-JDBC driver JVM
C-JDBC controller Partial replication C-JDBC driver
C-JDBC controller Partial replication
C-JDBC controller Partial replication
C-JDBC controller Partial replication
DB native JDBC driver
DB native JDBC driver
DB native JDBC driver
DB 1 http://c-jdbc.objectweb.org/ -
[email protected]
DB 2
DB 3
DB 4
DB 5
DB 6
DB7 39 - 07/11/2003
TPC-W benchmark (Amazon.com)
Throughput in requests per minute
Nearly linear speedups with the shopping mix 1600 1400 1200 1000
Single Database
800
Full replication
600
Partial replication
400 200 0 0
1
2
3
4
5
6
Number of nodes http://c-jdbc.objectweb.org/ -
[email protected]
40 - 07/11/2003
Outline
RAIDb C-JDBC Tomcat and C-JDBC Scalability High availability
http://c-jdbc.objectweb.org/ -
[email protected]
41 - 07/11/2003
Controller replication Prevent the controller from being a single point of failure Group communication for controller synchronization C-JDBC driver supports multiple controllers with automatic failover
jdbc:c-jdbc://node1:25322,node2:12345/myDB Java client program
C-JDBC driver JVM
C-JDBC driver JVM
C-JDBC Controller MySQL PostgreSQL JDBC driver JDBC driver
MySQL
http://c-jdbc.objectweb.org/ -
[email protected]
Java client program
PostgreSQL
JGroups
Java client program
Java client program
C-JDBC driver
C-JDBC driver
JVM
JVM
C-JDBC Controller PostgreSQL MySQL JDBC driver JDBC driver
PostgreSQL
MySQL
42 - 07/11/2003
Controller replication Client application (Servlet, EJB, ...)
Client application (Servlet, EJB, ...)
Client application (Servlet, EJB, ...)
C-JDBC driver
C-JDBC driver
C-JDBC driver
jdbc:cjdbc://node1,node2/myDB Total order reliable multicast
C-JDBC Controller
C-JDBC Controller
Distributed Request Manager
Distributed Request Manager
Virtual database 1
Virtual database 2
Authentication Manager
Authentication Manager
Request Manager
Request Manager
Scheduler
Scheduler
Query result cache
Query result cache
Load balancer
Load balancer
Database Backend
Database Backend
Database Backend
Database Backend
Connection Manager
Connection Manager
Connection Manager
Connection Manager
MySQL JDBC driver
MySQL JDBC driver
MySQL JDBC driver
Oracle JDBC driver
MySQL
MySQL
MySQL
Oracle
http://c-jdbc.objectweb.org/ -
[email protected]
43 - 07/11/2003
Client program
Mixing horizontal & vertical scalability
C-JDBC driver JVM
Client program
Client program
C-JDBC driver JVM
C-JDBC driver
C-JDBC controller Full replication
C-JDBC controller Full replication DB native JDBC driver
JVM
C-JDBC driver
C-JDBC driver
C-JDBC controller Full replication DB 1
DB 2
DB native C-JDBC JDBC driver driver
C-JDBC controller Full replication DB native JDBC driver
DB 3
http://c-jdbc.objectweb.org/ -
[email protected]
DB 4
DB 5
DB 6
DB 7
44 - 07/11/2003
Building initial checkpoint Octopus is an ETL tool Use Octopus to store a dump of the initial database state Currently done by the user using the database specific dump tool Octopus dump for initial checkpoint
EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...
C-JDBC driver JVM
disabled
C-JDBC Controller Recovery Log PostgreSQL JDBC driver
PostgreSQL disabled http://c-jdbc.objectweb.org/ -
[email protected]
45 - 07/11/2003
Logging Backend is enabled All database updates are logged (SQL statement, user, transaction, …)
EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...
C-JDBC driver JVM
JDBC Recovery Log
Octopus dump for initial checkpoint
enabled
C-JDBC Controller Recovery Log PostgreSQL JDBC driver
PostgreSQL enabled http://c-jdbc.objectweb.org/ -
[email protected]
46 - 07/11/2003
Adding new backends 1/3 Add new backends
EJB Container
while system online Restore dump corresponding to JDBC initial checkpoint Recovery Log with Octopus Octopus
JOnAS, WebLogic, JBoss, WebSphere, ...
C-JDBC driver JVM
enabled
C-JDBC Controller Recovery Log PostgreSQL JDBC driver
dump for initial checkpoint
PostgreSQL PostgreSQL disabled enabled http://c-jdbc.objectweb.org/ -
[email protected]
PostgreSQL disabled 47 - 07/11/2003
Adding new backends 2/3 EJB Container
Replay updates from the log
JOnAS, WebLogic, JBoss, WebSphere, ...
C-JDBC driver JVM
JDBC Recovery Log
enabled
C-JDBC Controller Recovery Log
Octopus
PostgreSQL JDBC driver
dump for initial checkpoint
PostgreSQL disabled http://c-jdbc.objectweb.org/ -
[email protected]
PostgreSQL enabled
PostgreSQL disabled 48 - 07/11/2003
Adding new backends 3/3 EJB Container
Enable backends
JOnAS, WebLogic, JBoss, WebSphere, ...
when done
C-JDBC driver JVM
JDBC Recovery Log
enabled
C-JDBC Controller Recovery Log
Octopus
PostgreSQL JDBC driver
dump for initial checkpoint
PostgreSQL enabled http://c-jdbc.objectweb.org/ -
[email protected]
PostgreSQL enabled
PostgreSQL enabled 49 - 07/11/2003
Making new checkpoints Disable one backend to have a coherent snapshot Mark the new checkpoint entry in the log Use Octopus to store JDBC the dump Recovery Log
EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...
C-JDBC driver JVM
enabled
C-JDBC Controller Recovery Log
dump for initial checkpoint
...
Octopus
PostgreSQL JDBC driver
dump for dump last for last checkpoint checkpoint
PostgreSQL disabled http://c-jdbc.objectweb.org/ -
[email protected]
PostgreSQL enabled
PostgreSQL enabled 50 - 07/11/2003
Making new checkpoints Replay missing updates from log
EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...
C-JDBC driver JVM
JDBC Recovery Log
enabled
C-JDBC Controller Recovery Log
dump for initial checkpoint
...
Octopus
PostgreSQL JDBC driver
dump for dump last forfor last checkpoint checkpoint
PostgreSQL disabled http://c-jdbc.objectweb.org/ -
[email protected]
PostgreSQL enabled
PostgreSQL enabled 51 - 07/11/2003
Making new checkpoints Re-enable backend
EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...
when done
C-JDBC driver JVM
JDBC Recovery Log
enabled
C-JDBC Controller Recovery Log
dump for initial checkpoint
...
Octopus
PostgreSQL JDBC driver
dump for dump last for for last checkpoint checkpoint
PostgreSQL enabled http://c-jdbc.objectweb.org/ -
[email protected]
PostgreSQL enabled
PostgreSQL enabled 52 - 07/11/2003
Handling failures A node fails! Automatically
EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...
disabled but should be fixed or changed JDBC by administrator Recovery Log
C-JDBC driver JVM
enabled
C-JDBC Controller Recovery Log
dump for initial checkpoint
...
Octopus
PostgreSQL JDBC driver
dump for dump last for for last checkpoint checkpoint
PostgreSQL disabled http://c-jdbc.objectweb.org/ -
[email protected]
PostgreSQL enabled
PostgreSQL enabled 53 - 07/11/2003
Recovery 1/3 Restore latest dump with Octopus
EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...
C-JDBC driver JVM
JDBC Recovery Log
enabled
C-JDBC Controller Recovery Log
dump for initial checkpoint
...
Octopus
PostgreSQL JDBC driver
dump for dump last for last checkpoint checkpoint
PostgreSQL disabled http://c-jdbc.objectweb.org/ -
[email protected]
PostgreSQL enabled
PostgreSQL enabled 54 - 07/11/2003
Recovery 2/3 Replay missing updates from log
EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...
C-JDBC driver JVM
JDBC Recovery Log
enabled
C-JDBC Controller Recovery Log
dump for initial checkpoint
...
Octopus
PostgreSQL JDBC driver
dump for dump last forfor last checkpoint checkpoint
PostgreSQL disabled http://c-jdbc.objectweb.org/ -
[email protected]
PostgreSQL enabled
PostgreSQL enabled 55 - 07/11/2003
Recovery 3/3 Re-enable backend
EJB Container JOnAS, WebLogic, JBoss, WebSphere, ...
when done
C-JDBC driver JVM
JDBC Recovery Log
enabled
C-JDBC Controller Recovery Log
dump for initial checkpoint
...
Octopus
PostgreSQL JDBC driver
dump for dump last for for last checkpoint checkpoint
PostgreSQL enabled http://c-jdbc.objectweb.org/ -
[email protected]
PostgreSQL enabled
PostgreSQL enabled 56 - 07/11/2003
Demo
Administration tools HypersonicSQL in-memory databases JMX
http://c-jdbc.objectweb.org/ -
[email protected]
57 - 07/11/2003
Conclusion
RAIDb C-JDBC open source middleware for database replication RDBMS independent
Tomcat & C-JDBC no application modification required performance scalability high availability transparent failover
http://c-jdbc.objectweb.org/ -
[email protected]
58 - 07/11/2003
Q&A _________ Thanks to all users and contributors ...
http://c-jdbc.objectweb.org http://c-jdbc.objectweb.org/ -
[email protected]
59 - 07/11/2003
Bonus slides
HORIZONTAL SCALABILITY
Horizontal scalability
JGroups for controller synchronization Groups messages for writes only Client 1
Client...
Client n
DB2
DB3
http://c-jdbc.objectweb.org/ -
[email protected]
Client...
Controller1
Controller1
DB1
Client 1
DB4
DB1
DB2
Client n
Controller2
DB3
DB4
62 - 07/11/2003
Horizontal scalability
Centralized write approach issues co
co
Con
D
D
Con
D
Con
D
D
D
co
Con
D
D
co
Con
D
D
?
D
D
? ?
?
Issues with transactions assigned to connections T1(Write)
T1(Read) Controller1
Controller2
Controller1
Controller2
Controller1
Controller2
DB1
DB2
DB1
DB2
DB1
DB2
http://c-jdbc.objectweb.org/ -
[email protected]
T1(R)
T1(W)
T1(R) T2(W) 63 - 07/11/2003
Horizontal scalability
General case for a write query 3 multicast + 2n unicast DistributedVirtualDatabase@controller2
DistributedVirtualDatabase@controller1
DistributedRequestManager
execWriteRequest(AbstractWriteRequest) return result
Distributed scheduling handler
DistributedRequestManager
Scheduler
Write query handler
Scheduler
Load Balancer
Distributed scheduling handler
Send request to all controllers Wait for schedulers replies Send execute request order to all controllers
Distributed load balancer handler
Wait for load balancer replies Notify scheduler of completion
Notify controllers of completion success or not Return result
Distributed load balancer handler
Load Balancer
Notify scheduler of completion
DistributedVirtualDatabase@controller3 DistributedRequestManager Distributed scheduling handler Distributed load balancer handler
Scheduler
Load Balancer
Notify scheduler of completion
http://c-jdbc.objectweb.org/ -
[email protected]
64 - 07/11/2003
Horizontal scalability
Solution: No backend sharing 1 multicast + n unicast [+ 1 multicast]
Controller1
DB1
DB2
Controller2
DB3
DB4
Controller3
DB5
DB6
DistributedVirtualDatabase@controller2 DistributedVirtualDatabase@controller1
DistributedRequestManager
execWriteRequest(AbstractWriteRequest) return result DistributedRequestManager
Distributed scheduling handler
Scheduler
Distributed load balancer handler
Load Balancer
Write query handler
Scheduler
Load Balancer
Send request to all controllers
Distributed scheduling handler
Wait if all backends failed
DistributedVirtualDatabase@controller3
Distributed load balancer handler Wait for load balancer replies
Wait if all backends failed
Notify controller(s) that have failed if other have succeeded Return result
DistributedRequestManager Distributed scheduling handler Distributed load balancer handler
Scheduler
Load Balancer
Wait if all backends failed
http://c-jdbc.objectweb.org/ -
[email protected]
65 - 07/11/2003
Horizontal scalability
Issues with JGroups resources needed by a channel instability of throughput with UDP performance scalability
TCP better than UDP but unable to disable reliability on top of TCP unable to disable garbage collection ordering implementation is sub-optimal
Need for a new group communication layer optimized for cluster http://c-jdbc.objectweb.org/ -
[email protected]
66 - 07/11/2003
Horizontal scalability JGroups performance on UDP/FastEthernet
http://c-jdbc.objectweb.org/ -
[email protected]
67 - 07/11/2003
USE CASES
Budget High Availability High availability infrastructure “on a budget” Typical eCommerce setup
http://www.budget-ha.com
http://c-jdbc.objectweb.org/ -
[email protected]
69 - 07/11/2003
OpenUSS: University Support System
eLearning High availability Portability
Apache
Apache
Apache
Enhydra Director
Enhydra Director
Enhydra Director
Linux, HP-UX, Windows InterBase, Firebird, PostgreSQL, HypersonicSQL
OpenUSS
OpenUSS
OpenUSS
OpenUSS
Enhydra server
Enhydra server
Enhydra server
Enhydra server
JOnAS J2EE server
JOnAS J2EE server
JOnAS J2EE server
JOnAS J2EE server
C-JDBC driver
C-JDBC driver
C-JDBC driver
C-JDBC driver
JVM
JVM
JVM
JVM
http://openuss.sourceforge.net
C-JDBC Controller Full replication
C-JDBC Controller Full replication
Firebird JDBC driver
Firebird JDBC driver
Firebird http://c-jdbc.objectweb.org/ -
[email protected]
Firebird 70 - 07/11/2003
Flood alert system Disaster recovery Independent nodes synchronized with C-JDBC VPN for security issues http://floodalert.org
Flood alert system
Flood alert system
Flood alert system
JBoss J2EE server
JBoss J2EE server
JBoss J2EE server
C-JDBC driver
C-JDBC driver
C-JDBC driver
C-JDBC Controller Full replication
C-JDBC Controller Full replication
MySQL JDBC driver
MySQL JDBC driver
MySQL JDBC driver
MySQL
MySQL
MySQL
http://c-jdbc.objectweb.org/ -
[email protected]
> 1000 miles
C-JDBC Controller Full replication
71 - 07/11/2003
J2EE benchmarking
Internet
Large scale J2EE clusters
emulated users
http://jmob.objectweb.org
http://c-jdbc.objectweb.org/ -
[email protected]
72 - 07/11/2003
PERFORMANCE
TPC-W Browsing mix performance
http://c-jdbc.objectweb.org/ -
[email protected]
74 - 07/11/2003
TPC-W Shopping mix performance
http://c-jdbc.objectweb.org/ -
[email protected]
75 - 07/11/2003
TPC-W Ordering mix performance
http://c-jdbc.objectweb.org/ -
[email protected]
76 - 07/11/2003