An Adaptable Infrastructure for Customized Persistent Object ...

An Adaptable Infrastructure for Customized Persistent Object Management Luciano Garc´ıa-Ba˜ nuelos

LSR/IMAG Laboratory 681, rue de la Passerelle 38400 Saint Martin d’Hères, FRANCE [email protected]

1

Context and Motivations

At the heart of information systems, database management system (DBMS) technology has been evolving continuously to meet new application requirements. Accordingly, existing DBMSs have achieved a remarkable complexity, so that adding and/or removing functionalities can be sometimes a hard engineering/research task. Although the need for more powerful DBMSs is more than ever evident, there are also contexts where overheads imposed by the whole DBMS machinery cannot be afforded. This is the case of data management applications on resource constrained systems, e.g. mobile computers/devices. The single solution for scaling down DBMS to those environments seems to be building new ad hoc software. But the problem is not only down sizing. We can evoke some examples of data intensive applications, where it has been decided not to use DBMSs. Unfortunately, current DBMSs are not meant to be used in detached parts, so developing new modules is the only available choice. This work is part of the NODS project [8] whose aim is to unbundle DBMSs into a set of customizable services. This vision is shared by several other researchers (see for instance [26], [10], [12], [6], and [9]). However, as far as we know, there are few active projects trying to componentize DBMSs. Our main goal is to factor out persistence management from the rest of functionalities found in full-fledged DBMSs. We intend to provide system programmers with a framework to build infrastructures for persistence management meeting very particular needs. Further, we aim at tackling the problem of serving transactional and non-transactional applications with different configurations of a single set of primitive components. Hence, our vision is to provide primitive components to:

Supported by the CONACyT scholarship program of the Mexican Government Thesis advisors: Christine Collet, LSR/IMAG Lab., 681, rue de la Passerelle, 38400 St. Martin d’Hères, France, and Pascal D´ echamboux, France Telecom R&D, 28, chemin du Vieux Chêne, 38243 Meylan, France.

A.B. Chaudhri et al. (Eds.): EDBT 2002 Workshops, LNCS 2490, pp. 607–615, 2002. c Springer-Verlag Berlin Heidelberg 2002

608

L. Garc´ıa-Ba˜ nuelos

– Build persistence management support for applications where existing technology (e.g. DBMS) is ill-suited. – Integrate database services into multi-tier applications, i.e. database middleware. Note that our work is not the first attempt to provide persistence management as a service. In fact, there are several standard proposals. Further, there have been some earlier attempts to develop more adaptable DBMS software architectures. However, all those earlier proposals keep full-functional DBMSs as the fundamental unit of packaging. In contrast to those works, we claim that a finer-grained decomposition is possible. The rest of this paper is organized as follows. Our approach is briefly introduced in Section 2. Section 3 overviews the main contributions of our work. Section 4 is a short survey of works on data management architectures and proposals for persistence service. Finally, Section 5 depicts the current status of the project, and the outgoing and future work.

2

Approach

While revisiting DBMS architectures, we claim that a component-based approach has to be taken. However, a correct componentization of DBMS software compels a deep theoretical study because of the subtle dependencies kept by their functional constituents. The road map towards dissecting persistence management from the whole DBMS machinery is shown in Figure 1. A general explanation follows: Componentization – This phase implied the definition of functional dimensions characterizing persistence management. In addition, we had to establish the boundaries of persistence management with respect to other system aspects such as: fault tolerance and concurrent access supports. Also, we defined software architecture principles to promote component reuse. Reasoning about dependencies – We continued by establishing component interdependencies and frameworks to provide ways to effectively separate and/or assemble components. In particular, we concentrated in characterizing complex relationships between persistence management and other system aspects, e.g. how to dissect fault tolerance from persistence management? Customized persistence management – Finally, we devised tools and techniques supporting to multiple software configurations such as: persistence with no fault tolerance, persistence with crash recovery, and persistence with transaction support. All of these steps are developed further in the following section.

An Adaptable Infrastructure for Customized Persistent Object Management

609

Full-fledged DBMS

TxMgmt Recovery Persistence Querying

TxMgmt Caching Locking

Componentizing Persistence Management

Logging Storage Persistence Recovery

Reasonning about dependencies

Non-transactional reliable persistent object management Caching flushing order Transactional persistent object management

Assembling customized infrastructures

Storage

Logging

Fig. 1. Towards a customizable persistence management platform: The road map

3

Contributions

3.1

Persistence Infrastructure Componentization

One important task was to establish the architectural principles allowing software component reuse. A deep analysis of the state-of-the-art techniques on persistence management has been carried out. After that, we defined a set coarse grain software components, that includes: Persistent storage management – This component is an abstraction to mask existing persistent infrastructures (e.g. DBMS) as well as ad hoc storage managers. In memory object management – This component packs aspects such as object caching, pre-fetching and flushing order management. Pointer swizzling and object faulting can be also considered as part of this component. Indexing framework – This is mostly a framework to build indexing platforms used by other components, e.g. the persistent storage manager. Log management – This component provides a generic service to record system information, useful to implement recovery related actions. We preferred logging instead of techniques such as shadow paging, because it has been shown as a more flexible scheme. Concurrency control – This component provides a generic interface to easily accommodate a large range of concurrency control schemes.

610


Transaction manager – This component provides implementation for flat transactions as well as nested transactions. Note that concurrency control and transaction management are out of the scope of this work. However, we are interested in studying the interaction of persistence management with these two external services. The current view of our infrastructure is shown in Figure 2. Note that this infrastructure uses advanced software technologies such as aspect oriented programming [2].

PersistenceService

provides

extends (by aspect weaving) PersistenceManager

requires

requires CacheManager

StorageManager

ReliablePersistenceMng

IndexManager

(a)

MultilevelTxManager

requires

LogManager

(b)

ConcurrencyCtrl

(c)

Fig. 2. Software architecture: Components to build (a) unreliable, (b) reliable nontransactional, and (c) transactional persistence services.

3.2

Component Dependencies

We found that persistence maintains a tight relation with recovery. Persistence and recovery are concerned with transaction durability and atomicity. To clarify this dependency we looked into related previous works. Thus, we determined that the cited dependency is tangled into (1) flushing management (i.e. disk write scheduling), and (2) recovery algorithms. Both of those aspects are described further below. Finally, we noted that pre-fetching, pointer swizzling and object faulting are orthogonal to concurrency control and recovery. Thus, the can be studied independently. Dependency 1: Flushing and log management. Cooperation between flushing management and logging is the source of an important dependency between persistence and recovery methods. Note that this dependency is due to not only performance related issues but also to meet correctness in recovery techniques [17]. Recently, Lomet and Tuttle [17][18] provided the quite complete formal framework to understand the impact of log management choices on cache management, i.e. flushing order. As a result of this formalization, they proposed a set


611

of algorithms to implement a general cache manager. Another interesting work is that of Chew and Silberschatz [7]. They proposed two primitive operations to provide explicit hints about flush order dependencies. They argue that these two operations are powerful enough to describe flush dependencies such as those dictated by protocols such as the well known Write Ahead Log (WAL) protocol (see for instance [30]) or those imposed by the ARIES recovery algorithm [19]. Dependency 2: Recovery management. Most DBMS-like recovery algorithms deal at the same time with both system recovery after crash and transaction recovery [30]. If we want primitive components to be reusable, we have to break out this dependency. A solution is to build systems in a layered fashion. Thus, a low level layer can cope with system recovery, while another upper level layer with transaction recovery. This approach is inspired in multi-level transaction management [29]. Multi-level transactions are a variant of open-nested transactions in which sub-transactions correspond to operations at different levels of a layered system. A two level system can be one where disk page management is done by the lowest level, while object accesses are handled by the highest level. An implementation of such a system is presented in [29]. Additionally, it is suggested in [16][29] that ARIES [19] is actually an algorithm intrinsically multi-level. Thus, our proposal is to extend multi-level recovery to separate crash recovery and transaction recovery management. In addition, nested transaction support is straightforward if multi-level recovery is available. 3.3

Customized Persistence Management

After the set of software components constituting the persistence management infrastructure have been defined, as well as their interdependencies, we intend to provide tools to assemble infrastructures according to application requirement profiles. Some sample configurations follow: Unreliable storage – This configuration supposes that system crashes do not impact negatively data integrity. Examples of usage of such a configuration are in the context of a web caching and in read only systems. Reliable storage – In this case, only crash recovery is added to storage management. This means that there is not transaction support. An example of usage of such a configuration is in computer supported collaborative work, where ACID transactions are not well suited. Transactional storage – This is probably the widely used configuration for persistent systems. It supposes support for ACID transactions, including flat and/or nested ones.

612


4

Previous Related Work

4.1

Extensible DBMS

By the end of 80’s, the industrial and academic research communities started efforts for developing what was called the extensible database management systems. In the following, we give a general classification of those approaches. DBMS generators – The idea of generating some parts of the DBMS has been explored by some researchers. Examples of this approach include the Genesis project [1] and the Volcano query optimizer generator [11]. This approach has been apparently abandoned, presumably because of the difficulties found due to the intrinsic complexity of the DBMS software. DBMS with an extensible type subsystem – Some researchers studied ways to let users extend DBMS type system (i.e. ADT), and underlying access methods (i.e. indexes). Prototypes such as Postgres [27] at the University of Berkeley, and Starburst [24] at IBM are examples of this approach. Note that this approach has effectively evolved, and it is at the origin of the so called object-relational DBMSs. DBMS kernel toolkits – This approach has been taken in the Exodus [3], Shore [4] and DASDBS [25] projects. The idea was to build a minimal (core functionality) DBMS kernel along with a set of tools allowing to build ad hoc DBMSs. These projects have been testbed for lots of research developments. Note however that these prototypes were almost full-fledged DBMSs, so that lighter weight DBMS implementations could not be envisioned. 4.2

Persistence Service Standards

Among the proposals for persistence service standards, we can mention three of the most widely accepted: the Object Management Group’s Persistent State Service (PSS 2.0) [20], the Object Data Management Group standard (ODMG 3.0) [5], and the Java Data Objects (JDO) [15] at the Java Community Process. All of them attempt to provide a standard approach to integrate existing persistence technology (e.g. DBMS) by using a single homogeneous set of interfaces. Effectively, they intend to provide a framework for building persistent applications independently of the underlying support, thus promoting in some scale multi-vendor interoperability. The three proposals provide a transaction-based processing mode, except for PSS which also provides a non transactional approach. The proposals promote high level APIs (Application Programming Interfaces) hiding implementation details. JDO goes beyond revealing some system internals. Among those details, there are notably APIs to provide hints for cache replacement, hints for swizzling/object faulting scopes, for selection of concurrency control strategies (e.g. locking-based or optimistic approaches), etc. The OMG has taken from the beginning an service oriented approach. Hence, they proposed the following persistent related services [21]: transactions, querying, and concurrency control. However, although PSS specifies interaction with


613

the transaction service, nothing is said about interactions with querying and concurrency control services. The Java Community Process has started defining a generic cache service [14]. Note that the definition process is led by Oracle, so that a DBMS-like approach can be expected. However, there is no up-to-date information about interactions between JDO and the cited cache service. 4.3

Industrial Prototypes

As long as we know, the only commercial product proposing an approach similar to ours is that of the Poet’s FastObjects j2 [23] (aka. Navajo). FastObjects j2 is in fact an embedded DBMS intended to be used in resource constrained mobile devices (e.g. PDAs). This DBMS has a highly modular architecture so that it is possibly to add/remove some components, i.e. XML import/export, logging and synchronization modules. 4.4

Our Position

Earlier works have kept in mind the goal of building more and more complex DBMS. We argue, however that the process inverse has to be done, trying to breaking down DBMSs in simpler but meaningful software components. Our vision is close to the RISC-style data management blocks described in [6]. Thus, instead of talking about a DBMS minimal kernel, as done in Exodus [3] and Shore [4], we go beyond by proposing fine-grain software components for mechanisms such as caching, logging and recovery. We have tracked the development of standards for persistence, and other related services. We have witnessed, unfortunately, the lack for integration between those services. We are convinced that the understanding of the subtle dependencies inside the DBMS machinery can be useful to define well integrated services. Finally, we want to position our work as capitalizing previous results. We dispose of new software techniques, tools and research findings, which were not available at the time previous works were active.

5

Project Status and Future Work

We have already defined the first system architecture for the platform, and developed the very first version of the software components. These components include: cache manager with several replacement managers, index manager with B+ Tree and R* Tree, in-disk object manager, log manager. The ongoing work corresponds with the validation of all these components for building a persistence support for the PING project (Platform for Interactive Networked Games [22]). In that context, we validate the platform in a reliable persistent configuration with no transactions. At the same time, we work in the implementation of a locking-based concurrency controller and the transaction manager supporting

614


both flat and nested transactions. The latter would let us validate the multi-level recovery proposed techniques. As part of the future work we want to develop further the flushing management. Finally, we want to validate the platform in a transactional system software, for instance to integrate a persistence service for an EJB container [13]. Acknowledgments. I would like to thank my advisors, Christine Collet and Pascal Déchamboux, for their guidance and encouragements.

References 1. Don Batory, et al. GENESIS: An Extensible Database Management System. IEEE Transactions on Software Engineering 14(11):1711-1730, November 1988. 2. K. Czarnecki and U. Eisenecker. Aspect-Oriented Decomposition and Composition. In Generative Programming: Methods, Techniques, and Applications, AddisonWesley, June 2000. 3. Michael Carey, et al. The Architecture of the EXODUS Extensible DBMS. In Proceedings International Workshop on Object-Oriented Database Systems, Asilomar Conference Center CA, USA, September 1986. 4. Michael Carey, et al. Shoring Up Persistent Applications. In Proceedings of ACMSIGMOD 1994 International Conference on Management of Data, Minneapolis Minnesota, USA, May 1994. 5. G. G. Cattell, et al. The Object Data Standard: ODMG 3.0. Morgan Kaufmann Publishers, January 2000. 6. Surajit Chaudhuri and Gehrard Weikum. Rethinking Database System Architecture: Towards a Self-tuning RISC-style Database System. In Proceedings of the 26th International Conference on Very Large Data Bases, Cairo, Egypt, 2000. 7. Khien-Mien Chew and Avi Silberschatz. Toward Operating System Support for Recoverable-Persistent Main Memory Database Systems. Technical Report TR92-05, Department of Computer Sciences, University of Texas at Austin, February 1992. 8. Christine Collet. The NODS project: Networked Open Database Services. ECOOP Symposium on Objects and Databases, June 2000. 9. Klaus R. Dittrich and Andreas Geppert (eds). Component Database Systems. Morgan Kaufmann Publishers, October 2000. 10. Andreas Geppert and Klaus. R. Dittrich. Bundling: Towards a New Construction Paradigm for Persistent Systems. Networking and Information Systems, 1(1), June 1998. 11. G. Graefe and W.J. McKenna. The Volcano Optimizer Generator: Extensibility and Efficient Search. In Proceedings of the IEEE Conference on Data Engineering, Vienna, Austria, April 1993. 12. James Hamilton. Networked data management design points. In Proceedings of the 25th International Conference on Very Large Data Bases, Edinburgh, Scotland, UK, September 1999. 13. Java Community Process. Enterprise JavaBeansT M Specification 2.0. Proposed Final Draft JSR 000019, October 2000.


615

14. Java Community Process. JCACHE – Java Temporary Caching API. Java Specification Request 107, March 2001. 15. Java Community Process. JavaT M Data Objects Specification 1.0. Java Data Objects Expert Group, Proposed Final Draft JSR 000012, May 2001. 16. David Lomet. MLR: A Recovery Method for Multi-level Systems. In Proceedings of the 1992 ACM SIGMOD Conference on Management of Data, San Diego CA, USA, June 1992. 17. David Lomet and Mark R. Tuttle. Redo Recovery after System Crashes. In Proceedings of the 21th International Conference on Very Large Data Bases, Zurich, Switzerland, 1995. 18. David Lomet and Mark R. Tuttle. Logical Logging to Extend Recovery to New Domains. In Proceedings of the 1999 ACM SIGMOD Conference on Management of Data, Philadelphia PA, USA, 1999. 19. C. Mohan, et al. ARIES: A Transaction Recovery Method Supporting FineGranularity Locking and Partial Rollbacks Using Write-Ahead Logging. ACM Transactions on Database Systems 17(1):94-162, March 1992. 20. Object Management Group. Persistent State Service Specification. http://www.omg.org/cgi-bin/doc?ptc/00-12-07, January 2000. 21. Object Management Group. CORBAServices. http://www.omg.org/technology/documents/formal/corbaservices.htm 22. PING Team. Platform for Interactive Networked Games. http://www.cordis.lu/ist/projects/99-11488.htm 23. Poet Software. FastObjectsT M j2, White paper. http://www.fastobjects.com/pdf/J2 WhitePaper 050901.pdf, 2001. 24. Peter M. Schwarz, et al. Extensibility in the Starburst Database System. In Proceedings International Workshop on Object-Oriented Database Systems, Asilomar Conference Center CA, USA, September 1986. 25. Hans-J¨ org Schek, et al. The DASDBS Project: Objectives, Experiences, and Future Prospects. IEEE Transactions on Knowledge and Data Engineering 2(1) 25-43, March 1990. 26. Avi Silberschatz and Stanley B. Zdonik. Database Systems – Breaking out the Box. SIGMOD Record, 26(3), September 1997. 27. Michael Stonebraker and Lawrence A. Rowe. The Design of Postgres. InProceedings of the 1986 ACM SIGMOD International Conference on Management of Data, Washington, D.C., May 28-30, 1986. 28. Gerhard Weikum, et al. Multi-Level Recovery. In Proceedings of the Ninth ACM International Symposium on Principles of Database Systems, Nashville Tennessee, USA, April 1990. 29. Gerhard Weikum and Christof Hasse. Multi-Level Transaction Management for Complex Objects: Implementation, Performance, Parallelism. VLDB Journal, 2(4), October 1993. 30. Gerhard Weikum and Gottfried Vossen. Transactional Information Systems: Theory, Algorithms, and the Practice of Concurrency Control and Recovery. Morgan Kaufmann Publishers, May 2001.