Incremental Maintenance of Hypertext Views Giuseppe Sindoni Universita di Roma Tre
[email protected] ttp://poincare.dia.uniroma3.it:8080/sindoni
h
Extended Abstract
Abstract A materialized hypertext view is a hypertext containing data coming from a database and whose pages are stored in HTML les. This paper deals with the maintenance issues required by these derived hypertext to enforce consistency between page content and database state. Here, hypertext views are de ned as nested oid-based views over the set of base relations. An algebra is proposed, which is based on a speci c data model and which allows to de ne views and view updates. A language has been de ned, whose instructions allow to update the hypertext to re ect the current database state. Incremental maintenance is performed by a simple algorithm that takes as input a set of changes on the database and automatically produces the update instructions. The motivation of this study is the development of the Araneus WebBase Management System, a system that provides both database and Web site management.
1 Introduction The great success of the Web among user communities is imposing the hypertext paradigm as a privileged means of data communication and interchange in distributed and heterogeneous environments. An ever-increasing number of Web sites are becoming huge repositories of data, often coming from existing databases. There is hence an emerging need to investigate the relationships between structured data, such as those usually managed by DBMSs, and semistructured data, such as those that can be found into a hypertext.
The most common way of automatically generating a hypertext for data presentation purposes has been so far based on the dynamical construction of virtual hypertext pages after a client request. These pages are often called pull pages, because it is up to the client browser pulling out the interesting information. Lately a new approach is becoming more and more important among the Web operators, which is based on the concept of materialized hypertext view : a derived hypertext whose pages are actually stored by the system on a server or directly on the client machine. This approach overcomes the two main disadvantages of the pull one: it avoids DBMS overloading, by preventing that a page is generated more than once on the same database state, and it allows the remote mirroring of the hypertext, enforcing platformindependence. The pages of a materialized hypertext are often called push pages, to remember the fact that it is the information delivering system that generates pages and stores them. The so-called "Web channels" (see for example [1]) are heavily based on this push technology and they are assuming an ever increasing importance among Web users. Unfortunately, some additional maintenance issues are required by the materialized hypertexts, to enforce consistency between page contents and the actual database state. In fact, every time a transaction is issued on the database, its changes have to be eciently and ecaciously propagated to derived hypertexts. The problem of dynamically maintaining con-
sistency between base data and derived hypertexts is the hypertext view maintenance problem 1 . The are a number of related issues: di erent maintenance policies should be allowed (immediate, on-demand, batch); this implies the design of auxiliary data structures to keep track of database updates; the management of these structures overloads the DBMS and they have to be as light as possible; nally, due to the particular network structure of a hypertext, not only has the consistency between the single pages and the database to be maintained, but also the consistency between page links. This paper deals with such issues, introducing an algebra for the de nition of a speci c class of derived hypertexts; an auxiliary data structure for (i) representing the dependencies between the database tables and hypertext and (ii) logging database updates; and nally, an algorithm for the automatic hypertext incremental maintenance. Hypertext views are here de ned as nested oidbased views over the set of base relations. The reference model is the Araneus Data Model (adm [2]), which, essentially, is a subset of the ODMG standard object model and which allows to describe structured hypertexts as usually are those derived from a database. A weaving algebra (walg), which is essentially a nested relational algebra extended with an OID-generation operator, is proposed as the abstract language to describe hypertext generation and update2 . walg is capable to describe nesting and OID generation operations, allowing to extend, to the maintenance of object views, previous results in the maintenance of views in nested data models [5]. An auxiliary data structure allows to maintain information about the dependencies between database tables and hypertext pages. It is based on the concept of view dependency graph, presented in [5], which is extended to the maintenance of the class of hypertexts described by 1 the
same problem has been addressed in the framework of the STRUDEL project [4] as the problem of incremental view updates for semistructured data 2 The weaving algebra is an abstraction of an implemented language, called Penelope [3], which allows the (re)generation and removal of hypertext pages.
adm.
Finally, incremental page maintenance can be performed by an algorithm that, when hypertext maintenance is required, takes as input a set of changes on the database and produces a minimal set of update instructions for the hypertext pages.
2 De ning Hypertext Views The basic features of walg are described by the following example concerning the scheme of a simple University department database, from which the department Web site is derived. Person(Name, Position, E-mail); PersonInGroup(Name, GName); ResearchGroup(GName, Topic); Course(CName, Description); CourseInstructor(CName, Iname); Lessons(CName, IName, LDate);
The reference data model for hypertext views is the Araneus Data Model (adm), whose features will be described in the following, with the help of the scheme of gure 1, which gives a graphical intuition of how the model can be used to describe the logical structure of a hypertext. adm is a page-oriented model, because page is the main concept. Each hypertext page is seen as an object having an identi er (its url) and a number of attributes. The structure of a page is abstracted by its Page Scheme and each page is an instance of a Page Scheme. In gure 1 home pages of persons are described by the PersonPage Page Scheme. Each PersonPage instance has three simple attributes (Name, Position and e-mail). Pages can also have complex attributes: lists, possibly nested at an arbitrary level, and links to other pages. The example shows the CoursePage Page Scheme, having a list attribute (InstructorList). Its elements are couples, formed by a link to an instance of PersonPage and the corresponding anchor (Name), and a nested list (LessonList) of lesson dates.
generate url, urlI K , is an operator that takes as input a relation R(K; X ). It adds a new attribute, I , to R; the values of I are identi ers. They can be page urls or links to other pages. Each value is the result of the application of a Skolem function to the values of the attributes in K.
For example, the walg statement corresponding to the CoursePage view is the following. CoursePage =
InstructorList (IName;ToInstr;LessonList) LessonList LDate urlurl CName;ToInstr IName CName;Description;IName;LDate
(1)
(Course 1 CourseInstructor 1 Lessons)
Figure 1: A portion of a University Department web scheme.
Here CourseInstructor.IName and Course.CName are the url attributes, i.e. the attributes whose values are the arguments of the OID generation function. The invention mechanism guarantees the integrity of the generated hypertext: the values of the ToInstr attribute and the corresponding urls are generated by the application of a Skolem function to the same base values. In the following, an approach to the maintenance of such hypertext views will be presented.
Finally, to each Page Scheme, a template is associated, which contains all graphical page presentation details. With walg it is possible to describe such a hypertext view over the department database according to its adm model. The weaving algebra is essentially a nested relational algebra extended with an url invention 3 The Page Maintenance Probprimitive. The operators of the algebra work on lem page-relations and return page-relations, as follows: It has so far been illustrated as the walg algebra can be used for describing derived hyper selection, , projection, and join, 1 are texts. In order to perform their creation and extended to the nested relational model, for maintenance, a simple language has been de ned, which is composed by two instructions: example in the way illustrated in [6]; GENERATE and REMOVE. They allow to manipulate nest [6], A B , is an operator that groups hypertext pages and they can refer to the whole together tuples which agree on all the at- hypertext, to all instances of a page scheme or tributes that are not in a given set of at- to pages that satisfy a condition. tributes, say B. It forms a single tuple, for The GENERATE statement has the following every such group, which has a new attribute, general syntax: say A, in place of B, whose value is the set of GENERATE | all the B values of the tuples being grouped [USING < TableList >]. together;
The semantics of the GENERATE statement is deletions on base tables not always correspond to removing pages. essentially to create the proper set of pages, taking data from the base tables as speci ed by the Page Scheme de nitions. It is possible to gener- The second point will be clari ed by the folate (i) the whole hypertetxt, (ii) all the instances lowing example. of a page scheme and (iii) only pages that are Given the relation instances, interested by base data changes. The optional Course USING clause in fact, operates on a list composed CName Description by base tables and auxiliary tables containing Databases The. . . tuples that have been inserted into and deleted Theoretical CS The. . . from the base relations, as it will be clari ed in CourseInstructor the following. CName IName The REMOVE statement has the following synDatabases Atkins Databases Mash tax: Theoretical CS Batheson Theoretical CS Cabrinsky
REMOVE |
[WHERE < urlCond >].
It allows to remove the speci ed sets of pages from the hypertext. < urlCond > is a condition specifying the urls of the single pages to be removed. The running example allows to introduce the basic concepts underlying the incremental maintenance of the hypertext views described by the walg algebra. Let us have an insertion into the CourseInstructor base relation, because a new instructor has joined a course sta : it will a ect the CoursePage view, because the view de nition involves the CourseInstructor.IName attribute. The simple "brute force" approach to the problem would simply re-generate the whole hypertext on the new state of the database, clearly performing a number of unnecessary page creation operations. A more sophisticated approach could re-generate only the instances of the CoursePage scheme, again unnecessarily creating all the pages relative to courses di erent from the new instructor's one. The optimal solution is hence to re-generate only the page instances corresponding to the course whose sta the new instructor has joined. It is necessary to put into evidence that:
Lessons
CName
IName
Databases Atkins Databases Mash Databases Mash Theoretical CS Batheson
LDate
1-4-1997 1-5-1997 1-6-1997 1-3-1997
the nested table corresponding to the Coursewalg expression is the following:
Page
CoursePage url
Name
Des.
http://...
Datab.
The...
http://...
Th. CS
The...
Name Atk. Mash Bath.
InstructorList ToInstr LessonL. Date http://... http://... http://...
1-4-1997 1-5-1997 1-6-1997 1-3-1997
where each nested tuple corresponds to a CoursePage instance. Now, suppose we delete all lessons by instructor Mash from the Lessons table. The resulting nested relation still contains a tuple for the Databases course. Hence, the corresponding page should not be removed from the hypertext, but simply re-generated, from the new state of the base tables. On the contrary, suppose we remove the whole Theoretical Computer Science course from the database, hence deleting the following tuples from the database: Course(Theoretical CS, The. . . ); CourseInstructor(Theoretical CS, there are no duplicates in a view (set of Batheson); CourseInstructor(Theoretical CS, pages), because urls are created from key Cabrinsky); Lessons(Theoretical CS, Batheson, values; 1-3-1997). The resulting nested relation will
not contain any tuple for the Theoretical Computer Science course, hence, the corresponding page should be actually removed from the hypertext. The main maintenance problem here, is to nd the urls of pages to be removed: the system needs to maintain information about which base tables are involved in url generation. In the following, an algorithm for incremental page maintenance will be presented. In particular it will be shown how to distinguish pages to be removed from pages to be re-generated in case of deletions.
4 An Algorithm for Incremental Page Maintenance
Figure 2: A portion of the Dependency Graph for the University Department web scheme.
The incremental maintenance of hypertext pages, in this framework, is focused on two main site of gure 1 is depicted. It allows to represent the dependency of a page scheme from a set issues: of base tables and, for each table-view pair, the designing an algorithm that is based on the possible attributes involved in OID generation whole nested tuple (re)generation, as the are represented in the edge label. Moreover, for page generation is taken as the main cost each base relation, the current are explicitly unit, due to the particular semistructured stored. Each view node has pointers to the entries from which the next maintenance operation nature of hypertext views; has to start. creating a core data structure to maintain In gure 2 for example, the CoursePage view essential information about nested object is connected with its base relations (Course, views, which can be easily extended to meet CourseInstructor and Lessons) and the confuture requirements. nection with the Course relation is labeled with the name of the url attribute (CName). The view In particular, log information about database has also pointers to the of each relation (for modi cations has to be maintained, to allow dif- simplicity, not all the are represented). In parferent maintenance policies. To this purpose, ticular, each pointer refers to the rst entry the proposed auxiliary data structure is designed that has to be used for the next view mainteto e ectively manage the sets of inserted and nance. The portion of a starting from the deleted tuples of each base relation since the last tuple pointed by a view will be addressed as hypertext maintenance. These sets will be ad- the Waiting Portion of the for the view. Afdressed as + and ? and their nature and use ter each maintenance operation, the pointers are moved to the end of the . An overall update will be clari ed in the following. operation is needed of course, to delete from The auxiliary data structure is a graph that allows to maintain information about dependen- all the tuples that are no more interested in view cies between base relations and derived views. maintenance operations. It is conceptually derived from the view depen- The structure is light enough to be maintained dency graph de ned in [5], which is extended to without excessive overhead for the system bemaintain information about url attributes. In cause entries are independent of the number gure 2, the Dependency Graph for the example of views. The pointer mechanism allows in fact
to "share" each among all the views that derive from the corresponding relation. Besides, it is possible to adopt a di erent maintenance approach for each view. One can decide for example to re-generate the CoursePage instances after each issued transaction on the base tables, to keep students early aware of lessons dates and changes; while adopting a batch approach, for example doing maintenance at the beginning of each course session, for the unique instance of the EducationPage. In the previous section, one of the main problems in maintaining the nested views in case of deletions has been pointed out: some deletions correspond to nested tuple (i.e. pages) updates, while some others correspond to actual nested tuple deletions. The capability of managing the of base relations allows also an ecient solution to this problem. Take again for example the deletion of the Theoretical Computer Science course from the database. Let CoursePageRem be the set of urls corresponding to pages that must be removed from the hypertext, that is:
CoursePageRem = urlurl CName ( CName (? Course))
(2) It essentially represents all the urls corresponding to tuples deleted from the Course base relation, which, in this case, is the only one involved in page url generation. In order to perform removal of pages, the following REMOVE statement is produced, which essentially deletes all the hypertext pages corresponding to the extracted urls. REMOVE CoursePage WHERE url IN
CoursePageRem.
After page deletions, the pointer from the CoursePage view to the ? of the Course relation has to be moved to the end of the ? itself. The of base relations that are not involved in url creation contribute to determine the set of pages to be generated again. Two separate sets of statements are produced for maintaining (i)
pages where values have to be inserted and (ii) pages from which values have to be deleted. For example, the generated statements to maintain the CoursePage view with respect to inserted tuples are the following. GENERATE CoursePage USING + Course, CourseInstructor, Lessons; GENERATE CoursePage USING Course, + CourseInstructor, Lessons; GENERATE CoursePage USING Course, CourseInstructor, + Lessons;
They generate the CoursePage instances, using the Waiting Portion of each + of the base relations, in such a way to generate only the pages corresponding to inserted tuples. The case for deletions is slightly more subtle. As it has been illustrated, deletions on url base tables correspond to physical deletions of hypertext les. Deletions on base tables whose attributes are not involved in url generation instead, must correspond to the re-generation of the pages that contain the deleted values, which have to disappear from the pages. The system here needs also the pre-change state of the base tables, in order to nd which pages (nested tuples) should be refreshed. GENERATE CoursePage USING Course, ? CourseInstructor, OldLessons, Lessons; GENERATE CoursePage USING Course, OldCourseInstructor, CourseInstructor, ? Lessons;
They generate the CoursePage instances, using the Waiting Portion of each ? of the base relations, in such a way to re-generate only the pages where the deleted values previously appeared, but without those values. Note that here the GENERATE statement for the ? of the Course relation is not needed, as this relation is the url relation: tuples deleted from it correspond to page removals as previously illustrated. For the same reason, the updated version of the Course relation is used. It is also worth noting that the same page instance could be generated by more than one statement. To overcome this redundancy one possible solution could be to explicitly create a log of the generated pages and checking if a page has already been refreshed during the same maintenance operation, in a manner similar to the one illustrated by Ceri and Widom in [7]. Another solution could be to use a technique similar to that presented by Kawaguchi et alt. [5], provided that it is suitably extended for walg views. The maintenance algorithm is based on the previously illustrated principles. It takes as input the Dependency Graph, and the database. It produces the maintenance statements for a page scheme instances and its basic steps are the following: 1. build the set of urls of view pages to be removed (RemPages); 2. produce
PageScheme
DELETE h i WHERE url IN RemPages;
3. move to the bottom the pointers from the view node to the ? of the url relations; 4. produce
PageSchemei R1; R2; : : :; Rn;
GENERATE h USING +
...
PageSchemei R ; : : :; Rn?1; +Rn;
GENERATE h USING 1
5. produce
GENERATE
hPageSchemei
USING
R1; : : :; Rm; ?Rm+1 ; OldRm+2; : : :; OldRn; .. . GENERATE hPageSchemei
USING
R1; : : :; Rm; OldRm+1; : : :; OldRn?1; ?Rn;
6. move to the bottom the pointers from the view node to ; 7. delete from all the tuples that are in no more Waiting Portion. Where R1 ; : : :; Rn are the view base relations (m is the number of url relations).
References [1] PointCast Home Page.
. [2] P. Atzeni, G. Mecca and P. Merialdo. To Weave the Web. In International Conf. on Very Large Data Bases (VLDB'97), Athens, pages 206{215, 1997. [3] G. Mecca, P. Atzeni, A. Masci, P. Merialdo, and G. Sindoni. The Araneus Web-Base Management System. To be published in ACM SIGhttp://www.pointcast.com
MOD International Conf. on Management of Data (SIGMOD'98), Seattle, Washington, 1998.
Exhibits Program. [4] M. Fernandez, D. Florescu, J. Kang, A. Levy, and D. Suciu. STRUDEL { a Web site management system. In ACM SIGMOD International Conf. on Management of Data (SIGMOD'97), Tucson, Arizona, 1997. Exhibits Program.
[5] A. Kawaguchi, D. F. Lieuwen, I. S. Mumick, K. A. Ross. Implementing Incremental View Maintainance in Nested Data Models. In Workshop on Data Base Programming Languages (DBPL'97), Boulder, USA, 1997.
[6] L. Colby. A Recursive Algebra and Query Optimization for Nested Relations. In ACM SIG-
MOD International Conf. on Management of Data (SIGMOD'89), Portland, USA, pages 273{
283, 1989. [7] S. Ceri, J. Widom. Deriving Production Rules for Incremental View Maintenance. In International Conf. on Very Large Data Bases (VLDB'91), Barcelona, pages 577{589, 1991.
