D Database Integrity Checking

961

Category: Data Mining & Databases

Database Integrity Checking Hendrik Decker Universidad Politécnica de Valencia, Spain Davide Martinenghi Free University of Bozen/Bolzano, Italy

IntroductIon Integrity constraints (or simply “constraints”) are formal representations of invariant conditions for the semantic correctness of database records. Constraints can be expressed in declarative languages such as datalog, predicate logic, or SQL. This article highlights the historical background of integrity constraints and the essential features of their simplified incremental evaluation. It concludes with an outlook on future trends.

Background Integrity has always been an important issue for database design and control, as attested by many early publications (e.g., Bernstein & Blaustein, 1982; Bernstein, Blaustein, & Clarke, 1980; Codd, 1970, 1979; Eswaran & Chamberlin, 1975; Fraser, 1969; Hammer & McLeod, 1975; Hammer & Sarin, 1978; Nicolas, 1978, 1982; Wilkes, 1972); later ones are too numerous to mention. Expressing database semantics as invariant properties persisting across updates had first been proposed by Minsky (1974). Florentin (1974) suggested expressing integrity constraints as predicate logic statements. Stonebraker (1975) proposed formulating and checking integrity constraints declaratively as SQL-like queries. Functional dependencies (Armstrong, 1974; Codd, 1970) are a fundamental kind of constraints to guide database design. Referential integrity has been part of the 1989 SQLANSI and ISO standards (McJones, 1997). The SQL2 standard (1992) introduced the CHECK and ASSERTION constructs (i.e., table-bound and table-independent SQL query conditions) as the most general means to express integrity constraints declaratively (Date & Darwen, 1997). Since the 1990s, uniqueness constraints, foreign keys, and complex queries involving EXISTS and NOT became common features in commercial databases. Thus, arbitrarily general and complex integrity constraints can now be expressed and evaluated in most relational databases. However, most of them offer efficient support only for the following three simple kinds of declarative constraints:

•

•

•

Domain Constraints: Restrictions on the permissible range of attribute values of tuples in table columns, including scalar SQL data types and subsets thereof, as well as options for default and null values. Uniqueness Constraints: As enforced by the UNIQUE construct on single columns, and UNIQUE INDEX and PRIMARY KEY on any combination of one or several columns in a table, preventing multiple occurrences of values or combinations thereof. Foreign Key Constraints: For establishing a relationship between the tuples of two tables, requiring identical column values. For instance, a foreign key on column emp of relation works_in requires that the emp value of each tuple of works_in must occur in the emp_id column of table employee, and that the referenced column (emp_id in the example) has been declared as primary key.

For more general constraints, SQL manuals usually recommend using procedural triggers or stored procedures instead of declarative constructs. This is because such constraints may involve nested quantifications over huge extents of several tables. Thus, their evaluation can easily become prohibitively costly. However, declarativity does not need to be sacrificed for efficiency, as shown by many methods of simplified integrity checking as cited in this survey. They are all based on the seminal paper (Nicolas, 1982).

sImplIfIed Incremental IntegrIty cHeckIng A common idea of all integrity checking methods is that not all constraints need to be evaluated, but at most those that are possibly affected by the incremental change caused by database updates or transactions. Anticipating updates by patterns, most incremental integrity checking methods allow for simplifications of constraints to be generated already at schema compilation time. Such compiled simplifications are parametric conditions to be instantiated, possibly further optimized, and evaluated upon given update requests. For generating them, only the database schema, the integrity constraints, and the update patterns are needed as input.

Copyright © 2009, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.

D

Database Integrity Checking

Their evaluation, however, may involve access to the stored data at update time. Methods that generate compiled simplifications are described, for example, by Christiansen and Martinenghi (2006), Decker (1987), and Leuschel and De Schreye (1998). For unanticipated ad-hoc updates, the generation of simplifications takes place at update time. Optimizations for efficient evaluation of simplified constraints are addressed, for example, by Sheu & Lee (1987). Simplifications can be distinguished by the database state in which they are evaluated. Post-test methods must evaluate their simplifications in the new, updated state, for example, Decker and Celma (1994), Grant and Minker (1990), Lloyd, Sonenberg, and Topor (1987), Nicolas (1982), and Sadri and Kowalski (1988). Pre-test approaches, for example, Bry, Decker, and Manthey (1988), Christiansen and Martinenghi (2006), Hsu and Imielinski (1985), McCune and Henschen (1989), and Qian (1988), only access the old state before the update, that is, they need not execute the update prematurely, since undoing an updated state if integrity is violated is costly. In case of integrity violation, the eagerness of pre-tests to avoid rollbacks is a clear performance advantage over post-tests. For convenience, a finite set of constraints imposed on a database D is called an integrity theory of D. For a database D and an integrity theory IC, let D(IC) = satisfied denote that IC is satisfied in D, and D(IC) = violated that it is violated. Further, for an update U, let DU denote the updated database. Any simplification method M can be formalized as a function that takes as input a database, an integrity theory and an update, and outputs either satisfied or violated. Thus, soundness and completeness of M can be stated as follows: Let D be any database, IC any integrity theory, and U any update. Suppose that D(IC) = satisfied. Then, an integrity checking method M is sound if the following holds: If M(D, IC, U) = satisfied then DU(IC) = satisfied. It is complete if the following holds: If DU(IC) = satisfied then M(D, IC, U) = satisfied. This formalism is applicable to most integrity checking methods in the literature. Many of them are sound and complete for significant classes of relational and deductive databases, integrity theories, and updates. Some methods, however, are only shown to be sound, that is, they provide sufficient conditions that guarantee integrity satisfaction of DU, for example, Gupta, Sagiv, Ullman, and Widom (1994); further checking is required if these conditions are not satisfied. The main advantage is that the evaluation of M(D, IC, U) is typically much simpler than that of DU(IC).

962

Most integrity checking methods can be described by distinguishing three (possibly interleaved) phases, namely the generation, optimization, and evaluation of simplified tests. Next, these phases, numbered I, II, III, are illustrated by an example.

eXample Consider a relational database with tables for workers and managers, defined as follows: CREATE TABLE(worker(CHAR name, CHAR department)) CREATE TABLE(manager (CHAR name)).

Suppose there is an integrity constraint requiring that no worker is a manager, expressed as a denial by the following SQL condition, which forms the body of a related SQL assertion: NOT EXISTS (SELECT * FROM worker, manager WHERE worker.name = manager.name).

If the number of workers and managers is large, then checking whether this constraint is violated or not can be very costly. The number of facts to be retrieved and tested is in the order of the cardinality of the cross product of worker and manager, whenever the constraint is checked. Fortunately, however, the frequency and amount of accessing stored facts can be significantly reduced when going through phases I -III. Beforehand, a possible objection at this stage should be dealt with. SQL programmers might feel compelled to point out that the previous constraint is probably much easier checked by some trigger such as the following one in MS SQL Server syntax: CREATE TRIGGER no_worker_manager ON worker FOR INSERT : IF EXISTS (SELECT * FROM inserted, manager WHERE inserted.name = manager.name) ROLLBACK.

Its evaluation would only need to access manager and a cached relation inserted containing the row to be inserted to worker, but not the stored part of worker. However, it is easily overlooked that the sample integrity constraint entails that somebody who is promoted to a manager must not be a worker, thus necessitating a second trigger for insertions into manager. In general, each occurrence of each atom occurring in a constraint requires a separate trigger, and it is by far not always as obvious as in the simple previous example how they should look. Apart from being error-prone, hand-


coded triggers may also bring about unpredictable effects of mutual interactions that are hard to control. As opposed to that, the generic approach illustrated next can be fully automatized. Now, let INSERT INTO worker VALUES (‘Fred’, ‘sales’) be an update. Then, phases I-III involve the following.

I. generation The constraint that no worker must be manager is clearly relevant for the given update and hence must be checked, but only for the newly inserted worker Fred. Any integrity constraint that is not relevant for insertions into the worker table need not be checked. For example, a constraint requiring that each department must have some least number of workers need not be checked for insertions but only for deletions in the worker table. Also, all constraints that do not involve the relation worker are not relevant. Hence, as a result of phase I, the SQL condition NOT EXISTS (SELECT * FROM worker, manager WHERE worker.name = manager.name)

can be simplified to the following much less expensive expression: NOT EXISTS (SELECT * FROM worker, manager WHERE worker.name = ‘Fred’ AND worker.name = manager. name).

If the worker table is involved in definitions of database views, then there may be implicit update consequences on such views. These, in turn, need to be run through I-III.

II. optimization Since the existence of a worker satisfying the subcondition worker.name = ‘Fred’ is assured by the update, the simplified condition in I can be further optimized to NOT EXISTS (SELECT * FROM manager WHERE name = ‘Fred’).

III. evaluation Evaluation of the query whether Fred is a manager means to look up a single fact in a stored relation. That, of course, is much less costly than evaluating all integrity constraints in their full generality, as would be necessary without having done I-III. The previous example is very simple. However, by running phases I-III, the same proportions of simplifica-

tion are obtained systematically for arbitrarily complex constraints.

future trends Future trends to be reckoned with for integrity checking can be grouped by the following thematic issues: growing demands, business rules, closing gaps between theory and practice, distributed and replicated databases, agents, extensions of the relational model, semi-structured data, XML, Web databases, and inconsistency tolerance. Editions in 1999 and 2003 of the SQL standard have tried to do justice to several of these trends by proposing norms for non-scalar data types, markup annotations, recursive view definitions, and triggers, which, however, have hardly been taken up uniformly by database vendors (cf. Gorman, 2001). The importance of database integrity is likely to grow further, hand in hand with increasing concerns for the quality and reliability of data. Also, the declarativity of data integrity is bound to become more important because of the growing complexity of data and applications, the development and maintenance of which would otherwise become too troublesome. A success story about commercial use cases of declarative integrity constraints is the growing demand of business rules (Date, 2000; Ross, 2005). Another prospering application area, based on the use of integrity constraints in semantic query optimization (Grant & Minker, 1990), is consistent query answering, as indicated by Bertossi (2006). Yet, commercial databases lag behind the state of the art of integrity checking in academia. Besides a lack of support for the simplification of ASSERTION statements, also the support for declarative constraints on views is scant, if not completely missing, in many database products. In general, technologies related to integrity constraints that are still to be transferred from theory to practice are aplenty (cf., e.g., Decker, 1998, 2002). Not only in practice, but also in theory, a void that continues to gape is related to simplified integrity checking for concurrent transactions. Locking policies and coordination algorithms, used to avoid or resolve conflicts and deadlocks of concurrent read-write accesses, may cause a considerable overhead. Performance is burdened even more by additional locks that may be necessary for integrity checking. A locking policy for checking the integrity of concurrent transactions and possible mitigations of the associated overhead are discussed by Martinenghi and Christiansen (2005). Not only with regard to concurrency, but also in general, integrity support in decentralized networks of databases is even worse than for centralized systems. For instance, no declarative way of ensuring referential integrity across tables stored in different sites exists. This and other problems of integrity checking in distributed databases are addressed,

963

D


(e.g., Bright & Chandy, 2003; Ibrahim, 2002, 2006). Some relief that may make up for part of this deficit can be expected from a convergence of distributed databases and agents technology. An approach to handle integrity constraints in this context is described, for example, by Sadri, Toni, and Torroni (2002). Both theory and practice of integrity support need to advance also for replicated databases. Beyond distributed databases, there is the problem of the integrity of data on the Web, which certainly is going to receive more attention in the future; for a primer, see work by Aldana, Yagüe, and Gómez (2002). In recent years, there has been a surge of interest in the design and control capacities of integrity constraints in XML-based data models and languages (Vianu, 2003). An initial focus on primary and foreign keys (Fan & Libkin, 2002) currently is being extended to more general kinds of constraints for XML (cf., e.g., Fan, 2005; Zhuge & Xing, 2005). One of the reasons for the lack of support for database integrity in practice is an alleged inapplicability of constraint checking methods in the presence of inconsistencies. Simplification methods usually assume that, for checking integrity incrementally, the state before the update must satisfy integrity. Clearly, this assumption is not very realistic since hardly any real-life database of significant size can be expected to be completely free of any inconsistency. Fortunately, however, this assumption can be abandoned. As shown by Decker and Martinenghi (2006), many (though not all) methods provide simplified conditions that are necessary and sufficient for checking that all cases of integrity constraints that have been satisfied before the update will remain so afterward, even in presence of any number of violated cases. Ramifications of this result are expected to further boost the adoption of integrity checking technology in practice.

conclusIon We have outlined basic concepts and principles for checking the semantic integrity of stored data that are more advanced than the usual SQL support for data integrity, but can be expressed in conventional SQL and processed by standard SQL engines. In the literature, integrity constraints are often represented in the language of first-order predicate logic. A systematic translation of constraints expressed in predicate logic into SQL is described by Decker (2002). There, also a fully automated and provably correct mechanism for translating declarative constraints into efficient triggers is specified. It takes advantage of simplifications as outlined previously. More details about general principles of integrity checking are documented by Martinenghi, Christiansen, and Decker (2006).

964

references Aldana, J., Yagüe, I., & Gómez, L. (2002). Integrity issues in the Web: Beyond distributed databases. In J. Doorn & L. Rivero (Eds.), Database integrity: Challenges and solutions (pp. 293-321). Hershey, PA: Idea Group Publishing. Armstrong, W. (1974). Dependency structures of database relationships. In J. L. Rosenfeld (Ed.), Proceedings of IFIP ’74 (pp. 580-583). Stockholm, Sweden: North-Holland. Bernstein, P., & Blaustein, B. (1982). Fast methods for testing quantified relational calculus assertions. In M. Schkolnick (Ed.), Proceedings of SIGMOD’82 (pp. 39-50). Orlando, FL: ACM Press. Bernstein, P., Blaustein, B., & Clarke, E. (1980). Fast maintenance of semantic integrity assertions using redundant aggregate data. In W. W. Armstrong et al. (Eds.), Proceedings of the Sixth VLDB (pp. 126-136). Montreal: IEEE-CS. Bertossi, L. (2006). Consistent query answering in databases. ACM SIGMOD Record, 35(2), 68-77. Bright, J., & Chandy, J. (2003). Data integrity in a distributed storage system. In J. R. Arabnia & Y. Mun (Eds.), Proceedings of PDPTA’03 (pp. 688-694). Las Vegas: CSREA Press. Bry, F., Decker, H., & Manthey, R. (1988). A uniform approach to constraint satisfaction and constraint satisfiability in deductive databases. In J. W. Schmidt, S. Ceri, & M. Missikoff (Eds.), Proceedings of the First EDBT (LNCS 303, pp. 488-505). Venice: Springer. Christiansen, H., & Martinenghi, D. (2006). On simplification of database integrity constraints. Fundamenta Informaticae, 71(4), 371-417. Codd, E. (1970). A relational model of data for large shared data banks. Commun. ACM, 13(6), 377-387. Codd, E. (1979). Extending the database relational model to capture more meaning. ACM Transactions on Database Systems, 4(4), 397-434. Date, C. (2000). What, not how: The business rules approach to application development. Boston: Addison-Wesley. Date, C., & Darwen, H. (1997). A guide to the SQL standard. Boston: Addison-Wesley. Decker, H. (1987). Integrity enforcement on deductive databases. In L. Kerschberg (Ed.), Experts database systems (pp. 381-395). Charleston, SC: Benjamin Cummings. Decker, H. (1998). Some notes on knowledge assimilation in deductive databases. In B. Freitag, H. Decker, M. Kifer, & A. Voronkov (Eds.), Transactions and change in logic


databases (LNCS 1472, pp. 249-286). Schloss Dagstuhl, Germany: Springer.

ings of the First VLDB (pp. 25-47). Framingham, MA: ACM Press.

Decker, H. (2002). Translating advanced integrity checking technology to SQL. In J. Doorn & L. Rivero (Eds.), Database integrity: Challenges and solutions (pp. 203-249). Hershey, PA: Idea Group Publishing.

Hammer, M., & Sarin, S. (1978). Efficient monitoring of database assertions (abstract). In E. I. Lowenthal & N. B. Dale (Eds.), Proceedings of SIGMOD’78 (p. 159). Austin, TX: ACM Press.

Decker, H., & Celma, M. (1994). A slick procedure for integrity checking in deductive databases. In P. Van Hentenryck (Ed.), Proceedings of the 11th ICLP (pp. 56-469). Boston: MIT Press.

Hsu, A., & Imielinski, T. (1985). Integrity checking for multiple updates. In S. B. Navathe (Ed.), Proceedings of SIGMOD’85 (pp. 152-168). Austin, TX: ACM Press.

Decker, H., & Martinenghi, D. (2006). A relaxed approach to integrity and inconsistency in databases. In M. Hermann & A. Voronkov (Eds.), Proceedings of the 13th LPAR (LNAI 4246, pp. 287-301). Berlin; Heidelberg: Springer-Verlag. Eswaran, K., & Chamberlin, D. (1975). Functional specifications of a subsystem for database integrity. In Douglass S. Kerr (Ed.), Proceedings of the First VLDB (pp. 48-68). Framingham, MA: ACM Press. Fan, W. (2005). XML constraints: Specification, analysis, and applications. In H. Christiansen & D. Martinenghi (Eds.), Proceedings of the 16th DEXA Workshop, LAAIC’05 (pp. 805809). Copenhagen, Denmark: IEEE Computer Society. Fan, W., & Libkin, L. (2002). On XML integrity constraints in the presence of DTDs. Journal of the ACM, 49(3), 368406. Florentin, J. (1974). Consistency auditing of databases. The Computer Journal, 17(1), 52-58. Fraser, A. (1969). Integrity of a mass storage filing system. The Computer Journal, 12(1), 1-5.

Ibrahim, H. (2002). A strategy for semantic integrity checking in distributed databases. In J.-P. Sheu (Ed.), Proceedings of the Ninth ICPDS (pp. 139-144). Taiwan: IEEE Computer Society. Ibrahim, H. (2006). Checking integrity constraints—How it differs in centralized, distributed and parallel databases. In H. Christiansen & D. Martinenghi (Eds.), Proceedings of the 17th DEXA Workshop, LAAIC’06 (pp. 563-568). Karkow, Poland: IEEE Computer Society. Leuschel, M., & De Schreye, D. (1998). Creating specialised integrity checks through partial evaluation of meta-interpreters. Journal of Logic Programming, 36(2), 149-193. Lloyd, J., Sonenberg, L., & Topor, R. (1987). Integrity constraint checking in stratified databases. Journal of Logic Programming, 4(4), 331-343. Martinenghi, D., & Christiansen, H. (2005). Transaction management with integrity checking. In K. V. Andersen, J. K. Debenham, & R. Wagner (Eds.), Proceedings of the 16th DEXA (LNCS 3588, pp. 606-615). Copenhagen, Denmark: Springer.

Fuxman, A., & Miller, R. (2005). First-order query rewriting for inconsistent databases. In T. Eiter & L. Libkin (Eds.) Proceedings of the 10th ICDT (LNCS 3363, pp. 337-351). Edinburgh, UK: Springer.

Martinenghi, D., Christiansen, H., & Decker, H. (2007). Integrity checking and maintenance in relational and deductive databases and beyond. In Z. Ma (Ed.), Intelligent databases: Technologies and applications (pp. 238-285), Hershey, PA: Idea Group Publishing.

Gorman, M. (2001). Is SQL really a standard anymore? Whitemarsh Information Systems Corp. Retrieved November 12, 2006, from www.ncb.ernet.in/education/modules/dbms/ SQL99/issqlarealstandardanymore.pdf

McCune, W., & Henschen, L. (1989). Maintaining state constraints in relational databases: A proof theoretic basis. Journal of the ACM, 36(1), 46-68.

Grant, J., & Minker, J. (1990). Integrity constraints in knowledge based systems. In H. Adeli (Ed.), Knowledge engineering (Vol. II) (pp. 1-25). New York: McGraw-Hill.

McJones, P. (1997). The 1995 SQL reunion: People, projects, and politics. SRC Technical Note 1997 – 018. Retrieved November 12, 2006, from www.mcjones.org/System_R/ SQL_Reunion_95/sqlr95.html

Gupta, A., Sagiv, Y., Ullman, J., & Widom, J. (1994). Constraint checking with partial information. In M. Yannakakis (Ed.) Proceedings of the 13th PODS (pp. 45-55). Minneapolis: ACM Press.

Minsky, N. (1974). On interaction with databases. In R. Rustin (Ed.), Proceedings of SIGMOD’74 (Vol. 1, pp. 5162). Ann Arbor, MI: ACM Press.

Hammer, M., & McLeod, D. (1975). Semantic integrity in relational database systems. In D. S. Kerr (Ed.), Proceed-

Nicolas, J-M. (1978). First order logic formalization for functional, multivalued and mutual dependencies. In E. I.

965

D


Lowenthal & N. B. Dale (Eds.), Proceedings of SIGMOD’78 (pp. 40-46). Austin, TX: ACM Press. Nicolas, J-M. (1982). Logic for improving integrity checking in relational databases. Acta Informatica, 18, 227-253. Qian, X. (1988). An effective method for integrity constraint simplification. In Proceedings of the Ninth ICDE (pp. 338345). Los Angeles, CA: IEEE Computer Society. Ross, R. (2005). Business rule concepts (2nd ed.). Business Rules Solutions Publishing. Sadri, F., & Kowalski. R. (1988). A theorem-proving approach to database integrity. In J. Minker (Ed.), Foundations of deductive databases and logic programming (pp. 313-362). Morgan Kaufmann. Sadri, F., Toni, F., & Torroni, P. (2002). An abductive logic programming architecture for negotiating agents. In S. Flesca, S. Greco, N. Leone, G. Ianni (Eds.), Proceedings of the Eighth JELIA (LCNS 2424, pp. 419-431). Cosenza, Italy: Springer. Sheu, P., & Lee, W. (1987). Efficient processing of integrity constraints in deductive databases. Future Generations Computer Systems, 3(3), 210-216. Stonebraker, M. (1975). Implementation of integrity constraints and views by query modification. In W. Frank King (Ed.), Proceedings of SIGMOD’75 (pp. 65-78). San Jose, CA: ACM Press. Vianu, V. (2003). A Web odyssey: From Codd to XML. SIGMOD Record, 32(1), 68-77. Wilkes, M. (1972). On preserving the integrity of databases. The Computer Journal, 15(3), 191-194. Zhuge, H., & Xing, Y. (2005). Integrity theory for resource space model and its application. In W. Fan, Z. Wu, & J. Yang (Eds.), Proceedings of the Fourth WAIM (LNCS 3739, pp. 8-24). Hangzhou, China: Springer.

key terms Business Rule: Statement for defining or constraining the evolution of data pertaining to an enterprise’s business. Business rules can be represented and enforced by integrity constraints. Declarative vs. Procedural: Since the evaluation of declarative integrity constraints can be very costly, potentially troublesome procedural triggers and stored procedures are often used instead. The main thrust of this article is about reducing the cost of declarative integrity constraints.

966

Inconsistency Tolerance: A practically indispensable property. Integrity checking is inconsistency-tolerant if the invariance of all satisfied cases of integrity constraints across updates can be ensured, even if some other cases are violated. Integrity: Semantic consistency, that is, the correctness of stored data with regard to their intended meaning, as expressed by integrity constraints. Not to be confused with namesake issues related to data security, serializability of concurrent transactions or sound failure recovery. Integrity Checking: Systematic test for checking whether integrity constraints remain satisfied or become violated by some update. Integrity Constraint: Integrity constraints are invariant properties of the database that evolve via updates. Often, it is convenient to state them as denials, that is, yes/no queries that are required to return the empty answer in each database state. Integrity Enforcement: Actions taken to ensure that integrity remains satisfied across database updates. If integrity is violated by some update, then that update is rejected or some other corrective action is taken. Integrity Satisfaction, Integrity Violation: Integrity is satisfied if each integrity constraint in the database schema, queried as a denial, returns the empty answer. Integrity is violated if any one of these integrity constraints returns a non-empty answer. Then, the database is also said to be inconsistent. Semantic Integrity: The adjective “semantic” distinguishes a set of explicitly defined integrity constraints from structural constraints that are implicitly enforced by the used data model. Simplification: A methodological approach to reduce complexity and costs of integrity checking. Also the simplified forms of integrity constraints generated by such methods are called simplifications. Static vs. Dynamic Integrity Constraints: Static integrity constraints are semantic properties that are invariant across evolving database states. Dynamic integrity constraints refer explicitly to several (mostly consecutive) states or to their transitions, typically involving temporal or procedural constructs. Trigger: An SQL statement that generates an action (e.g., an update or a reset) upon some anticipated event (e.g., an update request). Triggers are a procedural means to enforce integrity.