Implementing the Division Operation on a Database Containing Uncertain Data Frank S.C. Tseng
Department of Computer Science and Information Engineering National Chiao Tung University Hsinchu, Taiwan 300, R.O.C.
Arbee L.P. Chen
Department of Computer Science National Tsing Hua University Hsinchu, Taiwan 300, R.O.C. Email:
[email protected] Fax: 886-35-723694
Wei-Pang Yang
Department of Computer Science and Information Engineering National Chiao Tung University Hsinchu, Taiwan 300, R.O.C. ABSTRACT
Uncertain data in databases were originally denoted as null values, which were later generalized to partial values. Based on the concept of partial values, we have further generalized the notion to probabilistic partial values. In this paper, an important operation | division, is fully studied to handle partial values and probabilistic partial values. Due to the uncertainty of partial values and probabilistic partial values, the corresponding extended division may produce maybe tuples and maybe tuples with degrees of uncertainty, respectively. To process this extended division, we decompose a relation consisting of partial values or probabilistic partial values into a set of relations containing only denite values. Bipartite graph matching techniques are then applied to develop ecient algorithms for the extended division that handles partial values. The renement on the maybe result is also discussed. Finally, we study the extended division that handles probabilistic partial values. keywords: partial values, probabilistic partial values, graph matching, uncertain data.
To whom all correspondence should be sent.
Contents 1 Introduction
1
2 Division for Denite Data
2
3 Generalizing Division to Handle Data with Partial Values
4
3.1 3.2 3.3 3.4 3.5 3.6 3.7
Preliminaries for Partial Values : : : : : : : : : : : The Extended Division for Handling Partial Values Preliminaries for Graph Theory : : : : : : : : : : : Algorithms for the Extended Division \ b " : : : : : An Example : : : : : : : : : : : : : : : : : : : : : : Further Renement on Maybe Result : : : : : : : : Rening Maybe Answers into True Answers : : : :
: : : : : : :
: : : : : : :
: : : : : : :
: : : : : : :
: : : : : : :
: : : : : : :
: : : : : : :
: : : : : : :
: : : : : : :
: : : : : : :
: : : : : : :
: : : : : : :
: : : : : : :
4 7 10 11 15 18 20
4 Generalizing Division to Handle Data with Probabilistic Partial Values 21
4.1 Preliminaries for Probabilistic Partial Values : : : : : : : : : : : : : : : : : 21 4.2 The Extended Division for Handling Probabilistic Partial Values : : : : : : 25
5 Discussion and Conclusion
26
1
Introduction
Management of uncertain data has been recognized as an important research area in database systems 29]. Null values were originally adopted to represent the meaning of \values unknown at present" in database systems. Codd 12] pioneers the work on extended relational algebra to manipulate null values. From then on, incomplete information in relational databases have been extensively studied 5, 18, 21, 24, 25, 26]. The update semantics with null values in relational databases have been discussed in 1, 2]. Codd 13, 14] distinguishes null values into applicable and inapplicable values. Besides, the relationship between null values and functional dependencies has been studied in 22, 23, 39, 40]. 27] gives a concise review of handling null values by algebraic approaches. The null value concept has been generalized to the concept of partial values by Grant 19]. Instead of being treated as an atomic value, an attribute value in a table is considered as a nonempty subset of the corresponding domain. A partial value in Grant 19] is represented as an interval such that exactly one of the values in the interval is the \true" value of the partial value. In DeMichiel 16], however, a partial value is considered to correspond to a nite set of possible values such that exactly one of the values in that set is the \true" value of the partial value. Motro 29] refers to this kind of partial values as disjunctive data. Therefore, an applicable null value is a partial value which corresponds to the whole domain. Lipski 24] provides a general discussion on manipulating uncertain information including partial values. Following the notion of disjunctive data, we discuss the elimination of redundant partial values in 36], aggregate evaluation in 8] and the relationships among disjunctive data in 9]10]11]. Furthermore, in 37]38]33]34], we generalize partial values to probabilistic partial values and provide probabilistic approaches to query processing in heterogeneous database systems. Among the previous research works and current commercial database systems, an important operation | the division, has not been taken into account for discussion and direct support. However, as we have pointed out in 35], this operation is needed for dealing with a query involving universal quantiers. For example, based on the Suppliersand-Parts database in 15], the query 1
\Get supplier numbers for suppliers who supply all parts." corresponds to sno pno (Shipments) pno(Parts) (refer to 15, Section 6.5.9]). Since it is not a primitive operator in the relational model, we need to implement the division by other primitive operators, which can be very cumbersome. In the following sections, we rst dene the division operation for denite data and give an algorithm to directly implement this operation is given in Section 2. Then, we dene the extended division for handling partial values and generalize the algorithm in Section 3. This generalized algorithm is developed by employing the bipartite graph matching technique 7]. In Section 4, we study the extended division for probabilistic partial values. Finally, Section 5 concludes our work.
2
Division for Denite Data
Let r(X Y ) and s(Y ) be relations. The result of r q as follows 27]:
s can be dened to be the relation
q(X ) = ft j for every tuple ts 2 s there is a tuple tr 2 r such that tr:X = t and tr:Y = tsg: This denition can also be specied as:
q(X ) = ft j (ftg s) rg: In the following, an algorithm that directly implements this denition will be presented. A denition is needed for the algorithm and is stated as follows. Definition 2.1 For a relation r(X Y ), X and Y possibly composite, the grouping of r X
by X , denoted (r), is an operation that groups r(X Y ) by X into (X Y ) such that X
(r) (X Y ) f(x (x)) j (x 2 r:X ) ^ (8t 2 r)((t:X = x) ) t:Y 2 (x))g: Notice that (X Y ) is also called a nested relation 30]. For example, consider the following relation r(X Y )
2
X a a d d d
r
Y x z w x z
X
Then, (r) is
We give the algorithm as follows.
X Y a fx, zg d fw, x, zg
Algorithm 2.1 : An Algorithm That Derives the Result of r(X Y ) Input:
relations r(X, Y) and s(Y) without redundant tuples, X and Y possibly composite.
Output:
1. 2. 3. 4. 5. 6. 7.
s(Y ).
the result of r(X Y )
s(Y ).
Ans = /* initialize Ans to be an empty set */ r (X Y ) = r(X Y ) s(Y ), where \ " denotes natural semijoin X (X Y ) = (r (X Y )) i.e. group r (X Y ) by X into (X Y ) | a nested relation. for each tuple ti 2 do f if (j ti:Y j==j s j) Ans = Ans fti:X g 0
0
0
g
2 Return(Ans) The natural semijoin in Step 2, r (X Y ) = r(X Y ) s(Y ), can be considered as a generalization of selection 4] it restricts r by the values that appear in s. That is, r = ft j t 2 r ^ t:Y 2 sg. By this, we have the following lemma. 0
0
Lemma 2.1 For all ti 2 in Algorithm 2.1, j s j max if 8
> j s j) can never occur.
j ti:Y j g that is, (j ti:Y j
Proof: Suppose there is a tuple ti 2 such that (j ti:Y j > j s j) then, by Step 3, there exists a tuple t 2 r with t:Y 62 s, a contradiction to r = ft j t 2 r ^ t:Y 2 sg. Hence, (j ti:Y j > j s j) never occurs and our lemma follows. 2 We show the correctness of Algorithm 2.1 in the following. 0
0
3
Theorem 2.1 Algorithm 2.1 is correct.
Proof: We denote the result produced by Algorithm 2.1 as A1 and the result of r(X Y ) s(Y ) as A2. By denition, A2 = ft j (ftg s) rg. We want to show that A1 = A2. If t 2 A1 then there is a tuple 2 such that
:X Step = 5 t 2 :X r :X r:X and :Y Step = 5 fy1 y2 . . . y s g Step = 2 s: 0
j j
Step3
Step2
Thus, we have ftg s = f:X g :Y r r: Therefore, t 2 A2. Hence, A1 A2. Conversely, if t 2 A2 then ftgs r. By Step 2 of Algorithm 2.1, we know ftgs r and, by Step 3, there exists a tuple 2 such that :X = t and :Y = s. Therefore, j :Y j = j s j and t will be collected into Ans and returned by Algorithm 2.1 in Step 5 and 7, respectively that is, t 2 A1. Hence A2 A1. By the above discussion, we conclude that A1 = A2. 2 Although the division operation for denite data can be implemented by some primitive operators of relational algebra. In Section 3, however, we will show the extended division operation over partial values cannot be implemented using DeMichiel's denitions of primitive operations 16]. Therefore, instead of using the primitive operators, Algorithm 2.1 will be used as a basis for generalizing division to deal with partial values. 0
0
3
Generalizing Division to Handle Data with Partial Values
3.1 Preliminaries for Partial Values Partial values model data uncertainty in databases in the sense that, for an uncertain datum, its true value can be restricted in a specic set of possible values 16] or an interval of values 19]. In our work, a partial value is represented by a set of possible values, in which exactly one of the value is true. It is formally dened as follows. Definition 3.1 A partial value, denoted = a1 a2 . . . an ], associates with n possible
values, a1 a2 . . . an, n 1, of the same domain, in which exactly one of the values in is the \true" value of .
For a partial value = a1 a2 . . . an], a function is dened on it in 16], where maps the partial value to its corresponding nite set of possible values that is, () = 4
fa1 a2 . . . ang. Recall that an applicable null value 13], @, can be considered as a partial value with (@) = D, where D is the whole domain. Besides, a relation containing partial
values is called a partial relation 16]. Otherwise, it will be called a denite relation. Except for explicitly specied, a relation is considered to be partial hereafter. In the following, we will use and () interchangeably when it does not cause confusion. For example, v 2 if v 2 (). The cardinality of a partial value is dened as j () j in 16]. When the cardinality of a partial value equals to 1, i.e., there exists only one possible value, say d, in the partial value, then the partial value d] actually corresponds to the denite value d. On the other hand, a denite value d can be represented as a partial value d]. Besides, a partial value with cardinality greater than 1 is referred to as a proper partial value in 16]. For any two proper partial values, say 1 and 2, 1 6= 2 even if (1) = (2). This is because the true value of 1 may not be the same as the true value of 2 . Definition 3.2 If the proper partial values, 1 2 . . . k k 2 are elements of a set of
partial values, , and (1) = (2) = = (k ), then we say 1 2 . . . i 1 i+1 . . . k are quasi-duplicates of i 1 i k. ;
1
2
z }| { z }| {
By Denition 3.2, if = fa b] a b]g then 1 is a quasi-duplicate of 2, and vice versa. When a set of attributes fA1 A2 Amg is considered as a composite attribute A1A2 Am, its values can be computed from the values in this set of attributes as follows. A1 A2 Am A1A2 Am ^ ^ m1 11 21 m1 11^ 21 ^ ^ ^ m2 12 22 m2 () 1222 ... ... . . . ... ... ^ ^ mn 1n 2n mn 1n^ 2n The ^ denotes the cartesian product of partial values, which is dened as follows. ^ b of the partial values a = a1 a2 . . . am] Definition 3.3 The cartesian product a and b = b1 b2 . . . bn] is the partial value a ^ b with (a ^ b) being a set of the ordered pairs (ai bj ) for every ai 2 a and bj 2 b.
We regard (1j 2j . . . mj ) and 1j ^ 2j ^ . . . ^ mj as semantically-equivalent because if the \true" value of (1j 2j mj ) is the m-tuple (a1 a2 . . . am), where ai 2 ij and is 5
^ ^ mj is also (a1 a2 . . . am), the \true" value of ij , then the \true" value of 1j ^ 2j and vice versa. Example 3.1 Consider the following data values in the set of attributes fA1 A2g
A1 A2 1 a b] 2 3] c We can regard fA1 A2g as a composite attribute A1A2 and compute its data as A1A2 (1 a) (1 b)] (2 c) (3 c)]
2
A partial relation R(A1 A2 . . . Am) therefore can be regarded as a relation R(A1A2 Am) with the composite attribute A1A2 Am. That is, a partial relation can be considered as a set of partial values. By this, we have the following denitions. Definition 3.4 An interpretation, = (a1 a2 . . . am ), of a set of partial values, =
f1 2 . . . mg, is an assignment of values from such that ai 2 i 1 i m.
By Denition 3.4, for a set of partial values = f1 2 . . . mg, (1) (2) (m ) is the set of all interpretations of . Definition 3.5 For an interpretation = (a1 a2 . . . am ) of a set of partial values
= f1 2 . . . mg, the value set of is denoted S = S1
i m fai g.
Definition 3.6 For all interpretations, j , 1
j p, p =j 1 j j 2 j j m j,
of a set of partial values = f1 2 . . . mg, the family of value sets of is denoted F ( ) = S1 j p fSj g. If = then dene F ( ) = .
Example 3.2 Consider the following relation r(X Y ).
r
X Y a y a x y] We can regard fX Y g as a composite attribute XY and compute r to be 6
r XY (a y)] (a x) (a y)] Then there are two interpretations, 1 and 2, of r: 1 = ((a y) (a x)) and 2 = ((a y) (a y)): Their corresponding value sets are
S1 = f(a y)g f(a x)g = f(a y) (a x)g and S2 = f(a y)g f(a y)g = f(a y)g: Therefore, the family of value sets of r is F (r) = fS1 g fS2 g = ff(a y) (a x)g f(a y)gg which corresponds to the following two denite relations:
S2 X Y a y
S1 X Y a y a x
3.2 The Extended Division for Handling Partial Values In 16], DeMichiel presents a set of algebraic operations over partial values. However, the extended division for partial values is not studied. For relations containing partial values, due to the uncertainty of partial values, the result of an extended division may contain maybe tuples 5, 12, 16]. For a division operation with operands A(X Y ) and B (Y ) of denite data, the following equation always holds.
A(X Y ) B (Y ) = X (A) ; X ((X (A) B ) ; A) That is, the division operation for denite data can be implemented by some primitive operators of relational algebra. However, in the following, we will show the extended division operation over partial values cannot be implemented using the above denition and DeMichiel's denitions of primitive operations. That is,
A(X Y ) b B (Y ) 6= X (A) ; X ((X (A) B ) ; A) 0
0
7
0
0
0
0
where b denotes the extended division for partial values and , ; , and are as dened in 16]. 0
0
0
Lemma 3.1 If A(X Y ) and B (Y ) contain true tuples only, then
X (A) ; X ((X (A) B ) ; A) 0
0
0
0
0
0
can never produce maybe results, where , ; , and are as dened in 16]. 0
0
0
Proof: By DeMichiel's Denition, the maybe result of X (A) is 0
ft j (9u)(u 2 A ^ (status(u) = maybe) ^ (t = u:X ) ^(6 9v)(v 2 A ^ (v:X = u:X ) ^ (status(v) = true)))g: Since, by assumption, A has no maybe tuples, the maybe result of X (A) is therefore empty. Thus, 0
X (A) ; X ((X (A) B ) ; A) = ; X (( B ) ; A) = 0
0
0
0
0
0
0
0
0
0
2
By Lemma 3.1, we know that if we perform the extended division A(X Y ) b B (Y ) by X (A) ; X ((X (A) B ) ; A), then the maybe result tuples which should have been generated by the true tuples of the original relations A(X Y ) and B (Y ) will be lost. We dene A(X Y ) b B (Y ) as a primitive operation in the following. In our work, we will assume a divisor contains no partial values, otherwise the extended division will never produce true results. The extended division for partial values is dened as follows. Let the source relations be r(X Y ) and s(Y ), where X and Y are possibly composite. We denote the true result of r b s as T R. 0
0
0
0
TR
0
\
Si
2F
0
(Si s) ft j (t 2
(r)
ui r 2
(ui:X )) ^ (ftg s) rg:
The maybe result, denoted MR, is dened as follows.
MR
Si
2F
(Si s) ; T R
(r)
where denotes the conventional division for denite relations. The following example illustrates this denition. 8
Example 3.3 Consider the relations r and s given below and suppose we want to com-
pute r(X Y )
b
s(Y ).
X
r
s
Y
Y
a x a z x b c] x z b c] z By the denition, we can easily derive the true result of r(X Y ) b s(Y ) to be fag. To compute the maybe result, we rst transform r(X Y ) into the following equivalent relation r(XY ). r XY
(a x)] (a z)] (b x) (c x)] (b z) (c z)] There are four interpretations for r:
1 = ((a x) (a z) (b x) (bz)) 2 = ((a x) (a z) (b x) (c z)) 3 = ((a x) (a z) (c x) (b z)) and 4 = ((a x) (a z) (c x) (c z)) and the corresponding value sets are shown below. S1
X Y a x a z b x b z By these value sets, we have
S2
X a a b c
S3
Y x z x z
X a a c b
S4
Y x z x z
X a a c c
S1 s = fa bg S2 s = fag S3 s = fag and S4 s = fa cg: That is, SS i
2F
(r)
(Si s) = fa b cg. Therefore, the result q of r 9
b
s is
Y x z x z
q X status a true b maybe c maybe Note that an extra status column is added for the result of an extended division to distinguish between true and maybe tuples. It does not correspond to a real attribute. 2 However, from the above denition, it is not a trivial task to determine the result of an extended division. Since the interpretations of a partial relation r can be very large (refer to Denition 3.4), we need a more ecient algorithm for the extended division. In the following, we apply bipartite graph matching techniques to develop an algorithm for solving this problem.
3.3 Preliminaries for Graph Theory In the following, some terminologies about graph theory are dened 7, 6]. A graph G is denoted G = (V E ), where V is the set of vertices and E is the set of edges in the graph. An edge (x y) is said to join the vertices x and y. If (x y) 2 E then x and y are adjacent or neighboring vertices of G. For any set S V , we dene the neighbor set of S in G, denoted N (S ), to be the set of all vertices adjacent to the vertices in S . A bipartite graph G = (V E ) is one whose vertex set V can be partitioned into two subsets X and Y , such that each edge in G joins a vertex in X and a vertex in Y . If two vertices are not joined by an edge, they are said to be independent. Similarly, two edges that do not share a common vertex are said to be independent. Given a subset S of V , the subgraph induced by S , denoted GS ], is the graph with vertex set S and edge set containing those edges of G joining two vertices of S . A clique of G is a subset S of V such that GS ] is a complete graph, that is, each pair of distinct vertices in GS ] is joined by an edge. A set of pairwise independent edges is called a matching. A matching of maximum cardinality is called a maximum matching. Besides, two more terms are dened as follows. Definition 3.7 Let S = fS1 S2 . . . Sn g be a family of sets and s = fs1 s2 . . . sm g.
The membership graph of S over s is a bipartite graph G = (V E ) = (X Y E ), where
X = s = fs1 s2 . . . sm g, 10
Y = S = fS1 S2 . . . Sng, and E = f(si Sj ) j si 2 Sj 1 i m 1 j ng: Definition 3.8 For a bipartite graph G = (X Y E ), j X
j j Y j, we say that there is a complete matching M from X to Y if there is a matching of cardinality j X j that is, each vertex in X is adjacent to a distinct vertex in Y.
3.4 Algorithms for the Extended Division \ " c
In this section, we present two C -like algorithms to derive the (true and maybe) answer of an extended division operation. In the rst place, Algorithm 2.1 will be slightly modied for obtaining the true result of an extended division then this algorithm will be generalized to derive the maybe result. Algorithm 3.1 : An Algorithm That Derives the True Result of an Extended Division.
relations r(X Y ) and s(Y ), where X and Y are possibly composite and s contains no partial values.
Input:
Output:
1. 2. 3. 4. 5. 6. 7. 8.
the true result of r(X Y )
b
s(Y ).
TrueAns = = initialize TrueAns to be an empty set = r = r with all tuples that contain proper partial values eliminated Perform r (X Y ) = r (X Y ) s(Y ), where \ " denotes semijoin X (X Y ) = (r (X Y )) for each tuple ti 2 do f if (j ti:Y j==j s j) TrueAns = TrueAns fti:X g 0
00
0
00
g
Return(TrueAns)
Except for Step 2, this algorithm is the same as Algorithm 2.1. Step 2 guarantees r to be a denite relation. The following theorem veries the correctness of Algorithm 3.1. 0
Theorem 3.1 Algorithm 3.1 correctly produces the true result of r(X Y )
11
b
s(Y ).
Proof: The proof is the same as that of Theorem 2.1, except that r and r in the proof of Theorem 2.1 are replaced by r and r , respectively. 2 Before presenting the algorithm that derives the maybe result for an extended division, we need some preliminary denitions. 0
0
00
Definition 3.9 The partial semi-join of relation r(X Y ) by relation s(Y ), written r(X Y )
/ s(Y ), is an operation that produces the relation
r (X Y ) = ft j (8u 2 r)((u:Y \ s 6= ) ) (t:X = u:X ^ t:Y = (u:Y \ s)))g: 00
The following example illustrates this denition. Example 3.4 For the relations r and s given below,
r
X Y a z a w,u] b,c] x e x,y] e y,z]
s
Y x z
the result q = r(X Y ) / s(Y ) is
q
X a b,c] e e
Y z x x] z]
Definition 3.10 For a relation r(X Y ), X and Y possibly composite, the partial groupX
ing of r by X , denoted r (r), is an operation that groups r(X Y ) by X into (X Y ) X that is r (r) = , where
(X Y ) = f(x (x)) j (x 2 Sti r (t:X )) ^ (8t 2 r)((x 2 (t:X )) ) t:Y 2 (x))g: 2 X X It is easy to verify that when r(X Y ) is a denite relation, then r (r) = (r) (refer to Denition 2.1). 2
Example 3.5 Let r(X Y ) be the relation listed below.
12
r
X Y a x b y c z a y a z a, c] w, x] a, b] x, z] X
Then the partial grouping r (r) is X a b c
Y fx, y, z, w, x], x, z]g fy, x, z]g fz, w, x]g
We now discuss how a maybe result tuple of r(X Y ) b s(Y ) can be derived. The following theorem species the necessary and sucient condition for a true or maybe answer of an extended division. X
Theorem 3.2 For a tuple (x (x)) in =r(r), where (x) = f1 2 . . . n g, there
exists a complete matching M from s = fs1 s2 . . . sm g to S = (x) = f1 2 . . . n g in the membership graph G = (s S E ) if and only if x is an answer (true or maybe) of r(X Y ) b s(Y ). Proof: (() (1) If x is a true answer of r(X Y ) b s(Y ) then fxg s r. By regarding the elements of s as partial values with cardinality one, we have s = fs1] s2] . . . sm]g. X Then, by Denition 3.10, there exists a tuple (x (x)) 2 =r (r), such that s (x) = S . Hence, there is a complete matching M = f(s1 s1]) (s2 s2]) . . . (sm sm])g from s to S in the membership graph G = (s S E ). (2) If x is a maybe answer of r(X Y ) b s(Y ) then x 2 SSi (r)(Si s). That is, there exists an interpretation with value set S, such that fxg s S 2 F (r). Therefore, there is a tuple (x (x)) 2 with (x) = f1 2 . . . m . . .g, such that si 2 i, for i = 1 2 . . . m. Hence, there is a complete matching M = f(s1 1) (s2 2) . . . (sm m)g from s to S in the membership graph G = (s S E ). 2F
13
()) If there exists a complete matching M = f(s1 1) (s2 2) . . . (sm m)g in G = (s S E ), then si 2 i, for i = 1 2 . . . m. By Denition 3.10, r contains m tuples, t1 t2 . . . tm, such that x 2 ti:X and i = ti:Y , i = 1 2 . . . m. Therefore, there exists an interpretation = ((x s1) (x s2) . . . (x sm) . . .) of r with its value set S = f(x s1) (x s2) . . . (x sm) . . .g, which implies fxg s S 2 F (r). Hence, x 2 (S s) SSi (r)(Si s). When fxg s S r then x is a true result. Otherwise, x is a maybe result. 2 In the following, we present Algorithm 3.2, which is generalized from Algorithm 3.1, to derive the maybe result of an extended division. 2F
Algorithm 3.2 : An Algorithm That Derives the Maybe Result of an Extended Division
Operation.
relations r(X Y ), s(Y ) and the TrueAns obtained from Algorithm 3.1, where X and Y are possibly composite and s contains neither partial values nor maybe tuples.
Input:
Output:
1. 2. 3. 4. 5. 6. 7. 8.
the maybe result of r(X Y )
b
s(Y ).
MaybeAns = Perform r (X Y ) = r(X Y ) / s(Y ), where \/" denotes partial semi-join X =r(r ) i.e., apply the partial grouping operation to r by X for each tuple t 2 do f if (t:X 2 TrueAns) continue = true result overrides maybe result = if (j t:Y j
