Investigating Ontological Similarity Theoretically with ... - Springer Link

Investigating Ontological Similarity Theoretically with Fuzzy Set Theory, Information Content, and Tversky Similarity and Empirically with the Gene Ontology Valerie Cross1 and Xinran Yu2 1

Computer Science and Software Engineering Department, Miami University, Oxford, OH 45056 [email protected] 2 Department of Computer Science, University of Texas, San Antonio, TX 78249 [email protected]

Abstract. This paper theoretically and empirically investigates ontological similarity. Tversky’s parameterized ratio model of similarity [3] is shown as a unifying basis of many of the well-known ontological similarity measures. A new family of ontological similarity measures is proposed that allows parameterizing the characteristic set used to represent an ontological concept. The three subontologies of the prominent GO are used in an empirical investigation of several ontological similarity measures. A new ontological similarity measure derived from the proposed family is also empirically studied. A detailed discussion of the correlation among the measures is presented as well as a comparison of the effects of two different methods of determining a concept’s information content, corpus-based and ontology-based. Keywords: Semantic similarity, ontological similarity, information content, Tversky’s parameterized ratio model, Gene Ontology.

1 Introduction In ontologies, similarity measurement is needed to determine how similar one concept is to another. An ontological similarity measure [1] is a semantic similarity measure specific to assessing similarity between concepts within an ontology. Such measures have seen a proliferation in the last several years, particularly in the biomedical and bioinformatics area [2]. Just recently more new ontological similarity measures have been proposed based on intuitive combination of information-theoretic measures with Tversky’s model of similarity [3]. In [4] the contrast model is used and then modified in [5] to use the parameterized ratio model of similarity. These recently proposed intuitive models are not the first examination of integrating the information-theoretic model and the Tversky models of similarity [6][7]. As new ontological similarity measures are proposed, evaluations of them against existing ones have typically used one of three approaches: mathematical analysis, domain-specific applications of them, and comparison of them to human judgments of S. Benferhat and J. Grant (Eds.): SUM 2011, LNAI 6929, pp. 387–400, 2011. © Springer-Verlag Berlin Heidelberg 2011

388

V. Cross and X. Yu

similarity [8]. By far, the primary approach, however, has been to compare them to human similarity judgments. More recently, due to application of ontological similarity within the Gene Ontology (GO) [9] to determine gene product similarity, other physical similarity measures such as sequence similarity [10] are being used for performance comparisons. Some efforts have been made on mathematical analysis of ontological similarity measures [11] [12] [6]. This paper theoretically and empirically investigates ontological similarity. Tversky’s parameterized ratio model of similarity [3] is shown as a unifying basis of many of the well-known ontological similarity measures. A new family of ontological similarity measures is proposed that allows parameterizing the characteristic set used to represent an ontological concept. The three subontologies of the prominent GO are used in an empirical investigation of several ontological similarity measures. A new ontological similarity measure derived from the proposed family is also empirically studied. A detailed discussion of the correlation among the measures is presented. Section 2 briefly introduces similarity assessment and the role of Tversky’s two models of similarity. Section 3 presents the components of the framework for ontological similarity and the theoretical investigation of how these various components can be used to create existing ontological similarity measures. In section 4 an empirical investigation uses the GO to compare two historical ontological similarity measures, the two recently proposed intuitive ontological similarity measures using Tversky’s models, and a new ontological similarity systematically developed from the integration of fuzzy set compatibility measures, information content and Tversky’s models. Section 5 provides a summary of this research, conclusions, and directions for future efforts.

2 Similarity Measurement In the psychological literature two main approaches to assess similarity are content models and geometric or distance models. Distance models have been found to be contrary to human similarity judgments in psychological domains since satisfying the minimality, symmetry, and the triangle inequality axioms often conflicts with human similarity judgments and also such models require quantitative continuous dimensions where often human judgments of similarity use qualitative and discrete dimensions. The content models use characteristics which are conceptualized “as more or less discrete and common elements” to determine the similarity between objects [13]. Many of the proposed set theoretic measures in the content model category are generalized by Tversky’s parameterized ratio model of similarity [3]:

STverksy-ratio(X, Y) =

 



XY





.

(1)

In the model, X and Y represent sets describing respective objects x and y.  represents the common features that describe both x and y. X  Y represents the features describing only object x.  represents the features describing only object y. The value f(X) for object x is considered a measure of the overall salience of that object. Factors adding to an object’s salience include “intensity, frequency, familiarity, good form, and informational content” [3]. The function f is an additive

Investigating Ontological Similarity Theoretically with Fuzzy Set Theory

389

function on disjoint sets, i.e., whenever X and Y are disjoint sets and when all three terms are defined, then f(X ∪ Y) = f(X) + f(Y). Set cardinality is such a function. Parameters α and β allow priority to be given to one object over the other, i.e., one object serves as the referent object to which the other object is being matched. If x is selected as the referent object, then α should be greater than β to emphasize that features describing x but not y are more important than those describing y but not x. This measure is normalized so that 0 ≤ STverksy-ratio (X, Y) ≤ 1. With α = β = 1, STversky becomes the Jaccard index [3] Sjaccard(X, Y) =



.

∪

(2)

With α = β = 1/2, STverksy-ratio becomes Dice’s coefficient of similarity [3] 

Sdice(X, Y) =

.

(3)

With α = 1, β = 0, STverksy-ratio becomes the degree of inclusion for X, that is, the proportion of X overlapping with Y [3]. Inclusion is not symmetric since α ≠ β. 

Sinclusion(X, Y) =

.

(4)

Tversky [3] also proposed an unnormalized similarity model, the contrast model. It integrates the same components but uses different mathematical operations: STverksy-contrast(X, Y) = θ





XY





.

(5)

Goodman [14] argues that assessing similarity between objects x and y is vague and meaningless without a “frame of reference.” He states " 'is similar to' functions as little more than a blank to be filled." Asking the question "How similar are x and y?" begs the answer to a subtly different question "How are x and y similar?" [15]. Thus, crucial to similarity assessment is the method to select on what features similarity is being judged. For ontological similarity, “filling in the blank” or feature selection for two concepts has varied greatly depending on the perceived objectives of the task.

3 Synthesizing Ontological Similarity Early developers of ontological or semantic similarity measures incorporated the ontological structure into the similarity measures. How these measures relate to other existing similarity measures such as the Tversky models [3] or fuzzy set compatibility measures [16][17] was not explored. Historical ontological similarity measures are examples of such existing measures when one examines how the “blanks” are filled in, i.e., how the features are selected to describe a concept in ontology and how these features may have a fuzzy set-theoretic interpretation. 3.1 Information Content in Ontologies Information content (IC) of a concept in an ontology is a measure of how specific the concept is. The more specific (general) it is, then the higher (lower) is its IC. The

390

V. Cross and X. Yu

earliest method to measure IC uses an external resource such as an associated corpus for the problem domain. The corpus-based IC measure for concept c [18] is given as ICcorpus(c) = -log p(c)

(6)

The value p(c), the probability of concept c, is determined using the frequency count of the concept, i.e. the count of the number of occurrences of the concept within the corpus. The frequency of a concept is the number of occurrences in the corpus of all words representing that concept. The frequency of the concept also includes the total frequencies of all its children concepts. The taxonomic structure of the ontology, i.e., the is-a relationships, determine the children of a concept. In some ontologies such as the GO and WordNet, however, the part-of relationship is also used in the calculation of the IC value. The probability p(c) is calculated by dividing this total frequency count by the total number of words in the corpus. Because the formula is the negative logarithm of the probability, as the probability increases the information content decreases; therefore, concepts higher in the ontology which have a greater probability of occurring have less information content than those lower in the ontology. A more recent method [19] uses the ontology structure and does not require an external resource, one of its advantages. Leaf concepts are most specific, they contain the most information. Root concepts are the least specific and contain the least information. The ontology-based IC [19] is defined as ICont(c) = log

_

/log

= 1-

_

(7)

where num_desc(c) is the number of descendants for concept c and maxont is the maximum number of concepts in the ontology. It is normalized in [0...1] with the maximum for the leaf concepts and deceases to the minimum at the root concepts. An extended Information Content (eIC) measure [5] uses other relationships between concepts, not only the taxonomic structuring ones historically used. The eIC(c) is a parameterized weighting of IC(c) and its total average relationship EIC(c). EIC(c) is the summation for each kind of relationship k, of the average of the IC(ci) for all concepts ci that are “at the end of a particular relation” [5], i.e., k, with concept c. How non-taxonomic relationships and their inverses are handled, i.e., how is “at the other end of a particular relation” is not clear. If inverse relationships are not ignored or inverse relationships are not clearly identified, a circular calculation of eIC could occur. Another concern is how and which non-taxonomic relationships are used. 3.2 A Synthesis of Fuzzy Set Compatibility Measures, Information Content, and Tverksy’s Similarity Models A concept within an ontology has many contexts and can be represented by its many different features, for example, its properties, its children, its parents, etc [24]. The three standard IC ontological similarity measures Resnik [18[, Lin [12], and JiangConrath[20] use at most three ontological concepts within an ontology. Much more information, however, is conveyed in the structure of the ontology. Another view is to consider that each concept c can be represented by a fuzzy set. Which fuzzy set is used depends on how one proceeds in “filling in the blank.” A variety of sets can be transformed into a fuzzy set description of concept c where the membership degrees


391

are some function of the IC associated with each element in the set. For example, one fuzzy set describing the concept c uses its ancestor set and the concept itself as Fanc+(c)(cj) ={IC(cj)/cj | cj is an ancestor of c or c itself}

(9)

where the + indicates to include the concept c itself in the set. Here the function on IC is simply the identify function. This fuzzy set specifies each element cj and its respective membership IC(cj) in the fuzzy set. Note either ICont or ICcorpus could be used. Here the set used to describe the concept c consists of other concepts, but the set could also be a set of links for a path associated with concept c. Various methods to measure compatibility between fuzzy sets have been proposed [16] [17]. One of the most famous is Zadeh's partial matching index between two fuzzy sets F1 and F2: Ssup-min((F1 , F2) = sup min ( F1(u), F2(u))

(10)

where sup selects the supremum membership degree from the intersection of the two fuzzy sets. An early IC ontological similarity measure [18] is formulated as simRES(c1,c2) = max c in S(c1,c2) [IC(c)]

(11)

where S(c1,c2) is the set of concepts that subsume both c1 and c2 and IC is determined using a corpus. It can be seen as the partial matching index given in (10) where Fanc+(c1) and Fanc+(c2) represent the fuzzy ancestor sets for c1 and c2 respectively in the ontology and max or sup is the scalar evaluator. The minimum operation between Fanc+(c1) and Fanc+(c2) serves as the intersection operator between the two fuzzy ancestor sets and produces the set of all common ancestors for c1 and c2. The concept with the greatest membership degree, i.e., the maximum IC, in both fuzzy sets Fanc+(c1) and Fanc+(c2), therefore, represents the partial matching similarity between concepts c1 and c2. The minimum intersection operation just takes the minimum between identical IC values for elements in both Fanc+(c1) and Fanc+(c2) since each concept has only one IC value within an ontology. Typically, Zadeh’s partial matching or consistency measure assumes normalized fuzzy sets, i.e., each fuzzy set has one element with membership degree of 1. To satisfy this assumption, the membership degrees, i.e., IC values for each ancestor and c1 can be divided by the IC value of c1 and similarly for the ancestors of c2. This normalization results in different membership degrees for ancestor concepts in the two fuzzy sets. Then the minimum operator would be necessary when performing the intersection between the two fuzzy sets. Another option which follows the general formula for partial matching index [16] is to directly normalize the result of (11) by dividing it by the min(sup Fanc+(c1), sup Fanc+(c2)) = min (IC(c1), IC(c2)). Since the original Resnik measure is used for the experiments described in Section 4, each concept within an ontology has only one IC value and no function is applied to modify the IC value used as a membership degree in a fuzzy set and no normalization of the Resnik value is performed. Other experiments are needed to see how such modifications might affect performance of a normalized partial matching measure. A major criticism of the Resnik measure is that only the shared information between the two concepts is used and not their separate information. Lin [12] defined a measure to address this criticism. It is a normalized similarity measure related to Jiang and Conrath measure [20] and given as

392

V. Cross and X. Yu

simLin(c1,c2)=

(12)

where c3 is the common ancestor with the greatest IC. The Dice similarity measure given in equation (3) is similar in form to the Lin measure in equation (12) if one simply substitutes f(X) = IC(c) and interprets f(X ∩ Y) as IC(c3). This approach has been proposed in [7] with the goal of integrating information-theoretic and settheoretic similarity measures. However, the interpretation of Tversky’s model is somewhat misleading in simply replacing f with IC and assuming that the common subsumer represents the intersection of the set of features of c1 and the set of features of c2. What the sets of features are and how the IC measure provides an additive function as specified by Tversky’s similarity model is not clearly described. The IC of a concept is based on a function of the probability of the frequencies of all its descendent concepts and not simply the intersection of common features of c1 and c2. In [6] Dice’s coefficient given in equation (3) has been shown to be the basis for both the Wu-Palmer path-based semantic similarity measure [21] given as simWu-Palmer(c1,c2)=

, ,

,

(13)

and the Lin information-content based semantic similarity measure. Dice’s coefficient establishes the connection between the Lin and Wu-Palmer measures. Many fuzzy set compatibility measures are generalizations of Tverksy’s parameterized ratio model when fuzzy set cardinality, an additive set function, is used for f, X and Y are replaced by fuzzy sets F1 and F2, and the crisp set operators are transformed to the corresponding fuzzy set operators. The Jaccard fuzzy set compatibility measure can be used to calculate IC ontological similarity between two concepts represented by their fuzzy sets of ancestors Fanc+(c1) and Fanc+(c2). The set intersection uses a t-norm, typically minimum, and the set union uses a t-co-norm, typically maximum. The Jaccard ontological similarity measure with the anc+ set to fill in the blank then becomes simJacAnc(c1, c2) = c∈Fanc + (c1) ∩ Fanc + (c2)



c∈Fanc + (c1) ∪ Fanc + (c2)

IC (c ) IC (c )

(14)

Again the min and max fuzzy set operators do not need to be explicitly used because the membership degrees of the ancestors in the fuzzy sets representing both concepts c1 and c2 are simply the ancestor's IC value. The IC value of the concept in each fuzzy set is the same since it is a function of its number of descendents. As previously suggested, however, these IC values could be normalized so that the membership degree of an ancestor in each concept's fuzzy set could differ. For this approach, the min and max operators would then be needed since the IC membership degrees then differ. Ontological similarity could also be modified by describing a concept using a different set; for example, instead of the ancestor set to describe the concept, the descendent set could be used to describe each concept. A wide variety of fuzzy set compatibility measures, many of which are fuzzy generalizations derived from Tversky’s parameterized ratio model of similarity, can be and have been used as ontological similarity measures. With this model, there still


393

can be variations depending on how each researcher decides to approach “filling in the blank”, i.e., the objectives that determine exactly what set is used to describe the concept and what method is used to assign the membership degrees in the set. 3.3 Recent Ontological Similarity Based on Intuitive Uses of Tversky’s Models New ontological similarity measures proposed use an intuitive interpretation of Tversky’s contrast and parameterized ratio models. The assumption is the function f in Tversky’s models is simply replaced with an IC measure [4][5]. This assumption was used in [7] to show that the Lin semantic similarity measure is an example of Tversky’s parameterized ratio model. This interpretation of f differs from the fuzzy set theoretic and information-content modeling of ontological similarity in [6] and presented in 3.2 where a concept is directly described by a set of elements and a function of IC is used as the membership degree of the element in the fuzzy set. The P&S semantic similarity measure [4] uses the contrast model to formulate simP&S(c1,c2) = 3 (IC(c3)) - IC(c1) - IC(c2) if c1≠c2, = 1 if c1 = c2

(15)

where c3 is the common ancestor concept with the greatest IC. It is argued that this formula represents the information theoretic counterpart of Tversky’s set theoretic contrast model. The assumption made is that f(X ∩ Y) is the same as IC(c3) where X represents a set of features describing c1 and Y represents a set of features describing c2. Similarly, f(X – Y) and f(Y – X) map to (IC(c1) – IC(c3)) and (IC(c1) – IC(c3)) respectively. The parameters are set as θ =α = β =1 to produce IC(c3) – (IC(c1) - IC(c3)) – (IC(c2) – IC(c3))

(16)

which simplifies to the measure given in equation (15) when c1≠c2. When c1=c2, then IC(c1)=IC(c2)=IC(c3); therefore, the result of equation (16) is IC(c3) so they assign the value of 1 instead if the two concepts are the same. In [22] an investigation of the P & S measure was done mathematically and empirically and showed that the P&S measure can produce negative values which cause difficulties in understanding the similarity values. Also, the measure sumps had the worst correlation with the historical ontological similarity measures in a majority of the performed experiments. In [5] another ontological similarity measure is proposed. The FaITH ( Feature and Information Theoretic) measure still assumes f as simply the IC measure and uses the same mappings for f(X ∩ Y), f(X – Y), and f(Y – X) as fro simP&S. The only change is to use Tversky’s parameterized ratio model of similarity instead of the contrast model. These mappings are substituted into equation (1) with α = β =1 to produce simFaITH(c1,c2)=

(17)

which is very similar to the simLin measure in equation (12). Subtracting IC(c3) from both the numerator and denominator of equation (12) produces the FaITH measure. With this modification, the simFaITH measure must always produce a smaller than or equal to (only when both are 0) value compared to simLin. As IC(c3) approaches 0, the ratio of simFaITH /simLin approaches 0.5. There is a strong correlation between these two measures as the empirical investigations show in the next section.

394

V. Cross and X. Yu

In both simP&S and simFaITH, the key assumption is that an information theoretic view of Tversky’s models of similarity is possible by the simple substitution of IC for the function f. The f(X ∩ Y) which is a measure of the amount of shared elements, i.e., features, between sets X and Y in set-theoretic model is simply replaced by IC(c3), the amount of shared information for c1 and c2, i.e., the IC of their most specific ancestor concept c3. A difficulty with this interpretation is the difference between a set of features representing the intersection of two sets such as the shared set of properties between c1 and c2 and the shared information between c1 and c2 as measured by the IC of their most specific ancestor c3. For two sets X and Y, a function f of their intersection does not change if more sets exist that are included as part of that intersection, i.e., that set of shared features. As explained in section 3.1 IC(c3), however, representing a measure of shared information between c1 and c2, does not solely depend on IC(c1) and IC(c2) but is affected by the IC of all of the children of c3. These children also have the same shared information with c1 and c2. The amount of this shared information should not decrease simply because more children are added to c3. This difference makes a case against a simple substitution of IC(c3) for f(X ∩ Y) in both Tversky’s set models of similarity. This section has proposed fuzzy set compatibility measures, many based on the Tversky ratio model of similarity, that can be combined with information content measures to produce a general model for a wide variety of ontological similarity measures. The key considerations are what set is used to represent a concept and what method is used to determine the degree of membership of each element in the set describing a concept. To further explore this model, the following empirical study uses the fuzzy set Fanc+(c) for concept c in addition to the two historical Resnik and Lin measures and the two recently proposed measures P&S and FaITH.

4 Empirical Investigations Using the Gene Ontology The bioinformatics domain is serving as a primary impetus for the creation of new ontological similarity measures. The Gene Ontology (GO) [9] is an important ontology used in this domain. It contains concepts used to annotate genes or gene products. The GO relationships include two major types of structuring links: “is-a” and “part-of”. For determining IC, the standard practice has been to use only these two types of relationships [10]. Besides being a major bioinformatics ontology, the GO contains three mutually exclusive subontologies: biological process (BP), cellular component (CC) and molecular function (MF) each with varying sizes. The CC subontology has 2636 concepts, of which 1724 are leaf concepts (approximately 65% leaves). The MF ontology has 8668 concepts, of which 6956 are leaf concepts (about 80.2%). The BP ontology has 18059 concepts, of which 8442 are leaf concepts (about 46.7%). The MF ontology is more single parent structured averaging 1.2 parents per concept while the CC and BP average 1.9 parents per concepts. Another objective of this investigation is to use a much larger number of concept pairs than the very small number of pairs ranging from 28 to 65 used in human similarity judgment experiments with WordNet and MeSH. For each subontology,


395

GO concepts are randomly selected and similarities between all pairs of selected concepts within a subontology are calculated. Around 5% of each subontology is randomly selected: 100 for CC, 500 for MF and 880 for BP, resulting in 5050, 125250, and 387640 concept pairs, respectively. A direct comparison approach is used that does not require an arbitrary gold standard for performance comparison. Because the plethora of ontological similarity measures are mostly the result of numerous researchers developing a new measure for what is perceived as a very specific objective, these measures have typically been assessed with that objective in mind. Performance comparisons are limited to a small group of measures to an experimentally developed gold standard. The typical conclusion, more often than not, is that the new measure performs as well if not better than the others in the group. Finding one gold standard to encompass the wide variety of uses for ontological similarity is a daunting challenge. In the bioinformatics domain, measuring the degree of gene similarity between genes requires an aggregation operator in addition to a semantic similarity measure. A wide variety of aggregation operators have been used to produce the final gene pair similarity. The gene product semantic similarity is then correlated with actual gene sequence similarity, a potential gold standard in the bioinformatics domain [1]. The focus here is on the ontological similarity measures themselves within the GO subontologies so that the need for the selection of various aggregation operators can be eliminated. IC based ontological similarity measures are the focus of this study since within the bioinformatics, they have been predominantly used [1], and the primary pathbased measure Wu-Palmer has been shown equivalent to the Lin measure when each path edge is weighted by the difference in the child’s and parent’s IC value [6]. In the following ICont given in equation (7) is used since it is simpler to calculate and has also been reported in several studies [4][5][19][23] to perform as well if not better than ICcorpus. The five IC ontological similarity measures selected are the Resnik (R), the Lin (L), the FaITH (F), the P&S (P), and the proposed JacAnc (J). Performance evaluations of semantic similarity measures to human similarity judgments typically use the Pearson correlation coefficient. It measures the degree of linear relationship between two variables. The assumption is each variable is approximately normally distributed. The Spearman coefficient assumes that the variables under consideration are measured on at least a rank order or ordinal scale. It is can be viewed as the Pearson coefficient in terms of proportion of variability accounted for, only the ranks of the observations for each variable are used to calculate the Spearman coefficient. The Spearman correlation coefficient produces a 1 when the two variables being compared are monotonically related and does not require the strict linear relationship of the Pearson correlation Both coefficients have been calculated. For this investigation, each correlation coefficient calculated had a pvalue < 0.001, which indicate that the result is significant. Figures 1 through 3 are for the Pearson coefficient for the CC, MF, and BP, respectively. Figures 4 through 6 are for the Spearman. The y-axis is the coefficient value; the x-axis is the respective measure: R-Resnik, L-Lin, F-FaITH, P-P&S, and JJacAnc. Each line shows how a measure correlates with all the other measures. For both coefficients, the Resnik measure correlates best with the Lin and FaITH measures across all three subontologies. The FaITH measure is strongly connected to the Lin measure. It uses a consistent reduction to both the numerator and denominator

396

V. Cross and X. Yu

of the Lin measure by the IC of the ancestor with the greatest IC. The experiments verify this showing that these two measures have 1.0 correlation with each other across all subontologies for the Spearman coefficient and have approximately the same high Pearson correlation, 0.97, across the three subontologies. Across all three subontologies, the Spearman correlation for the Lin measures with each the other three ontological similarity measures is identical to that of the corresponding Spearman correlation for FaITH.

1 R-P 0.95

L-P

0.9

F-P P-P

0.85 R

L

F

P

J

J-P

Fig. 1. Pearson Correlation between Measures for CC

1 0.95

R-P

0.9

L-P

0.85

F-P

0.8

P-P

0.75 R

L

F

P

J

J-P

Fig. 2. Pearson Correlation between Measures for MF

1 0.95 0.9 0.85 0.8 0.75 0.7 0.65

R-P L-P F-P P-P R

L

F

P

J

Fig. 3. Pearson Correlation between Measures for BP

J-P


1 0.95 0.9 0.85 0.8 0.75 0.7 0.65

397

R-S L-S F-S P-S R

L

F

P

J-S

J

Fig. 4. Spearman Correlation between Measures for CC

1 0.95 0.9 0.85 0.8 0.75 0.7

R-S L-S F-S P-S R

L

F

P

J

J-S

Fig. 5. Spearman Correlation between Measures for MF

1 0.95 0.9 0.85 0.8 0.75

R-S L-S F-S P-S R

L

F

P

J

J-S

Fig. 6. Spearman Correlation between Measures for BP

Resnik correlates (Pearson) about the same with P&S and JacAnc for CC but then it correlates slightly better with the P&S for both MF and BP. The Resnik has the worst Spearman correlation with the P&S measure across all subontologies. The correlation results for Lin parallel those for the Resnik since across all three subontologies and both correlations coefficients the smallest correlation between the two is 0.988. This result occurs because Resnik and Lin measures are identical when ICont is used, i.e., simRes(c1, c2) = simLin(c1, c2) = 2IC(c3)/(1 + 1) =IC(c3). The FaITH measure correlates (both coefficients) best with the Resnik and Lin measures for all subontologies. It has a much higher Spearman correlation with the

398

V. Cross and X. Yu

JacAnc measure than with the P&S measure for all subontologies. Only for Pearson and the BP does it correlate slightly better with the P&S than with the JacAnc. The P&S measure correlates (Pearson) best with the Resnik, Lin, and FaITH measures across all subontologies, with about a 0.04 difference in the range of values. Pearson correlation with JacAnc measure is more varied with the best being for the CC and the worst being for the BP. A similar result can be seen for the Spearman coefficient with lower correlation values and a slightly bigger range in values. The JacAnc measure correlates best (Pearson) with the FaITH measures across all three subontologies. For the Spearman correlation coefficient with JacAnc, the Resnik, Lin, and FaITH have all very close coefficients for each of the subontologies. In general Pearson correlations across all subontologies show the same patterns. P&S and JacAnc have the poorest correlation with the other three measures though there is a flip-flop between the P&S measure and the JacAnc measure for the Lin and FaITH measures for the CC and BP only. The worst correlation for P&S occurs with JacAnc. In general Spearman correlations show a similar pattern, but the P&S measure clearly correlates less with all the other measures. The distinction in the correlation values for the other four decreases going from CC to MF to BP. This result could be due to the increasing number of concept pairs from CC to MF to BP. Generally, Resnik, Lin and FaITH are strongly correlated. The Lin and FaITH produce identical results with respect to the Spearman coefficient. From these experiments, the lowest correlation between these three measures occurs between FaITH and Res for the MF with a 0.961. Space limitations prevent fully discussing the IC ontological similarity using ICcorpus. The correlation for each ICont ontological similarity with its ICcorpus version is summarized. The corpus used is the GOA-UniProt Version 79 database (http://www.ebi.ac.uk/GOA/). The experiments showed a very strong correlation between the ICont and ICcorpus versions of the ontological similarity measures. P&S has the lowest correlations across both coefficients and all subontologies (Pearson 0.74 on MF, Spearman 0.58 on MF). The JacAnc has the highest Pearson correlations for all subontologies with it lowest value being 0.976 for the BP. For the Spearman correlations, all measures but P&S had very high and similar correlation values. With FaITH we did not use the eIC discussed in section 3.1. In [5] simFaITH + eIC always produced higher Pearson correlations with human similarity judgments than any of the others. The paper seems to indicate that the other similarity measures used ICont. Since the specific kinds of relationships used to calculate eIC are not clearly stated, ICont with only the is-a and part-of links is used in our work. Another experiment in [5] uses a data set consisting of 36 concept pairs from the MeSH (Medical Subject Headings) ontology to evaluate the performance of simFaITH on a domain related ontology. The conclusion again was that simFaITH had the highest correlation with the human similarity judgments. It is understood that for MeSH all ontological similarity measures including simFaITH were calculated using ICont. As previously discussed, the simFaITH measure should always produce a smaller or equal value to that of simLin. However, a problem in either the calculation of one or both of these measures is seen in [5] since in the reported similarity values for the 36 pairs, 14 of the simFaITH values are greater than the corresponding simLin values.


399

4 Conclusions and Future Work Several IC based ontological similarity measures are theoretically and empirically investigated. The motivation of the study is to establish a general fuzzy set-theoretic framework of IC ontological similarity that has as its basis Tversky’s similarity models, fuzzy set compatibility measures, and information content and to examine other recently proposed measures that use an intuitive IC version of Tversky’s model [4][5]. Historical ontological similarity measures are shown to be examples of fuzzy set compatibility measures when a concept is described by a fuzzy set of elements and their membership degrees are determined as a function of IC. The empirical study uses two historical measures Resnik [18] and Lin [12], two recently proposed measures, P&S [4] and FaITH [5], and JacAnc, a measure presented here and derived from the general fuzzy set-theoretic framework for IC ontological similarity. ICont is used in comparing these ontological measures. The cellular component (CC), the molecular function (MF) and the biological process (BP) of the Gene Ontology are used to empirically compare these five measures. The comparison does not use a gold standard but instead uses the correlation between their similarity values on sets of randomly selected concept pairs from each of the three GO subontologies. The FaITH and Lin measures are shown to have a mathematical relationship validated by their very high Pearson correlation and identical Spearman correlations values. The Resnik and Lin measure are strongly correlated. This result can be partly explained by the equivalence of the two measures when the similarity between leaf concepts is being determined. The JacAnc measure shows overall the strongest correlation between its ICont and ICcorpus versions over all subontologies although each measure shows strong correlation between its two IC versions. It appears for JacAnc that incorporating more information to represent a concept, i.e. a concept’s set of ancestors, mitigates the difference in using ICont in place of ICcorpus. The correlations for the two P&S versions are the smallest. Future research is to explore the use of several of these ontological similarity measures in various tasks, for example ontology alignment in order to further compare their performance.

References 1. Cross, V.: Ontological Similarity. In: Popescu, M., Xu, D. (eds.) Data Mining in Biomedicine Using Ontologies, pp. 23–43. Artech House, Norwood, MA (2009) 2. Pesquita, C., Faria, D., Falcão, A.O., Lord, P., Couto, F.M.: Semantic Similarity in Biomedical Ontologies. PLoS Comput. Biol. 5(7), e1000443, doi:10.1371/journal.p(c)bi.1000443 (2009) 3. Tversky, A.: Features of Similarity. Psychological Rev. 84, 327–352 (1977) 4. Pirrò, G., Seco, N.: Design, Implementation and Evaluation of a New Semantic Similarity Metric Combining Features and Intrinsic Information Content. In: Chung, S. (ed.) OTM 2008, Part II. LNCS, vol. 5332, pp. 1271–1288. Springer, Heidelberg (2008) 5. Pirrò, G., Euzenat, J.: A Feature and Information Theoretic Framework for Semantic Similarity and Relatedness. In: Proceedings of International Semantic Web Conference, vol. (1), pp. 615–630 (2010)

400

V. Cross and X. Yu

6. Cross, V.: Tversky’s Parameterized Similarity Ratio Model: A Basis for Semantic Relatedness. In: Proceedings of the 2006 Conference of North American Fuzzy Information Processing Society (NAFIPS), Montreal, Canada (June 3-6, 2006) 7. Cazzanti, L., Gupta, M.R.: Information-theoretic and Set-theoretic Similarity. In: Proc. IEEE Intl. Symposium on Information Theory (2006) 8. Budanitsky, A., Hirst, G.: Semantic Distance in WordNet: An Experimental, Applicationoriented Evaluation of Five Measures. In: Workshop on WordNet and Other Lexical Resources, Second meeting of the NAACL, Pittsburgh (2001) 9. The Gene Ontology Consortium, http://www.geneontology.org/ 10. Lord, P., Stevens, R., Brass, A., Goble, C.: Investigating semantic similarity measures across the Gene Ontology: the relationship between sequence and annotation. Bioinformatics 19, 1275–1283 (2003) 11. Wei, M.: An Analysis of Word Relatedness Correlation Measures. Master’s thesis, University of Western Ontario, London, Ontario (May 1993) 12. Lin, D.: An information-theoretic definition of similarity. In: Proc. of the 15th Int. Conf. on Machine Learning, pp. 296–304. Morgan Kaufmann, San Francisco (1998) 13. Attneave, F.: Dimensions of Similarity. American J. of Psychology 63, 516–556 (1950) 14. Goodman, N.: Seven strictures on similarity. In: Goodman, N. (ed.) Problems and projects, pp. 437–447. Bobbs-Merrill, New York (1972) 15. Medin, D.L., Goldstone, R.L., Gentner, D.: Respects for Similarity. Psychological Review 100(2), 254–278 (1993) 16. Cross, V.: An Analysis of Fuzzy Set Aggregators and Compatibility Measures, Ph.D. Dissertation, Computer Science and Engineering, Wright State University, Dayton, OH, 264 pages (March 1993) 17. Cross, V., Sudkamp, T.: Similarity and Compatibility in Fuzzy Set Theory: Assessment and Applications. Physica-Verlag, New York (2002) ISBN: 3-7908-1458 18. Resnik, P.: Using information content to evaluate semantic similarity in taxonomy. In: Proc. of the 14th Intl Joint Conference on Artificial Intelligence, pp. 448–453 (1995) 19. Seco, N., Veale, T., Hayes, J.: An Intrinsic Information Content Metric for Semantic Similarity in WordNet. In: ECAI, pp. 1089–1090 (2004) 20. Jiang, J., Conrath, D.: Semantic similarity based on corpus statistics and lexical taxonomy, In: Proc. of the 10th International Conference on Research (1997) 21. Wu, Z., Palmer, M.: Verb semantics and lexical selection. In: Proc. of the 32nd Annual Meeting of the Assoc. for Computational Ling, NM, Las Cruces, pp. 133–138 (1994) 22. Yu, X.: A Mathematical and Experimental Investigation of Ontological Similarity Measures and their Use in Biomedical Domains. Master’s Thesis, Computer Science and Software Engineering, Miami University, Oxford OH (2010) 23. Cross, V., Sun, Y.: Semantic, Fuzzy Set and Fuzzy Measure Similarity for the Gene Ontology. In: Proceedings of the IEEE International Conference on Fuzzy Systems. Imperial College, London (2007) 24. Rodriguez, M.A., Egenhofer, M.J.: Determining Semantic Similarity among Entity Classes from Different Ontologies. IEEE Transactions on Knowledge and Data Engineering 15(2), 442–456 (2003)