TEXT KNOWLEDGE ENGINEERING BY

LIF

Computational Linguistics Research Group

Albert-Ludwigs-Universitat Freiburg im Breisgau Germany

TEXT KNOWLEDGE ENGINEERING BY QUALITATIVE TERMINOLOGICAL LEARNING Udo Hahn & Klemens Schnattinger

1997 LIF

REPORT 8/97

TEXT KNOWLEDGE ENGINEERING BY QUALITATIVE TERMINOLOGICAL LEARNING Udo Hahn & Klemens Schnattinger LIF

Computational Linguistics Research Group Albert-Ludwigs-Universitat Freiburg Werthmannplatz 1 79085 Freiburg, Germany

http://www.coling.uni-freiburg.de fhahn,[email protected]

Abstract

We propose a methodology for enhancing domain knowledge bases through natural language text understanding. The acquisition of previously unknown concepts is based on the assessment of the \quality" of linguistic and conceptual evidence underlying the generation and re nement of concept hypotheses. Text understanding and concept learning are both grounded on a terminological knowledge representation and reasoning framework.

Appeared in: A.Hameurlain & A.M.Tjoa (Eds.), Database and Expert Systems Applications. Proc. of the 8th Intl. Conference - DEXA '97. Toulouse, France, September 1-5, 1997. Berlin etc.: Springer, 1997, pp.623-632 (LNCS, 1308).

In: A.Hameurlain & A.M.Tjoa (Eds.), Database and ExpertSystems Applications. Proc. of the 8th Intl. Conference - DEXA ’97.Toulouse, France, Sept

Text Knowledge Engineering by Qualitative Terminological Learning Udo Hahn & Klemens Schnattinger Computational Linguistics Lab { Text Knowledge Engineering Group Freiburg University, Werthmannplatz 1, D-79085 Freiburg, Germany

LIF

http://www.coling.uni-freiburg.de/

Abstract. We propose a methodology for enhancing domain knowledge bases through natural language text understanding. The acquisition of previously unknown concepts is based on the assessment of the \quality" of linguistic and conceptual evidence underlying the generation and re nement of concept hypotheses. Text understanding and concept learning are both grounded on a terminological knowledge representation and reasoning framework.

1 Introduction Text knowledge engineering is a research area in which, on the one hand, natural language processing technology is applied for the automatic acquisition of knowledge from textual documents and, on the other hand, knowledge acquired from texts is integrated into already existing, yet underspeci ed domain knowledge bases. A constructive update of such knowledge repositories is performed according to formal constraints underlying proper knowledge base management and heuristic principles guiding knowledge engineering. Such a task is unlikely to be solved by simply plugging in o-the-shelf components. This is due to the vivid interactions between text analysis and knowledge acquisition processes. By this we mean the initial creation of various hypotheses upon the rst mention of an unknown concept, the continuous re nement of and discrimination among competing hypotheses as more and more knowledge becomes available and, nally, the convergence on the most plausible hypothesis. Hence, the learning mechanism we describe proceeds incrementally, in a bootstrapping fashion, and is fairly knowledge-intensive, as we provide a classi cation-based reasoning scheme for the assessment of the dierent forms of evidence being encountered. Two types of evidence are taken into account for continuously discriminating and re ning the set of concept hypotheses | the type of linguistic construction in which an unknown lexical item occurs and conceptually motivated annotations of concept hypotheses re ecting structural patterns of consistency, mutual justi cation, analogy, etc. in the knowledge base. These forms of initial evidence are represented by a set of quality labels. Concept acquisition can then be viewed as a quality-based decision task which is decomposed into three constituent parts: the continuous generation of quality labels for single concept hypotheses (re ecting the reasons for their formation and their signi cance in the light of other

metacontext

initial context KB kernel

translation rules

reified KB kernel

qualification rules

hypothesis generation translation rules [i-th hypo space]

qualified hypothesis space

hypothesis space

qualified reified hypothesis space

translation rules [(i+1)-th hypo space]

text KB

qualification rules

reified hypothesis space

qualifier

selection criteria

reified text KB

Fig. 1. Quality-Based Concept Acquisition System hypotheses), the estimation of the overall credibility of single concept hypotheses (taking the available set of quality labels for each hypothesis into account), and the computation of a preference order for the entire set of competing hypotheses based on these accumulated quality judgments.

2 System Architecture The methodology and corresponding system architecture (cf. Fig. 1) we propose serves the representation of quality-based assertions about certain propositions and the reasoning about characteristic properties and relations between these assertions. The text understanding processes use a terminological model of the underlying domain, the knowledge base (KB) kernel, on the basis of which the parser [6] generates a conceptual interpretation of the text in the text KB. Whenever an unknown lexical item occurs during the text understanding process, and this item is considered relevant for learning according to distributional criteria, conceptual hypotheses are generated [5]. These take linguistic criteria (mirrored by the assignment of corresponding linguistic quality labels) as well as conceptual conditions into account. Multiple concept hypotheses for a single lexical item are organized in terms of a corresponding hypothesis space as part of the text KB, each subspace holding dierent or further specialized concept hypotheses. In order to reason about the credibility of these hypotheses a mirror image of the initial context which combines the KB kernel and the text KB is generated | the so-called metacontext. This is achieved by a truth-preserving mapping which includes the rei cation of the original terminological assertions from the initial context [13]. These rei ed representation structures are then submitted to conceptual quali cation rules which determine purely conceptual indicators of credibility of the associated hypotheses and assign corresponding conceptual quality labels to them in the rei ed hypothesis space. A classi er extended by an evaluation metric for quality-based selection criteria, the quali er, then determines the most credible concept hypotheses [11]. Only those will be remapped from their rei ed to the original terminological form by way of (inverse) translation rules, and thus become available again for the text understanding process.

Syntax

Semantics

C uD C I \ DI C tD C I [ DI ? 1 R f(d; d0 ) 2
I 
I j (d0 ; d) 2 RI g RuS RI \ S I f(d; d0 ) 2 RI j d 2 C I g C jR RjC f(d; d0 ) 2 RI j d0 2 C I g (R1Table ; ::;Rn )1. Some RI1  ::Concept  RIn and Role Terms

Axiom A =: C a:C

Q =: R

Semantics

AI = C I aI 2 C I

Q I = RI

(aI ; bI ) 2 RI Table 2. Axioms for Concepts and Roles

aRb

Thus, we come full circle. The entire cycle is triggered for each new evidence that becomes available for a concept to be learned as the text understanding process proceeds. In the following, we will illustrate the working of this architecture in more detail based on processing the example phrase (1) \.. the case of the PS/2-57SX .." as the rst learning step for the unknown item \PS/2-57SX". Other learning steps are based on statement (2) \.. the PS/2-57SX with 4MB ..", and (3) \.. the PS/2-57SX is equipped with a 3.5 inch disk drive ..".

2.1 Terminological Logics We use a concept description language (for a survey, cf. [8]) with a standard settheoretical semantics (the interpretation function I ). The language has several constructors combining atomic concepts, roles and individuals (see Table 1). By means of terminological axioms a symbolic name can be de ned for each concept and role term; concepts and roles are associated with concrete individuals by assertional axioms (see Table 2). Consider the following example: (P1) case-01 : Case (P2) PS=2-57SX: has-case case-01 (P3) has-case = jhas-partj (Computer-System t Device)

Case

The assertions P1 and P2 read as: the instance case-01 belongs to the concept Case and the tuple hPS=2-57SX; case-01i belongs to the binary relation has-case, respectively. The relation has-case is de ned as all has-part relations which have their domain restricted to the disjunction of the concepts Computer-System or Device and their range restricted to the concept Case.

2.2 Quality Labels Linguistic quality labels re ect structural properties of phrasal patterns or dis-

course contexts in which unknown lexical items occur | we here assume that the type of grammatical construction exercises a particular interpretative force on the unknown item and, at the same time, yields a particular level of credibility for the hypotheses being derived. As a concrete example of a high-quality label, consider the case of Apposition. This label is generated for constructions

such as \.. the printer @A@ ..", with \@..@" denoting the unknown item. The apposition almost unequivocally determines \@A@" (considered as a potential noun)1 to denote a Printer. This assumption is justi ed independent of further conceptual conditions, simply due to the nature of the linguistic construction being used. Still of good quality but already less constraining are occurrences of the unknown item in a CaseFrame construction as illustrated by \.. @B@ is equipped with a 3.5 inch disk drive ..". In this example, case frame speci cations of the verb \equip" that relate to its patient role carry over to \@B@". So \@B@" may be anything that is equipped with a 3.5 inch disk drive, e.g., a computer system. Conceptual quality labels result from comparing the conceptual representation structures of a concept hypothesis with already existing representation structures in the underlying domain knowledge base from the viewpoint of structural similarity, incompatibility, etc. The closer the match, the more credit is lent to a hypothesis. For instance, a very positive conceptual quality label such as M-Deduced is assigned to multiple derivations of the same concept hypothesis in dierent hypothesis (sub)spaces. Still positive labels are assigned to terminological expressions which share structural similarities, though they are not identical. For instance, the label C-Supported is assigned to any hypothesized relation R1 between two instances in case another relation, R2, already exists in the KB involving the same two instances, but where the role llers occur in \inverted" order (note that R1 and R2 need not necessarily be conceptually inverse relations such as with \buy" and \sell"). This rule captures the inherent symmetry between concepts related via quasi-inverse conceptual relations.

2.3 Hypothesis Generation Depending on the type of the syntactic construction in which the unknown lexical item occurs dierent hypothesis generation rules may re. In our example (1): \.. the case of the PS/2-57SX ..", a genitive noun phrase places only few constraints on the item to be learned. In the following, let target be the unknown item (\PS/2-57SX") and base be the known item (\case"), the conceptual relation of which to the target is constrained by the syntactic relation in which their lexical counterparts co-occur. The main constraint for genitives says that the target concept lls (exactly) one of the n roles of the base concept. Since it cannot be decided on the correct role yet, n alternative hypotheses have to be opened (unless additional constraints apply) and the target concept is assigned as a ller of the i-th role of base in the corresponding i-th hypothesis space. As a consequence, the classi er is able to derive a suitable concept hypothesis by specializing the target concept (initially Top, by default) according to the value restriction of the base concept's i-th role. Additionally, this rule assigns a syntactic quality label to each i-th hypothesis indicating the type of syntactic construction in which target and base co-occur. 1

Such a part-of-speech hypothesis can directly be derived from the inventory of valence and word order speci cations underlying the dependency grammar model we use [6].

=: Hypo u Cred1 =: Thresh2 u max(Apposition term) max(M-Deduced term) Thresh2 =: Thresh1 u Cred2 =: Cred1 u max(CaseFrame term) (C-Supported term) Table 3. Threshold Levels Table 4.max Credibility Levels Thresh1

In the given KB kernel, for our example ve roles must be considered for the base concept Case. Three of them, has-weight, hasphysical-size, has-price, are ruled out due to the violation of a simple integrity constraint (\PS/2-57SX" does not denote a measurable unit). Hence, two hypotheses remain to be made, one Fig. 2. Three Hypothesis Spaces treating \PS/2-57SX" as a producer of hardware via the role develops in the hypothesis space 1.1, the other one stipulating that \PS/2-57SX" may be a kind of Hardware via the role has-case in the hypothesis spaces 2.1 and 3.1, respectively (cf. Fig. 2).2

2.4 Conceptual Quali cation

H

Quality annotations of the conceptual status of concept hypotheses are derived from quali cation rules. For instance, one of the rules applies to the case where the same assertion is deduced in at least two H’ dierent hypothesis spaces (cf. H and H in Fig. 3). We take this quasi-con rmation as a strong support for the hypothesis under consideration. Hence, the very positive conceptual quality label M-Deduced Fig. 3. A Conceptual is derived (for a formal speci cation of several quali- Quali cation Scenario cation rules, cf. [4]). Considering our example, for \PS/2-57SX" the concept hypotheses ComputerSystem and Device were both derived independently of each other in dierent hypothesis spaces. Hence, Hardware as their common superconcept has been multiply derived by the classi er in each of these spaces, too. O2

O1

R

0

O2

O1

R

2

As the classi er aggressively pushes hypothesizing to be maximally speci c, two distinct hypotheses are immediately stipulated for Hardware, namely Device and Computer-System.

2.5 Quality-Based Classi cation Whenever new evidence for or against a concept hypothesis is brought forth in a single learning step the entire set of concept hypotheses is reevaluated. First, a selection process eliminates weak or even untenable hypotheses from further consideration. The corresponding quality-based selection among hypothesis spaces is grounded on threshold levels as de ned in Table 3 (in Section 3 this selection level will be referred to as TH). Their de nition takes mostly linguistic evidence into account and evolved in a series of validation experiments. At the rst threshold level, all hypothesis spaces Hypo with the maximum of Apposition label are selected. If more than one hypothesis is left to be considered, at the second threshold level only concept hypotheses with the maximum number of CaseFrame assignments are approved. Those hypothesis spaces that have ful lled these threshold criteria will then be classi ed relative to two dierent credibility levels as de ned in Table 4 (in Section 3 this selection level will be referred to as CB). The rst level of credibility contains all of the hypothesis spaces which have the maximum of M-Deduced labels, while at the second level (again, with more than one hypothesis left to be considered) those are chosen which are assigned the maximum of C-Supported labels. Threshold and credibility criteria make use of composed roles, a speci c domain and range restriction on roles (in Tables 3 and 4 abbreviated as \X term"), and a new constructor max for the path computation. A complete terminological speci cation is given in [11]. To illustrate the use of threshold criteria, consider phrase (3): \.. the PS/257SX is equipped with a 3.5 inch disk drive ..", for which a CaseFrame assignment is triggered in those hypothesis spaces where the unknown item is considered a Computer-System or Device (or a specialization of any of them).3 Therefore, the hypothesis space associated with the Producer-Hardware reading (cf. hypothesis space 1.1 in Fig. 2) is ruled out by Thresh2 in the third learning step. As far as the sample phrase (1) is concerned, three hypothesis spaces are generated two of which stipulate a Hardware hypothesis. As the quality label M-Deduced has been derived by the classi er, the processing of the rst sample phrase already yields a preliminary ranking with these two Hardware hypotheses preferred over the one associated with ProducerHardware (cf. Fig. 2). Note that only in the third learning step this preference leads to an explicit selection (as discussed above) such that the ProducerHardware hypothesis is actually ruled out from further consideration.

3 Evaluation In this section, we brie y discuss some data from an empirical evaluation of our concept acquisition system (more detailed results are presented in [12]). We focus here on its learning accuracy and learning rate. Due to the given learning environment, the measures we apply deviate from those commonly used in 3

The Producer-Hardware hypothesis cannot be annotated by CaseFrame, since a hardware producer cannot be equipped with a 3.5 inch disk drive.

Step Phrase

Semantic Interpretation

1. the case of the PS/2-57SX .. (GenitiveNP,Case.1,case-of,PS/2-57SX) 2. the PS/2-57SX with 4MB .. (PP-Attach,PS/2-57SX,has-degree,MB-Degree.1) 3. the PS/2-57SX is equipped .. (CaseFrame,equip.1,patient,PS/2-57SX) .. with a 3.5 inch disk drive (CaseFrame,equip.1,co-patient,DiskDrive.1) Table 5. Semantic Interpretation7!of(PS/2-57SX,has-disk-drive,DiskDrive.1) a Text Fragment Featuring \PS/2-57SX"

the machine learning community. For instance, conceptual hierarchies naturally emerge in terminological frameworks. So, a prediction can be more or less precise, i.e., it may approximate the goal concept at dierent levels of speci city. This is captured by our measure of learning accuracy which takes into account the conceptual distance of a hypothesis to the goal concept of an instance, rather than simply relating the number of correct and false predictions as is usually done in machine learning environments. Furthermore, in our approach learning is achieved by the re nement of multiple hypotheses about the class membership of an instance. Thus, the measure of learning rate we propose is concerned with the reduction of hypotheses as more and more information becomes available about one particular new instance, rather than just considering the increase of correct predictions as more and more instances are being processed. We investigated a total of 101 texts from a corpus of information technology magazines. For each of them 5 to 15 learning steps were considered. A learning step is operationalized here by the representation structure that results from the semantic interpretation of an utterance which contains the unknown lexical item. In order to clarify the input data available for the learning system, cf. Table 5. It consists of three single learning steps for the unknown lexical item \PS/2-57SX" already discussed. Each learning step is associated with a particular natural language phrase in which the unknown lexical item occurs and the corresponding semantic interpretation in the text knowledge base (the data also incorporate the type of syntactic construction in which the unknown item occurs { this indicates the kind of linguistic quality label to be issued; \7!" provides the results from the application of verb interpretation rules). Learning Accuracy. First, we investigated the learning accuracy of the system, i.e., the degree to which the system correctly predicts the concept class which subsumes the target concept under consideration. Learning accuracy (LA) is a path-sensitive measure for concept graphs. It relates the distance, i.e., the number of node being traversed, between those parts of a concept prediction which are correct and those which are not (cf. [12] for a technical treatment). Fig. 4 depicts the learning accuracy curve for the entire data set (101 texts). The evaluation starts at LA values in the interval between 48% to 54% for LA -, LA TH and LA CB, respectively, in the rst learning step. In the nal step, LA rises up to 79%, 83% and 87% for LA -, LA TH and LA CB, respectively. The pure terminological reasoning machinery (denoted as LA -) which does not incorporate the quali cation calculus always achieves an inferior level of learning accuracy (and also generates more hypothesis spaces) than the learner equipped with the quali cation calculus. Furthermore, the inclusion of conceptual criteria

ALL

ALL 200 LR -LR TH LR CB

LA CB LA TH LA --

1.0 0.9

150

0.8

0.6

LR

LA

0.7

0.5

100

0.4 0.3

50

0.2 0.1 0

0 1

2

3

4

5

6

7 8 9 10 11 12 13 14 15 Learning steps

Fig.4. Learning Accuracy (LA) for the Entire Data Set

1

2

3

4

5

6 7 8 9 10 11 12 13 14 15 Learning steps

Fig.5. Learning Rate (LR) for the Entire Data Set

(CB) supplementing the linguistic criteria (TH) helps a lot to focus on the relevant hypothesis spaces and to further discriminate the valid hypotheses (on the range of 4% of precision). Note that an already signi cant plateau of accuracy is usually reached after the third step viz. 67%, 73%, and 76% for LA -, LA TH, and LA CB, respectively, in Fig. 4. This indicates that our approach nds the most relevant distinctions in a very early phase of the learning process, i.e., it requires only a few examples. Learning Rate. The learning accuracy focuses on the predictive power of the learning procedure. By considering the learning rate (LR), we turn to the stepwise reduction of alternatives of the learning process. Fig. 5 depicts the mean number of transitively included concepts for all considered hypothesis spaces per learning step (each concept hypothesis denotes a concept which transitively subsumes various subconcepts). Note that the most general concept hypothesis in our example denotes Object which currently includes 196 concepts. In general, we observed a strong negative slope of the curve for the learning rate. After the rst step, slightly less than 50% of the included concepts are pruned (with 93, 94 and 97 remaining concepts for LR CB, LR TH and LR -, respectively). Again, learning step 3 is a crucial point for the reduction of the number of included concepts (ranging from 9 to 12 concepts). Summarizing this evaluation experiment, the quality-based learning system yields competitive accuracy rates (a mean of 87%), while at the same time it exhibits signi cant and valid reductions of the predicted concepts (up to two, on the average).

4 Related Work Our approach bears a close relationship to the work of [3], [9], [10], [15], and [7], who aim at the automated learning of word meanings from context using a knowledge-intensive approach. But our work diers from theirs in that the need to cope with several competing concept hypotheses and to aim at a reason-based selection is not an issue in these studies. Learning from real-world textual input usually provides the learner with only sparse and fragmentary evidence so that multiple hypotheses are likely to be derived requiring subsequent assessment.

The work closest to ours has been carried out by Rau et al. [9]. As in our approach, concept hypotheses are generated from linguistic and conceptual data. Unlike our approach, the selection of hypotheses depends only on an ongoing discrimination process based on the availability of these data but does not incorporate an inferencing scheme for reasoned hypothesis selection. The dierence in learning performance { in the light of our evaluation study in Section 3 { amounts to 8%, considering the dierence between LA - (plain terminological reasoning) and LA CB values (terminological metareasoning based on the quali cation calculus). Hence, our claim that we produce competitive results. Note that the requirement to provide learning facilities for large-scale text knowledge engineering also distinguishes our approach from the currently active eld of information extraction (IE) [2]. The IE task is de ned in terms of a xed set of a priori templates which have to be instantiated (i.e., lled with factual knowledge items) in the course of text analysis. In particular, no new templates have to be created. This step would correspond to the procedure we described in this contribution. As far as the eld of knowledge engineering from texts is concerned, i.e., text understanding and knowledge assimilation, our system represents a major achievement, since the problem has so far only been solved by either hand-coding the content of the textual input [14], or providing semi-automatic devices for text knowledge acquisition [16], or using simplistic keyword-based content analysis techniques [1].

5 Conclusion We have presented a concept acquisition methodology which is based on the incremental assignment and evaluation of the quality of linguistic and conceptual evidence for emerging concept hypotheses. The principles underlying the selection and ordering of quality labels are general, as are most conceptual quality labels. The concrete de nition of, e.g., linguistic quality labels, however, introduces a level of application-dependence. Nevertheless, as quality criteria are ubiquitous, one may easily envisage quality labels coming from sources other than linguistic and conceptual knowledge (e.g., a vision system may require quality labels which account for dierent degrees of signal distortion, 2D vs. 3D representations, etc. in order to interpret visual scenes in the course of learning new gestalts). No specialized learning algorithm is needed, since learning is a (meta)reasoning task carried out by the classi er of a terminological reasoning system. However, heuristic guidance for selecting between plausible hypotheses comes from the dierent quality criteria. Our experimental data indicate that given these heuristics we achieve a high degree of pruning of the search space for hypotheses in very early phases of the learning cycle. Acknowledgements. We would like to thank our colleagues in the CLIF group for fruitful discussions and instant support, in particular Joe Bush who polished the text as a native speaker. K. Schnattinger is supported by a grant from DFG (Ha 2097/3-1).

References 1. R. Agarwal and M. Tanniru. Knowledge extraction using content analysis. Knowledge Acquisition, 3(4):421{441, 1991. 2. D. Appelt, J. Hobbs, J. Bear, D. Israel, and M. Tyson. FASTUS: A nite-state processor for information extraction from real-world text. In IJCAI '93 { Proc. of the 13th Int'l. Joint Conf. on Arti cial Intelligence, pages 1172{1178, 1993. 3. F. Gomez and C. Segami. The recognition and classi cation of concepts in understanding scienti c texts. Journal of Experimental and Theoretical Arti cial Intelligence, 1:51{77, 1989. 4. U. Hahn, M. Klenner, and K. Schnattinger. Learning from texts: A terminological metareasoning perspective. In S. Wermter, E. Rilo, and G. Scheler, editors, Connectionist, Statistical and Symbolic Approaches to Learning for Natural Language Processing, pages 453{468. Berlin: Springer, 1996. 5. U. Hahn, M. Klenner, and K. Schnattinger. A quality-based terminological reasoning model for text knowledge acquisition. In N. Shadbolt, K. O'Hara, and G. Schreiber, editors, EKAW '96 { Proc. of the 9th European Knowledge Acquisition Workshop, pages 131{146. Berlin: Springer, 1996. 6. U. Hahn, S. Schacht, and N. Broker. Concurrent, object-oriented natural language parsing: The ParseTalk model. International Journal of Human-Computer Studies, 41(1/2):179{222, 1994. 7. P. Hastings. Implications of an automatic lexical acquisition system. In S. Wermter, E. Rilo, and G. Scheler, editors, Connectionist, Statistical and Symbolic Approaches to Learning in Natural Language Processing, pages 261{274. Berlin: Springer, 1996. 8. B. Nebel. Reasoning and Revision in Hybrid Representation Systems. Berlin: Springer, 1990. 9. L. Rau, P. Jacobs, and U. Zernik. Information extraction and text summarization using linguistic knowledge acquisition. Information Processing & Management, 25(4):419{428, 1989. 10. U. Reimer. Automatic acquisition of terminological knowledge from texts. In ECAI '90 { Proc. of the 9th European Conf. on Arti cial Intelligence, pages 547{ 549. London: Pitman, 1990. 11. K. Schnattinger and U. Hahn. A terminological quali cation calculus for preferential reasoning under uncertainty. In KI '96 { Proc. of the 20th Annual German Conf. on Arti cial Intelligence, pages 349{362. Berlin: Springer, 1996. 12. K. Schnattinger and U. Hahn. Intelligent text analysis for dynamically maintaining and updating domain knowledge bases. In IDA '97 { Proc. of the 2nd Int'l. Symposium on Intelligent Data Analysis. Berlin: Springer, 1997. 13. K. Schnattinger, U. Hahn, and M. Klenner. Terminological meta-reasoning by rei cation and multiple contexts. In EPIA '95 { Proc. of the 7th Portuguese Conf. on Arti cial Intelligence, pages 1{16. Berlin: Springer, 1995. 14. D. Skuce, S. Matwin, B. Tauzovich, F. Oppacher, and S. Szpakowicz. A logic-based knowledge source system for natural language documents. Data & Knowledge Engineering, 1(3):201{231, 1985. 15. S. Soderland, D. Fisher, J. Aseltine, and W. Lehnert. CRYSTAL: Inducing a conceptual dictionary. In IJCAI '95 { Proc. of the 14th Intl. Joint Conf. on Arti cial Intelligence, pages 1314{1319, 1995. 16. S. Szpakowicz. Semi-automatic acquisition of conceptual structure from technical texts. International Journal of Man-Machine Studies, 33:385{397, 1990.