Sabino Miranda-Jiménez, INFOTEC - Centro de Investigación e Innovación en .... For instance, Figure 1 stands for sentence Joe buys a necktie from Hal for $10 ...
Conceptual Graphs as Framework for Summarizing Short Texts Sabino Miranda-Jiménez, INFOTEC - Centro de Investigación e Innovación en Tecnologías de la Información y Comunicación, Cátedra Conacyt, México Alexander Gelbukh, Centro de Investigación en Computación, Instituto Politécnico Nacional, México Grigori Sidorov, Centro de Investigación en Computación, Instituto Politécnico Nacional, México
ABSTRACT In this paper, a conceptual graph-based framework for summarizing short texts is proposed. A semantic representation is implemented through conceptual graph structures that consist of concepts and conceptual relations that stand for texts. To summarize conceptual graphs, the most important nodes are selected using a set of operations: generalization, association, ranking, and pruning, which are described. The importance of nodes on weighted conceptual graphs is measured using a modified version of HITS algorithm. In addition, some heuristic rules are used to keep coherent structures based on information from WordNet (hierarchy of concepts) and VerbNet (semantic patterns of verbs). The experimental results show that this approach is effective in summarizing short texts. Keywords: Automatic summarization, conceptual graphs, graph-based ranking algorithms, HITS algorithm. INTRODUCTION Summarization technologies are essential in today‘s information society. In order to handle huge amount of information efficiently, users need to have short documents that stand for the essential information from one or more source documents, that is, their summaries. High-quality automatic text summarization is a challenging task that involves text analysis, text understanding, the use of domain knowledge, and language generation. There are several points of view that lead the automatic summarization. The main factors used for automatic summarization are (1) the kind of information source: text, images, video, or voice, (2) the number of documents to be summarized: single- or multi-document, (3) the resulting summary: extractive or abstractive, (4) the purpose: generic, user-oriented, query-focused, indicative or informative, and (5) the number of languages: monolingual or multilingual (Spärck Jones, 1999; Spärck Jones, 2007; Das & Martins, 2007; Nenkova & McKeown, 2011; Lloret & Palomar, 2012; Elhadad, Miranda-Jiménez, Steinberger, & Giannakopoulos, 2013; Torres-Moreno, 2014). In this research, the interest is in single-document text summarization for English. The resulting summaries are considered generic and abstractive at conceptual level.
The single-document summarization task was addressed in Document Understanding Conference (DUC1) for years 2001 and 2000, the main forum for evaluating text summarization systems, now is a track of Text Analysis Conference (TAC2). In both years, none of the systems outperformed the baseline, which consisted of the first 100 words of the original documents. Summaries produced by humans were significantly better than all the systems. DUC data used were newswire/paper documents; thus, the genre of documents affected the results because news documents have important ideas at the beginning of text. The single-document summarization task was not kept in later years of DUC because of the poor results in the competitions, but it still remains an open problem (Nenkova, 2005; Nenkova & McKeown, 2011). Multi-document summarization is motivated by information on Internet. Given the large amount of redundancy on documents, summarization is more useful if it can provide a brief description of a group of documents about the same topic or event. This approach is the main trend in TAC competition and researches. Also, multi-document multilingual summarization is gaining attention because same information can appear in several languages; MultiLing competition provides a collection of documents and evaluations for this sort of systems (Elhadad et al., 2013; Lei, Forascu, El-haj, & Giannakopoulos, 2013). According to the resulting summary, the extractive approach is very popular and wellknown. In this approach, a summary is made of excerpts from one or more documents; it is produced by concatenating sentences selected verbatim as they appear in the documents to be summarized. The limitations of this approach are well-known: low quality, lack of coherence, among others. Abstractive summaries are produced to convey the important information from the original document, and sentences can be reused, combined, or pruned from it (Barzilay & McKeown, 2005; Genest & Lapalme, 2012). This approach has not been widely explored because deep text analysis is required for understanding texts. Such a deep analysis is indispensable to improve the quality of summaries (Spärck Jones, 2007; Lloret & Palomar, 2012; Saggion & Poibeau, 2013). In this paper, we propose a framework for single-document abstractive summarization for English, based on conceptual graphs as the underlying text representation (Sowa, 1984). Our approach is based on a set of operations on conceptual graphs in order to simplify conceptual structures, namely, generalization, association, ranking, and pruning. Ranking is a key operation to identify the most important nodes according to a modified version of HITS algorithm (Kleinberg, 1999) on weighted conceptual graphs (MirandaJiménez, Gelbukh, & Sidorov, 2013). Pruning operation is supported by using heuristics rules based on semantic patterns from VerbNet (Kipper, Trang Dang, & Palmer, 2000), and WordNet (Fellbaum, 1998) is used as a hierarchy of concepts in order to generalize graphs. The summary at semantic level is the resulting structure after applying such operations. Text generation from conceptual graphs is beyond of the scope of this paper. We evaluate our approach using a set of conceptual graphs based on DUC-2003 data. The results show that our framework is adequate to summarize single-document short text. The rest of the paper is organized as follows. First, basic concepts of conceptual graphs 1 2
http://duc.nist.gov/ http://www.nist.gov/tac
are outlined. Second, the proposed framework and conceptual graph operations are described. Third, the experimental results are showed. Finally, the conclusions and future work are given. RELATED WORK We mentioned that extractive summaries are produced by selecting and extracting the important sentences or paragraphs from one or more documents. Generally, sentences selected are presented sequentially as they appear in the original document. Methods that implement a sentence-extraction paradigm decide if a sentence should be included or excluded in the summary. Thus, the resulting summary is usually incoherent because of techniques used. The approaches to sentence selection use statistical information or take into account linguistic or semantic information. Methods often use a linear model to determine the importance of text units (words, sentences, or paragraphs) according to different features such as position, frequency counts, co-occurrence, cue phrases (in conclusion, in summary, etc.), among others. This approach has been widely researched (Spärck Jones, 2007; Nenkova & McKeown, 2011; Lloret & Palomar, 2012; Saggion & Poibeau, 2013; TorresMoreno, 2014). The problem of generating a new cohesive and coherent text has received less attention. We can mention the most representative methods based on sentence compression, which reduce sentences to minimal grammatical form using probabilistic model (Knight & Marcu, 2000; Knight & Marcu, 2002; Cohn & Lapata, 2009), rhetorical relations, which determine importance of a sentence using nucleus and satellite relations, depending on how the relevant information is (Marcu, 2000; Molina, Torres-Moreno, da Cunha, SanJuan, & Sierra, 2012), sentence fusion (Barzilay, 2003; Barzilay & McKeown, 2005; Filippova & Strube, 2008), which rewrites sentences by fusing together portions of related sentences using syntactic information. Also, Abstraction Schemes have been designed to address a theme, which consist of Information Extraction rules, content selection heuristics, and generation patterns (Genest & Lapalme, 2012). This representation answer to specific information required, the summary is created from schemes using generation patterns. Another approach is Semantic Graphs that use the semantics of document (Leskovec, Grobelnik, & Milic-Frayling, 2004; Tsatsaronis, Varlamis, & Nørvåg, 2010). This method uses syntactic analysis to extract logical form triplets, subject–predicate–object. Each triplet is characterized by a set of linguistic, statistical, and graph attributes. A linear SVM (Support Vector Machine) classifier (Vapnik, 1998) is used to identify triplets in order to create the summary. In the case of graph-based methods, LexRank (Erkan & Radev, 2004) and TextRank (Mihalcea & Tarau, 2004) have been used for keyword extraction and extractive summarization. In these approaches, graphs are usually considered undirected and unweighted. The graph nodes are sentences, words, or other kind of textual units, and edges are defined by overlaps of the content between units, others consider syntactic relations as edges (Litvak & Last, 2008). In these approaches, well-known iterative algorithms are used
such as HITS (Kleinberg, 1999) or PageRank (Page, Brin, Motwani, & Winograd, 1999) to rank the nodes in order to select salient ones. The nodes selected represent the most important parts of the text, and the summary is created from salient nodes. The aforementioned approaches use rewriting techniques based on partial semantic or syntactic representations that involve some sematic patterns or semantic relations. Nevertheless, a fine-grained and expressive semantic representation for text summarization is starting to be used (Miranda-Jiménez et al., 2013), which take advantage of semantic roles (Jackendoff, 1972; Fillmore & Atkins, 1992) such as agent, patient, theme, etc, in the conceptual graph context. In the following sections, we present our approach based on a set of operations to manipulate conceptual graphs that stand for texts in order to summarize them. SUMMARIZATION FRAMEWORK BASED ON CONCEPTUAL GRAPHS The proposed framework is based on a semantic representation of texts. We use conceptual graphs as intermediate representation. In this context, we consider text summarization problem as graph simplification problem. To simplify graphs, some operations are performed on graphs in order to reduce them, ensuring coherent structures. First, we introduce some basic concepts about conceptual graph formalism and weighted conceptual graphs. Second, we describe our architecture of conceptual graph summarization focusing on conceptual graph operations. Conceptual Graphs Formalism Conceptual graphs (CGs) are structures for knowledge representation based on first-order logic. These structures are natural, simple, and fine-grained semantic representations to describe texts. A conceptual graph is a finite, connected, and bipartite graph. It has two kinds of nodes: conceptual relations (ovals) and concepts (rectangles) (Sowa, 1984; Sowa, 1999). A concept is connected to another related concept by a conceptual relation. Each relation must be linked to some other concept. For instance, Figure 1 stands for sentence Joe buys a necktie from Hal for $10 (Sowa, 1984). In this graph, the semantics of sentence is detailed by means of concepts and relations: who bought (AGNT), what is bought (OBJ), from whom it is bought (SRC), and how it is bought (INST). Figure 1. Conceptual Graph
Another important element of CGs is concept types. Concept types represent classes of entities (Person, Money), attributes, states and events. It is also called concept type hierarchy that represents an ISA (is a) hierarchy, and it is used to map concepts into the
hierarchy for inference purposes (Sowa, 1984; Chein & Mugnier, 2009). For example, in Figure 1, Money:@$10 denotes the concept type Money, and its referent @$10 is an instance of Money that its measure is 10. CG framework allows graph-based operations for reasoning. A number of operations can be performed to create, manipulate and retrieve large sets of conceptual graphs such as restriction, simplification, unification (join), indexing (Sowa, 1984; Chein & Mugnier, 2009), and graph matching (projection) (Montes-y-Gómez, Gelbukh, & López-López, 2000; Montes-y-Gómez M. , Gelbukh, López-López, & Baeza-Yates, 2001). Weighted Conceptual Graphs The notion of weighted conceptual graph (WCG) was introduced by Miranda-Jiménez et al. (2013). The idea behind this kind of CG is the interest in flows on graphs that are called semantic flows. Figure 2 depicts the WCG for sentence The cat Yojo is chasing a brown mouse. Numbers on edges stand for more interest or less interest on flows throughout conceptual relations. Basically, a semantic flow is the accumulative weight of a node, and it is transferred to other nodes increasing or decreasing the value, depending on the type of conceptual relations that it passes. Thus, if a semantic flow throughout agent, location, attribute, or other thematic roles is important, the edge weight that passes through it should have a high value. For example, if we are interested in the flows that pass through agent relations (AGNT), the incoming and outgoing edges for these conceptual relations should have high value such as in Figure 2. WCG are useful for ranking operation to determine salient nodes. Figure 2. Weighted Conceptual Graphs
Architecture of CG Summarization Our model is based on a semantic representation of texts using WCGs as intermediate representation. We take advantage of simplicity, expressiveness, and detailed semantics of such graphs. In this context, text summarization task is similar to selecting salient nodes and reducing the conceptual structures selected, keeping coherence among structures. The resulting structures represent the summary at conceptual level. Figure 3 shows the architecture of our approach. First, a text preprocessing is performed in order to obtain syntactic information from text in order to build semi-automatically CGs according to some heuristic rules. During the transformation stage of conceptual graphs other semantic and syntactic information is added from external resources. In the synthesis stage, conceptual graphs are reduced by applying a set of CG operations. The summary is
generated from resulting conceptual structures. In the following subsections, we detail each stage of the architecture. Text generation is beyond of the scope of this research. Figure 3. Framework for CG Summarization
Linguistic Resources Our model uses a conceptual graph formalism, which requires additional semantic and syntactic information, namely, a hierarchy of concepts for generalization purposes and verbal patterns to keep structural coherence among conceptual structures. We apply our framework to English language as case study because this language has two key resources to support our approach: WordNet and VerbNet. WordNet (Fellbaum, 1998) is a lexical database for English language. It is organized in synsets (set of synonyms); each synset is connected to other synsets by semantic relations such as hyperonymy / hyponymy (class / subclass), which defined a hierarchy of concepts. For example, atmospheric phenomenon / storm, and residence / home (class / subclass). VerbNet (Kipper et al, 2000) is a lexicon for English verbs that gathers Levin‘s verbal patterns (Levin, 1993). VerbNet combines syntactic and semantic information. For example, the class chase has a basic pattern ‗NP V NP‘ (Noun Phrase, Verb, and Noun Phrase) and is labeled as transitive verb. Verb patterns are a tool to rule the coherent sentences and are used to keep coherent structures of graphs when nodes are removed. Collection of test documents We used texts from data collection of DUC-2003 competition (DUC, 2003) because in that year very short text summarization was considered, at headline level. In addition, there are short documents (~100 words) explicitly created for the competition. We consider the summary created by hand as original source, and headlines as document summaries. We selected short text to simplify the creation of conceptual graphs because each conceptual graph should be examined by hand, corrected the wrong conceptual relations assigned, established verbal patterns, and connected each concept to its concept type from
WordNet. To ensure the correct creation of conceptual graphs, this work must be supervised manually. The CG collection consists of 30 documents from newswire related to natural disasters and events. The texts selected are short between 50 and 100 words. Building Conceptual Graphs The creation of a conceptual graph from text is not direct. It requires an additional process to discover relationships among text units. Some approaches have been proposed for the automatic generation of conceptual graphs (Hensman & Dunnion, 2004; Hensman, 2005; Ordoñez-Salinas & Gelbukh , 2010), but tools are not available. Thus, we created semiautomatically a collection of conceptual graphs based on collection of news from DUC in order to prove our ideas. We use simple conceptual graphs (without negations, contexts, and such called situations described in (Sowa, 1984)) to simplified our task. Preprocessing. To identify syntactic relations between units, we use Stanford parser (de Marneffe, MacCartney, & Manning, 2006) to obtain dependency trees (Mel‘cuk, 1988) to build conceptual graphs. Afterwards, heuristic rules are applied to build the graph according to the relation extracted by the parser. Building Conceptual Graphs. After text preprocessing, conceptual graphs are built by means of a set of transformation rules. For instance, nsubj relation (nominal subject) and agent become AGNT (agent), amod relation (adjectival modifier) become ATTR (attribute), dobj (direct object) become in THME (theme), etc. The set of grammatical relation produced by the parser is described in Stanford parser manual (de Marneffe & Manning, 2008). The relations assigned that were wrong or unknown because of transformation rule are fixed by hand. Figure 4 shows the conceptual graph for sentence Intel presented the new microprocessor. For this example, the Stanford parser creates dependency relations such as nsubj(presented, Intel), amod(microprocessor, new), and dobj(presented, microprocessor). The triplet consists of relation name, governing word, and dependent word. The nsubj relation creates nodes: Intel, AGNT (agent) and present. The dobj relation creates nodes such as THME (theme) and Device: microprocessor. Syntactic features of the concept are coded in the node, for instance, the verb presented (label VBD produced by parser means verb, third person singular, past tense). Only canonical word forms are shown as concept node labels. Figure 4. Creation of CG based on Stanford typed dependencies
After, the concept type is added (hyperonym) for each concept from WordNet. For example, in Figure 4, Company is assigned to Intel concept, and Device to concept microprocessor, see Sowa(1984) for syntactic details of representation. Finally, a verb class is manually assigned that defines verbal pattern for nodes that stand for verbal concepts. For instance, fulfilling class from VerbNet contains the verbal concept present. This class defines the basic verbal pattern (transitive verb) as “NP V NP” (Noun Phrase / Verb / Noun Phrase as theme), it means that the graph must have a direct object as the theme. Simple CGs are considered in order to simplify the problem, and we not follow rigorously all suggestions to depict CGs (Sowa, 1984; Sowa, 1999; Chein & Mugnier, 2009). For instance, in Figure 4 the conceptual relation (PAST) is not included because it is coded in the concept node labeled as present tense. We follow similar procedures with other relation markers although some canonical sematic representations of CGs are lost. Figure 5 is an example of CGs that we used in the experiments. It stands for the following text. “A local controversy exploded into an international, vote-buying bribery scandal. The world was rocked by evidence of gifts, favors, humanitarian aid and scholarship payments to International Olympic Committee site selection members by officials of the winning site, Salt Lake City. They amounted to more than one million dollars. Further, an IOC official cited alleged irregularities in the selection of at least three other Olympic cities over the past ten years. The Salt Lake scandal led to the ouster of six IOC executive board members and the criminal indictment of two Salt Lake bid officials by the U.S. Justice Department.” In the graph, the dotted line represents coreference of concepts because of the anaphora detected in text by Stanford parser, other concepts with associations are labeled with (*) symbol and the referent concept is labeled with (#) symbol. In our approach, content words are considered as conceptual nodes (that is, all words except for stop words such as prepositions, conjunctions, among others), and semantic roles (Jackendoff, 1972) are considered as conceptual relations: agent, causer, instrument, experiencer, patient, location, time, object, source, and goal, as well as some other relations, such as attribute, quantity, measure, etc.—ca. 30 relations used in (Sowa, 1984).
Figure 5. Example of news as conceptual graph
General Process to Summarize Conceptual Graphs As we mentioned, the framework proposed for text summarization consist of two main stages: creation of CGs and synthesis of CGs by means of several CG operations. The general procedure in order to identify and select the most important structures in CGs is carried out in the following steps: 1. 2. 3. 4. 5. 6. 7. 8. 9.
Identify all generalizations by means of generalization operation Identify all associations by means of association operation Set hub and authority scores associated to each node to value of 1. Apply the operation ranking, equations (1) and (2). Normalize the Authority and Hub values by Euclidian norm. Repeat steps from 4–6 up to converge or reach N iterations. Sort nodes by Authority values in descending order. Expand the connected concepts for each selected conceptual relation. Expand the associated nodes for each selected concept (verb concept) according to its semantic pattern. 10. Select the top concepts according to compression rate in order to prune the graphs. In steps 1 and 2, relationships between nodes of different graphs are identified and associated to improve the ranking operation. Steps 3–7 compute the HITS scores based on equations (1) and (2) in order to determine salient nodes. Step 8 applies rules to expand concepts that connect a conceptual relation, if it is selected. For instance, the relation selected, AGNT(lead,scandal) (see Table I), is expanded into two concepts lead and scandal, the concepts selected stand for the resulting summary. Step 9 applies verb pattern rules in order to keep coherent structures. For example, the verbal pattern for the lead concept (Figure 2) is identified in the VerbNet class as force, its pattern is NP V NP (Noun Phrase / Verb / Noun Phrase), and the verb is Basic Transitive. The role for the first NP is agent, and the second NP is Theme. Both of them are required for the concept lead because it is defined as transitive verb. Thus, the node for the agent and the node for the theme must be included in the summary if concept lead is selected. Step 10 applies the pruning operation by means of compression rate provided by the user. This operation selects nodes without duplicating them according to their position in the final table, and considering the compression rate. The nodes selected are the summary at conceptual level (see Table II). Conceptual Graph Operations We define several operations for synthesis stage: generalization, association, ranking, and pruning. Also, we use a comparison operation between conceptual graphs (Montes-yGómez et al., 2001). a) Comparison This operation consists of the comparison of two CGs. First, the two CGs are matched and their common elements are identified (concepts and relations). Second, their
similarity measure is computed as a relative size of their common elements. This measure is a value between 0 and 1 (0 stands for no similarity; 1 stands for maximal similarity). A hierarchy of concepts is used to determine the similarity at different levels of generalization. For example, in Figure 6, to determine the similarity between concepts crocodile and bird the hierarchy (a) in Figure 7 is used. That is, Animal(3,3): 0.66 means that concept Animal is the minimal concept for crocodile and bird. Matching of two conceptual graphs (G1 and G2) allows finding all their common elements, that is, all their common generalizations. Since the projection is not necessary one-to-one and unique, some of these common generalizations may express redundant or duplicate information. In order to define a precise description of similarity between two conceptual graphs, it is necessary to identify the sets of common generalizations that form a compatible maximal common generalization. Each set is called overlap. A set of common generalizations O = {g1, g2,… gn}is called compatible if and only if there exist a projection maps {π1, π2, … πn } such that the corresponding projections in G1 and G2 do not intersect. ⋂
⋂
The set of common generalizations O = {g1, g2,… gn}of conceptual graphs (G1 and G2) is maximal if and only if there does not exist any common generalization g of G1 and G2 such that either of the conditions holds O’ = {g1, g2,… gs }is compatible, , , and O’ = {g1, …, gi-1, g } is compatible A set O = {g1, g2,… gn} of common generalizations of two CGs is called overlap if and only if it is compatible and maximal. Let us remember that u ≤ v, v is a supertype of v, and u is a subtype of v. Different overlaps may denote different and independent ways of visualizing and interpreting their similarity. It is well-known that matching CGs is an NP-complete problem (Myaegn & LópezLópez, 1992; Marie, 1995). However, the algorithm presented by Montes-y-Gómez et al. (2001) does not imply any serious limitations for its practical application. In general terms, the algorithm to find all overlaps between two conceptual graphs consists of two stages. At the first stage, all correspondences (i.e., common generalizations for each concept and relations from G1 and G2) are found, that is, a kind of product graph is constructed (Myaegn & López-López, 1992). At the second stage, all overlaps are found, i.e., all maximal sets of compatible elements are detected. Initially, each concept of the product graph is a possible overlap. At each subsequent step, it is started with the overlaps found in previous step. These overlaps are used as the seed set for generating new large overlaps. At the end of the step, the overlaps of the previous step that were used to construct the new overlaps are deleted because they are not maximal overlaps and the new overlaps are the seed for the next step. This process continues until no new large enough overlaps are found. Finally, the relations of the
product graph are inserted into the corresponding overlaps. For further details of the algorithm, see Montes-y-Gómez et al. (2001). Similarity Measure The similarity measure used to compare two conceptual graphs G1 and G2 is a combination of two values: conceptual similarity (sc) and relational similarity (sr). The similarity measure is defined as follows (Montes-y-Gómez et al., 2001). S= where coefficients reflect user-specified balance ( ). Coefficient is related to the importance of common concepts and coefficient is related to the importance of connections of these common concepts. The conceptual similarity depends on the common concepts of G1 and G2 that indicates how similar their concepts (entities, actions, attributes, etc.) mentioned are. The conceptual similarity is defined as follows. (∑
) (∑
∑
)
⋃
where ⋃ is the union of all graphs in their overlaps O; the function w(c) gives the relative importance of the concept c (entities, actions, attributes, etc.). The function represents the level of generalization of the common concept ⋃ relative to the original concepts . The
function is calculated as follows: value
condition
1
if
(
)
(
)
(
)
(
)
depth/(depth+1)
if
(
)
(
)
(
)
(
)
if
(
)
(
)
2
where depth indicates the number of levels of ISA hierarchy, di indicates the distance (number of nodes) from the type i to the root of hierarchy. Let us remember that the function type(c) and referent(c) return the type and referent of the concept c. The relational similarity indicates how similar the neighbors of the overlap in both original graphs are. An immediate neighbor of the overlap O in a conceptual graph G1 is the set of all relations connected to the common concepts in the graph. (
⋃
), where
{r | r is connected to c in G}
The relational similarity is calculated as follows. (∑
) ( ∑
∑
)
where indicates the relative importance of the conceptual relation r in the conceptual graph G. This value is calculated as follows. ∑
|
|, where
{c | c is connected to r in G}
b) Generalization Generalization combines two CGs according to theirs common elements using the similarity measures defined above. For example, the following graphs could be read as G1: Peter buys a crocodile. G2: Mary buys a bird. Figure 6. Generalization
Figure 7. Fragments of a hierarchy
A comparison between CGs is performed; afterwards, it is determined the minimal common concepts in order to combine the common concepts. According to the hierarchy (a) in Figure 7 for crocodile and bird, Animal is the minimal common concept. Also, Person is the common concept for Peter and Mary concepts. Some heuristic rules are applied in order to join the concepts associated. For instance, adding the number of entities in G3. Thus, G3 is the resulting graph after combining the two other graphs. G3 could be read as Two persons buy two animals.
c) Association This operation merges two related concepts from two CGs. Our assumption is that a source text is coherent and cohesive, where sentences probably refer to concepts that were previously mentioned or other related concepts (Halliday & Hasan, 1976). This operation is needed to support and improve ranking process. In Figure 8, the two CGs could be read as G4: Typhoon Babs weakened into severe storm. G5: Storm killed at least 156 people in Philippines. Figure 8. Association
In the above graphs, there are two overlaps of related concepts: the first association identified by (1,1), Typhooon: Babs (concept 1, G4) and atmospheric_phenomenon: storm (concept 1, G5), and the other association identified by (3,1), atmospheric_phenomenon: storm (concept 3, G4) and atmospheric_phenomenon: storm (concept 1, G5), which has the maximal similarity, using the hierarchy (b) in Figure 7. Both associations are valid, but we use the association with maximal similarity to set the association between nodes. Thus, we consider as one node both concepts concept 3 from G4 and concept 1 from G5 (dotted line). d) Ranking Ranking operation selects the most important nodes on graphs. We use HITS method to rank nodes. HITS is an iterative algorithm that takes into account both in-degree and out-degree of nodes for ranking. The algorithm makes a distinction between authorities (nodes with a large number of incoming links) and hubs (nodes with a large number of outgoing links). For each node, HITS produces two sets of scores: AUTHority and HUB. We use the authority score (means that a node is good as information source) in order to choose the nodes that will take part in the summary. We used a modified version of HITS algorithm.
∑
∑ Equations (1) and (2) are used to compute authorities and hubs scores, where I is the set of incoming links for node Vi; O is the set of outgoing links for node Vi; Wki is the edge weight stands for semantic flow; and PREF is the node preference, i.e., the degree of interest in the specific topic. Equations (1) and (2) are computed iteratively for each node until converge or a predefined number of iterations. Mihalcea y Tarau (2005) use 20 to 30 iterations in their experiments, other researches use only one iteration (Litvak & Last, 2008). We identify that 15 iterations are enough for our data; more iterations not improve the results. e) Pruning In order to reduce graphs, the pruning operation is applied. It takes into account the ranking scores, verbal pattern to remove nodes, and compression rate to set how many nodes should be included into the resulting summary. This operation selects the nodes according to its AUTH score and they must be within the compression rate. Final concepts are selected from larger conceptual graphs, that is, we consider a whole CG as the original structure added to other CGs throughout the associations related to them. Second, if it is needed to cover other concepts for the summary the remaining CGs are used. According to Hovy (2005) a summary is useful if it is between 15% and 35% of length of original source. We use a 20% of compression rate because most summaries that we used to compare our approach are in this percentage of compression of the original source. 4. Experimental results We carried out our experiments on the collection of news articles provided by the DUC2003 competition (DUC, 2003). We selected news with length from 50 to 100 words. For each article, there are 3 summaries on average made by humans. We created three groups of documents from DUC: 3-senteces (group I), 4-sentences (group II), and (4+)-sentences (group III) length. Each group consists of 10 documents represented as conceptual graphs. We set the compression rate for pruning operation to 20% of concepts from the original document. The semantic flow for agent relations were set to value 2 for incoming and outgoing flows, other semantic flows were set to 1. We consider that agents are more important in news domain, and values were determined heuristically. For the similarity measure, the coefficients a and b were set to 0.5, and all weights for concepts were set to value 1.
We evaluate our method by comparing with a baseline. The baseline method consists of selecting the first concepts from the beginning at the first paragraphs up to the established compression rate (except for stop words). Also, standard metrics such as precision and recall were used to evaluate our method. Precision is the fraction of concepts chosen by the method that were correct. Recall is the fraction of concepts chosen by the human that were also correctly identified by the method. F-measure is the harmonic mean of precision and recall. |
| |
|
|
| |
|
Table I shows an example of the selected nodes by ranking method including conceptual relations for the CG in Figure 5. Also, expansions of conceptual relations are shown such as agent relations AGNT(lead, scandal). The table is sorted by Authority score in descending order. A concept labeled with (req) means that the concept is required, that is, the concept was included because of this concept is needed to complete the structural coherence, in this case, scandal and ouster concepts are necessary. The concept lead is a verbal concept and has an associated verbal pattern NP-V-NP (transitive verb), where the concepts match in the following way (scandal / NP)–(lead / V)–(ouster/ NP) (req). The last NP is required to complete the semantics of the structure, thus, this concept is included to be one of the eligible concepts. Table II shows the concepts selected by the method that take part in the summary. Also, conceptual relations that connect the concepts selected take part in the summary. In fact, other concepts are expanded because of conceptual relations selected that they already were in the summary. Finally, the concepts selected in Table II represent the summay at conceptual level, according to the Figure 5. For instance, the resulting selection could be read as follows. ―The scandal led to ouster and indictment. An official cited irregularities. The world was rocked by evidence.‖ Table III shows the overall evaluation of our approach. On average, the method outperforms baseline by 11% of precision: group I (3 sentences) 7%, group II (4 sentences) 15 %, and group III (more than 4 sentences) 10%. In Group I, data indicate that our method for extremely short text is performed as equal as the baseline that is much easier to implement. However, data from group II and group III, we infer that our method is better for identifying salient concepts at paragraph level (67% on average). We can deduce that the method takes advantage of features, at paragraph level,
that is, the text is cohesive and well structured. These features are well represented in the graphs and exploited properly by our method. Also, we observe that the baseline improves when sentences are added, but our method is still above the baseline. We expect that the baseline improves because there are works that show that the first and last lines are good indicators to identify important information (Edmundson, 1969; Hovy & Chin-Yew, 1999; Luhn, 1958). In addition, the DUC competition reported for 2001 and 2002 years that no singledocument summarization systems outperformed the baseline (100 first words of the document) (Nenkova, 2005). Although we did not use the same collection because of practical reasons (short texts), our method uses all the net and outperforms a similar short baseline. Table I. Concepts and conceptual relations selected by ranking method with expansion of conceptual relations Node
Expansion of relations
AGNT(lead–scandal) AGNT(cite–Official:@1) Member:{*} scandal AGNT(rock–Evidence)
AUTH
INST(led–Organization:Department_Justice) RSLT{lead–indictment)
scandal / lead / ouster (req) Official:@1 / cite / irregularity:{*} (req) — — world / rock / Evidence scandal / lead / Organization:Department_Justice scandal* / lead* / indictment
RSLT(lead–ouster)
Scandal* / lead* / ouster*
0.4836 0.4836 0.3021 0.3021 0.2770
0.0746 0.0746 0.0002 0.0002 0.0368
0.1612 0.1612 0.1612
3.292E-06 3.292E-06 3.292E-06
Table II. Final concepts selected by ranking method Node
AUTH
scandal lead ouster Official:@1 cite irregularity:{*} world rock evidence indictment
0.4836 0.4836 0.4836 0.4836 0.4836 0.4836 0.2770 0.2770 0.2770 0.1612
HUB 0.0746 0.0746 0.0746 0.0746 0.0746 0.0746 0.0368 0.0368 0.0368 3.292E-06
Table III. Evaluation Precision
Recall
HUB
F-Measure
Baseline
Method
Baseline
Method
Baseline
Method
Group I
0.45
0.52
0.44
0.67
0.45
0.58
Group II
0.53
0.68
0.53
0.74
0.53
0.71
Group III
0.56
0.66
0.56
0.69
0.56
0.67
Average
0.51
0.62
0.51
0.70
0.51
0.65
CONCLUSIONS and FUTURE WORK We proposed a framework for single-document summarization based on conceptual graphs as underlying semantic representation. The approach is based on a set of operations, which are applied on weighted conceptual graphs. It combines text content and semantic roles based on the connecting structure of concepts and semantic ranking. In addition, two external linguistic resources are used, namely, WordNet and VerbNet. WordNet is used as a hierarchy of concepts to generalize and combine concept nodes, and VerbNet is useful for keeping coherent structure by means of semantic verbal patterns. These external resources are a limitation for applying our approach to other languages. Furthermore, we showed that weighted conceptual graphs provide us a flexible schema to focus on certain semantic flows or topics by means of weights and preferences. Our approach was evaluated with DUC-2003 data. The results show that our approach works well in short text that outperforms the baseline on average 11%. None of the systems tested in DUC 2001-2002 competitions outperformed the baseline similar that we used in the experiments, and single-document summarization task still remains an open problem. Finally, in order to extent our analysis and compare the results with other systems of DUC competition, more short texts as conceptual graphs are indispensable and large conceptual graphs based on DUC data are required. The main constraint is to build highquality conceptual graphs automatically. Acknowledgments. This work was done under partial support of the Mexican Government (SNI, Cátedras Conacyt), Instituto Politécnico Nacional, México (projects SIP 20144274 and 20144534, COFAA, PIFI), and FP7-PEOPLE-2010-IRSES: Web Information Quality Evaluation Initiative (WIQ-EI) European Commission project 269180.
REFERENCES Barzilay, R. (2003). Information Fusion for MultiDocument Summarization: Paraphrasing and Generation. Thesis (Ph.D.), Columbia University, New York, USA. Barzilay, R., & McKeown, K. R. (2005). Sentence Fusion for Multidocument News Summarization. Computational Linguistics, 31(3), 297-328. Chein, M., & Mugnier, M.-L. (2009). Graph-based Knowledge Representation: Computational Foundations of Conceptual Graphs. London: Springer-Verlag. Cohn, T., & Lapata, M. (2009). Sentence Compression as Tree Transduction. Journal of Atificial Intelligence Research, 34, 637–674. Das, D., & Martins, A. F. (2007). A Survey on Automatic Text Summarization. Literature Survey for the Language and Statistics II course at CMU, 4, 192-195. de Marneffe, M.-C., & Manning, C. D. (2008, 11). Stanford Parser Manual. Retrieved 01 15, 2013, from Stanford Parser: http://nlp.stanford.edu/software/dependencies_manual.pdf de Marneffe, M.-C., MacCartney, B., & Manning, C. D. (2006). Generating Typed Dependency Parses from Phrase Structure Parses. In Proceedings of the 5th
International Conference on Language Resources and Evaluation (LREC 2006), (pp. 449-454). Genoa, Italy. DUC. (2003). Document Understanding Conference. Retrieved 09 15, 2010, from http://duc.nist.gov/pubs.html#2003 Edmundson, H. P. (1969). New Methods in Automatic Extracting. Journal of the Association for Computing, 16(2), 264-285. Elhadad, M., Miranda-Jiménez, S., Steinberger, J., & Giannakopoulos, G. (2013). Multidocument multilingual summarization corpus preparation, Part 2: Czech, Hebrew and Spanish. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. Sofia, Bulgaria. Erkan, G., & Radev, D. (2004). LexRank: Graph-based Lexical Centrality as Salience in Text Summarization. Journal of Artificial Intelligence Research, 22(1), 457-479. Fellbaum, C. (Ed.). (1998). WordNet: An Electronic Lexical Database. Cambridge MA: MIT Press. Filippova, K., & Strube, M. (2008). Sentence fusion via dependency graph compression. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, (pp. 177-185). Waikiki, Honolulu, Hawaii. Fillmore, C., & Atkins, B. T. (1992). Towards a Frame-Based Lexicon: the Case of RISK. In A. Lehrer, & E. Kittay(Eds.), Frames and Fields (pp. 75-102). Erlbaum Publ. Genest, P.-E., & Lapalme, G. (2012). Fully Abstractive Approach to Guided Summarization. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, (pp. 354–358). Jeju, Korea. Halliday, M., & Hasan, R. (1976). Cohesion in English. Longman. Hensman, S. (2005). Constructing conceptual graphs using linguistic resources. In Proceedings of the 4th WSEAS International Conference on Telecommunications and Informatics, (pp. 1-6). Prague, Czech Republic. Hensman, S., & Dunnion, J. (2004). Automatically Building Conceptual Graphs Using VerbNet and WordNet. In Proceedings of the 3rd International Symposium on Information and Communication Technologies, (pp. 115-120). Las Vegas, USA. Hovy, E., & Chin-Yew, L. (1999). Automating Text Summarization in SUMMARIST. In I. Mani, & M. T. Maybury (Eds.), Advances in Automatic Text Summarization (pp. 81-94). Cambridge MA: MIT Press. Jackendoff, R. (1972). Semantic Interpretation in Generative Grammar. Cambridge : MIT Press. Kipper, K., Trang Dang, H., & Palmer, M. (2000). Class-Based Construction of a Verb Lexicon. In Proceedings of the Seventeenth National Conference on Artificial Intelligence (AAAI-2000), (pp. 691 – 696). Austin, Texas. Kleinberg, J. (1999). Authoritative Sources in a Hyperlinked Environment. Journal of the ACM, 46(5), 604–632. Knight, K., & Marcu, D. (2000). Statistics-Based Summarization - Step One: Sentence Compression. In Proceedings of the 17th National Conference of the American Association for Artificial Intelligence AAAI'2000, (pp. 703-710). Austin, Texas. Knight, K., & Marcu, D. (2002). Summarization beyond sentence extraction: A Probabilistic Approach to Sentence Compression. Artificial Intelligence, 139(1), 91–107.
Lei, L., Forascu, C., El-haj, M., & Giannakopoulos, G. (2013). Multi-document multilingual summarization corpus preparation, Part 1: Arabic, English, Greek, Chinese, Romanian. In Proceedings of the MultiLing 2013 Workshop in ACL 2013, (pp. 1-12). Sofia, Bulgaria. Leskovec, J., Grobelnik, M., & Milic-Frayling, N. (2004). Learning Semantic Graph Mapping for Document Summarization. In Proceedings of the ECML/PKDD-2004 Workshop on Knowledge Discovery and Ontologies. Levin, B. (1993). English Verb Classes And Alternations: A Preliminary Investigation. The University of Chicago Press. Litvak, M., & Last, M. (2008). Graph-based keyword extraction for single-document summarization. In Proceedings of the Workshop on Multi-source Multilingual Information Extraction and Summarization. Manchester, United Kingdom. Lloret, E., & Palomar, M. (2012). Text Summarisation in Progress: A Literature Review. Artificial Intelligence Review, 37(1), 1-41. Luhn, H. (1958). The Automatic Creation of Literature Abstracts. IBM Journal of Research and Development, 2(2), 159-165. Marcu, D. (2000). The Theory and Practice of Discourse and Summarization. Cambridge MA: MIT Press. Marie, M. (1995). On generalization / specialization on conceptual graphs. Journal of Experimental and Theoretical Artificial Intelligence, 7, 325-344. Mel‘cuk, I. (1988). Dependency Syntax: Theory and Practice. Albany: State University Press of New York. Mihalcea, R., & Tarau, P. (2004). TextRank: Bringing Order into Texts. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2004), (pp. 404-411). Barcelona, Spain. Miranda-Jiménez, S., Gelbukh, A., & Sidorov, G. (2013). Summarizing Conceptual Graphs for Automatic Summarization Task. In Proceedings of the Conceptual Structures for STEM Research and Education, ICCS 2013, (pp. 245-253). Mumbai, India. Molina, A., Torres-Moreno, J. M., da Cunha, I., SanJuan, E., & Sierra, G. (2012). Sentence Compression in Spanish driven by Discourse Segmentation and Language Models. Cornell University ArXiv :1212.3493. Computation and Language (cs.CL), Information Retrieval (cs.IR) vol.1212. Montes-y-Gómez, M., Gelbukh, A. F., López-López, A., & Baeza-Yates, R. A. (2001). Flexible Comparison of Conceptual Graphs. In Proceedings of the 12th International Conference and Workshop on Database and Expert Systems Applications, (pp. 102-111). Munich, Germany. Montes-y-Gómez, M., Gelbukh, A., & López-López, A. (2000). Comparison of Conceptual Graphs. In Proceedings of the Mexican International Conference on Artificial Intelligence, (pp. 548-556). Acapulco, Mexico. Myaegn, S. H., & López-López, A. (1992). Conceptual Graph Matching: a Flexible Algorithm and Experiments. In Proceedings of the Symposium on Document Analysis and Information Retrieval. Las Vegas, Nevada. Nenkova, A. (2005). Automatic Text Summarization of Newswire: Lessons Learned from the Document Understanding Conference. In Proceedings of the 20th national conference on Artificial intelligence (pp. 1436-1441). Pittsburgh, Pennsylvania.
Nenkova, A., & McKeown, K. (2011). Automatic Summarization. Foundations and Trends in Information Retrieval, 103-233. Ordoñez-Salinas, S., & Gelbukh , A. F. (2010). Generación de Grafos Conceptuales. In M. González Mendoza, & M. Herrera Alcántara (Eds.), Avances en sistemas inteligentes en México. Sociedad Mexicana de Inteligencia Artificial, (pp. 139– 150). Mexico city , Mexico. Page, L., Brin, S., Motwani, R., & Winograd, T. (1999). The PageRank Citation Ranking: Bringing Order to the Web. Technical report, Stanford University. Stanford. Saggion, H., & Poibeau, T. (2013). Automatic Text Summarization: Past, Present and Future. In Poibeau, T., Saggion, H., Piskorski, J., Yangarber, R. (Eds.), Multisource, Multilingual Information Extraction and Summarization Theory and Applications of Natural Language Processing (pp. 3-21). Springer-Verlag Berlin Heidelberg. Sowa, J. F. (1984). Conceptual Structures: Information Processing in Mind and Machine. Reading, MA: Addison-Wesley. Sowa, J. F. (1984). Conceptual Structures: Information Processing in Mind and Machine. Reading, MA: Addison-Wesley. Sowa, J. F. (1999). Knowledge Representation: Logical, Philosophical, and Computational Foundations. Pacific Grove, CA: Brooks Cole Publishing Co. Spärck Jones, K. (1999). Automatic Summarising: Factors and Directions. In I. Mani, & M. T. Maybury (Eds.), Advances in Automatic Text Summarization (pp. 1-12). Cambridge MA: MIT Press. Spärck Jones, K. (2007). Automatic summarising: The state of the art. Information Processing & Management, 43(6), 1449-1481. Torres-Moreno, J. M. (2014). Automatic Text Summarization. London : Hoboken, NJ: John Wiley & Sons. Tsatsaronis, G., Varlamis, I., & Nørvåg, K. (2010). SemanticRank: ranking keywords and sentences using semantic graphs. In Proceedings of the 23rd International Conference on Computational Linguistics, (pp. 1074-1082). Beijing, China. Vapnik, V. (1998). Statistical learning theory. Nueva York, USA: John Wiley & Sons.
