Using Design Patterns to Abstract a Software Architecture for Natural ...

Download PDF
33 downloads 561 Views 115KB Size Report
Comment
of his problem. As a result, a generic approach to NLG application development must ... modules into the output for the application being developed. Finally ...
Using Design Patterns to Abstract a Software Architecture for Natural Language Generation Raquel Hervás and Pablo Gervás Dep. Sistemas Informáticos y Programación Universidad Complutense de Madrid, Spain [email protected]

Abstract. There is no consensus about the architecture a Natural Language Generation (NLG) system must follow. Our proposal looks for a reusable software architecture for NLG systems by applying design patterns to abstract the main design decisions involved in the construction of a system of this kind. Its applicability is being tested through the development of a particular instantiation of a simple NLG application.

1 The Problem of System Architectures in Natural Language Generation There are many ways of organizing a Natural Language Generation (NLG) system, and their merits are still a matter of discussion [5,9]. We can find many architectures with big differences in relation to division in modules and connection topology between them. In [5], several of these architectures are discussed in detail. Each one of them is shown to have merits and disadvantages. The main conclusion of this survey is that the prospective developer of a NLG application needs to consider a wide range of architectural solutions, because any one of them may be relevant to particular aspects of his problem. As a result, a generic approach to NLG application development must allow easy combination and interconnection of different architectural solutions. According to the division into modules, one extreme is an integrated architecture [7], where the system is a single module in which knowledge at all levels acts together. The other extreme is an architecture with separated modules for each of the common tasks in NLG [1,2]: content determination, discourse planning, sentence aggregation, lexicalization, referring expression generation and linguistic realization. In between the extremes, we have the division Conceptualizer-Formulator [8], for example. The Conceptualizer is a nonlinguistic module which composes the content of a discourse by selecting and organizing information. The Formulator casts a message into a linguistic form by using the resources of a particular language. Taking into account the flow of control information, there are several possibilities: from a pipeline architecture with a one-way information flow [9], where modules are maximally independent, to a blackboard architecture [3], where modules provide

information to a common storage space (the blackboard) with no concern about which other modules use it. From the point of view of data structures, NLG applications have the additional problem of being based on specific linguistic theories. The choice of linguistic theory for a particular implementation imposes a number of constraints on the general architecture. These constraints need to be combined appropriately with these arising from the architectural choices described above. The RAGS project [1,2] reviewed a large number of existing systems and proposed a general solution for NLG architectures through a set of standard data structures and a sketch of the type of modules that might operate over them, allowing many varied instantiations according to specific linguistic theories. From the point of view of application development, the proposal of the RAGS project is limited to a set of interfaces between stages that developers should comply with. These interfaces, together with the proposed standard data structures, are intended to guarantee compatibility between modules with different implementations. However each developer gets little assistance for developing his particular modules. In the field of natural language analysis and understanding, the GATE architecture [4] goes beyond the definition of interfaces and standard data structures to provide a set of resources known as CREOLE ( a Collection of REusable Objects for Language Engineering ). In its current state, GATE does not cover the field of NLG. In this paper we describe an initial approximation to a set of reusable objects of the kind included in GATE, but centered in NLG. In order to address the problems described above, a high degree of flexibility is required. The present proposal considers the role that design patterns may have in guaranteeing the required flexibility. Section 2 presents a high level description of the architecture we propose. Section 3 describes the current library of reusable objects that we have implemented. Section 4 presents an example of an instantiation of the library and its performance over a simple NLG task. Section 5 outlines preliminary conclusions.

2 Looking for a Solution Starting from the ideas of the RAGS project, we attempt to look for a reusable software architecture for NLG systems by applying design patterns [6] to abstract the main design decisions involved in the construction of a system of this kind. As described in the previous section, three basic decisions must be taken: a division of the required funcionality into a set of modules, how control information flows between the modules, and which data structures are used to store the relevant information. Our solutions for these three decisions are the following. For the choice of the data structures, we use inheritance as in the Strategy design pattern. In this way, we have an abstract class that allows to implement different structures, and its subclasses are the ones which the modules use and have the needed information. According to the division into modules, we also use the Strategy design pattern for the implementation of the different modules, which will be instantiations of an unique

abstract class. The choice of which modules to use is taken using the Abstract Factory design pattern, in order to have a factory which defines concrete sets of modules. With respect to the flow of control information, the decision is taken in an abstract class implemented following the Strategy design pattern. A set of modules is passed to the constructor of this class, so that the control flow knows which modules the user has decided to use. In the implementations of the abstract class the user will define a concrete sequence of steps, that will transform the input data through the different modules into the output for the application being developed. Finally, connection between the control flow and the set of modules chosen by the user is provided using the Abstract Factory design pattern. In the concrete implementations of this abstract factory, the user will decide which is the combination of modules and flow of control information he needs in his application. The conjunction of these decisions and design patterns results in our current proposal, given in figure 1.

Fig. 1. Proposed software architecture for NLG systems (BasicStructures package)

3 cFROGS A framework-like library of architectural classes has been developed to test these ideas. This library, named cFROGS (a C++ Framework based on the Rags architecture for Object-oriented Generation Systems), includes a set of architecture templates that can help users to develop their own applications. The general structure for a concrete application is given in figure 2.

Fig. 2. Structure of a concrete application implemented using cFROGS Application is the external application that uses the NLG module made by the user, and generally its behaviour would be ignorant of the working of the NLG module. Application must only know the interface between them, and which is the format for the input and output information. Between the application and the library are the concrete modules implemented by the user, with the funcionality requested depending on the type of application. In this example these classes are in the PipelineWeather package. In the cFrogs package are the classes given by the cFROGS library, and it is divided into two basic packages. On the one hand, we have the BasicStructures package already shown in figure 1, that contains the skeleton of classes provided for the users of the library. On the other hand, we took the decision of giving the user the opportunity of choosing between some concrete architectures already defined. These can be found in the ArchitectureTemplates package, where several abstract classes, according to different architectures depending on modules, control flow and data structures, are provided. At the present time, the ArchitectureTemplates package provides the classes for a single architecture template (NLGTasksArchitecture) shown in figure 3, based in the common division of NLG tasks. The user has the possibility of using the classes contained in DataStructures and Module packages in his application. If he does, two of the important decisions to be taken in a NLG system would be covered: the choice of data structures and set of modules. In the future this package will be extended with more representative examples of architectures.

Fig. 3. Structure of the ArchitectureTemplates package

A more deep description of BasicStructures (figure 1) is the following. For the choice of the data structures, we have an abstract class Draft that allows to implement different structures using inheritance as in the Strategy design pattern, depending on the application in development. Draft subclasses centralize all the information, so that modules can use it completely or in part. All the modules will work with the concrete implementations of Draft, adding or taking information from it depending on their needs. The decision of which subclass of Draft to use is taken in the concrete specializations of the abstract class ConceptualRepresentation, that decides how to convert the input data into a subclass of Draft. With respect to the division into modules, we provide the following classes. Each module would be an instance of the abstract class Module, as in the Strategy design pattern, and will redefine its abstract method exec. The input and output of this method is a Draft, so the module takes the information it needs, works with it, and finally adds new information to the Draft if necessary. The choice of which modules to use is taken using the abstract class ModuleSetFactory, the implementations of which, following the AbstractFactory design pattern, define concrete sets of modules that are stored in ModuleSet. In ModuleSetFactory another decision is taken: which will be the concrete class for ConceptualRepresentation, that, as we have mentioned before, decides how to convert the input data into a subclass of Draft. With regard to the flow of control information, the decision is taken in the abstract class ControlFlowStrategy implemented following the Strategy design pattern. A ModuleSet is passed as parameter to the constructor of ControlFlowStrategy, so that the control flow knows which modules the user has decided to use. In the implementations of the execute method the user will define a concrete sequence of steps, that will transform the input Draft through the different modules of ModuleSet into the output Draft for the application being developed. Finally, connection between ControlFlowStrategy and ModuleSetFactory is found in the abstract class ArchitectureFactory, as in the AbstractFactory design pattern. Here the user can decide which is the combination of modules and flow of control information he needs in his application.

4 Example of use (PipelineWeather) The applicability of cFROGS is being tested through the development of a particular instantiation named PipelineWeather, a simple NLG application for reading meteorological information based on a pipeline architecture. The structure of this application is given in figure 4.

Fig. 4. Structure of PipelineWeather

We are going to describe the different modules we have implemented for PipelineWeather as users of cFROGS. In this example we will use the following tasks: content determination, discourse planning, lexicalization and surface realization. Therefore we will have in the Module package an instantiation of Module for each one: ContentDeterminationWeather, DiscoursePlanningWeather2, LexicalizationWeather2 and SurfaceRealizationWeather2. The decision of using these modules is taken in the instantiation of the abstract class ModuleSetFactory, ModuleSetFactoryWeather2: ConceptualRepresentation * ModuleSetFactoryWeather2:: createConceptualRepresentation() { return new ConceptualRepresentationWeather2(); } ModuleSet * ModuleSetFactoryWeather2::createModuleSet() { ModuleSet* set = new ModuleSet(); set->insertModule(new ContentDeterminationWeather()); set->insertModule(new DiscoursePlanningWeather2()); set->insertModule(new LexicalizationWeather2()); set->insertModule(new SurfaceRealizationWeather2()); return set; }

In ModuleSetFactoryWeather2 we have also decided the concrete strategy for the abstract class ConceptualRepresentation, which is ConceptualRepresentationWeather2 in this case. In the code of this class the decision of which implementation of Draft we are going to use (Txt2) is taken:

Draft* ConceptualRepresentationWeather2::convert(string in) { Txt2* result = new Txt2(); ... return result; }

On the other hand, the flow of control information is specified in the execute method of the instantiation of the abstract class ControlFlowStrategy, ControlFlowStrategyWeather. In this case we are using a pipeline architecture, so the execute method reads the modules from the ModuleSet and executes them with a Draft as input and output, as follows: Draft* ControlFlowStrategyWeather::execute(Draft* in) { for (int i=0; i< modSet->moduleNumber(); i++) { in = modSet->getModule(i)->exec(in); } return in; }

Finally, the choice of which modules and flow of control information use is taken in the implementation of the abstract class ArchitectureFactory, ArchitectureFactoryWeather2. This decision is specified in the BusinessFacade class which, as in the Facade design pattern, shields the application from the rest of modules: string BusinessFacade::execute(string in) { ArchitectureFactoryWeather2 * fac=new ArchitectureFactoryWeather2(); ModuleSetFactory * modSetFac = fac->createModuleSetFactory(); ControlFlowStrategy *flowStr=fac->createControlFlowStrategy(); return ((Txt2*)(flowStr->execute(modSetFac-> createConceptualRepresentation()->convert(in))))->getText(); }

The data structures have been implemented as shown in figure 5. The class Txt2 is a specialization of the abstract Draft, and it is the one the modules of PipelineWeather work with. Inside Txt2 we have semantic representations, represented by the SemRep class, documental representations, represented by the DocRep class, and rhetorical representations, represented by the RhetRep2 class. In the case of RhetRep2 we have defined a tree structure using the Composite design pattern. The leaves of the tree are RhetRep2AtomicNode objects, that only contain a SemRep, and the intermediate nodes are RhetRep2ComplexNode objects, that have a list of sons that are RhetRep2 objects at the same time.

Fig. 5. Data structures for PipelineWeather

A possible file with the input data will have the following structure: temperature(Madrid,15,today) cloudiness(Madrid,5,today) temperature(Madrid,20,tomorrow) cloudiness(Barcelona,7,today) cloudiness(Bilbao,10,tomorrow)

Each fact is written in a single line. This input is read by Application, and passed to BusinessFacade into a string of characters. Before we can execute the sequence of actions specified by the modules, it is necessary to transform the string of characters into the data structure the modules work with: a Draft. ConceptualRepresentationWeather2 takes charge of this, decomposing the list of facts, and the facts into predicates and arguments as in the semantic representation of SemRep. All this information is introduced in a concrete subclass of Draft, Txt2. So, the output Txt2 from ConceptualRepresentationWeather2 contains a list of SemRep, each representing one of the input facts. In the control flow the first step is ContentDeterminationWeather. Here the facts are filtered by city and date, keeping the facts with the values “Madrid” and “today”. The next step is DiscoursePlanningWeather2. Here the list of SemRep is transformed into a rhetorical representation RhetRep2, separating the facts into new SemRep that now are in the leaves of the tree rhetorical structure. The following step is LexicalizationWeather2. Using templates the SemRep of the rhetorical structure are transformed to a documental structure DocRep. The last step is SurfaceRealizationWeather2. Here the strings of characters of DocRep are concatenated following the

correspondent relations between the leaves in the rhetorical structure. In addition, the first letter of the first string is turned to capital and a full stop is added at the end. The output of each step, as well as the final output, is shown in table 1. Table 1. Output of the modules of PipelineWeather

MODULE

OUTPUT SemRep list:

CONCEPTUAL REPRESENTATION

CONTENT

DETERMINATION

DISCOURSE PLANNING

temperature(Madrid,15,today) cloudiness(Madrid,5,today) temperature(Madrid,20,tomorrow) cloudiness(Barcelona,7,today) cloudiness(Bilbao,10,tomorrow) SemRep list: temperature(Madrid,15,today) cloudiness(Madrid,5,today) Rhetorical representation: description[localization(Madrid,today), conjunction[temperature(15),cloudiness(5)]] Documental representation:

LEXICALIZATION

the weather in Madrid for today is the following the temperature is 15 degree centigrade it is cloudy Final output:

SURFACE

REALIZATION

The weather in Madrid for today is the following: the temperature is 15 degree centigrade and it is cloudy.

5 Preliminary Conclusions We are very optimistic about the advantages of implementing a particular application using cFROGS, because in the preliminary work in this direction the flexibility of the library and the centralization of the decision points have been demonstrated. To make use of an architecture of this kind would be beneficial in the field of research as well as in teaching. In relation to investigation, an architecture like cFROGS would permit a fast development of NLG modules for projects where the implementa-

tion of this modules is not the main task. In regard to teaching, this architecture would represent an example of which are the available techniques in this field, and would give the opportunity of putting them into practice. While developing, the revisions of the architecture and additions of new design patterns and ideas, have been constant. With the incorporation of new funcionality, it is necessary to look for solutions for the new problems, and therefore feedback is frequent. In this way, cFROGS is changing at present. Some of the priority points we are working on are to include configuration mechanisms for certain features - like for the set of linguistic resources to be used by the application - that would be independent of the text. In addition, we are contemplating the use of cFROGS in another project, that will integrate 3D environment with a module of NLG built using our library to give textual explanations.

References 1. Cahill, L., Doran, C., Evans, R., Mellish, C., Paiva, D., Reape, M., Scott, D. and Tipper, N. Towards a reference architecture for natural language generation systems. Technical Report ITRI-99-14, Information Technology Research Institute, University of Brighton, 1999. 2. Cahill, L., Evans, R., Mellish, C., Paiva, D., Reape, M., Scott, D. The RAGS reference manual. Technical Report ITRI-01-07, Information Technology Research Institute, University of Brighton, 2001. 3. Calder, J., Evans,R., Mellish, C., Reape,M.: “Free choice” and templates: how to get both at the same time. In “May I speak freely?” Between templates and free choice in natural language generation,number D-99-01. Saarbr¨ucken, pages 19–24.1999. 4. Cunningham, H., R.G. Gaizauskas, and Y. Wilks. 1995. A General Architecture for Text Engineering (GATE) - a new approach to Language Engineering R&D. Technical Report CS - 95 - 21, Department of Computer Science, University of Sheffield. 5. DeSmedt, K., Horacek, H., Zock M.: “Architectures for Natural Language Generation: Problems and Perspectives”, in: Ardoni, G. and Zock, M. (eds.) Trends in natural language generation: an artificial intelligence perspective (Springer Verlag lecture notes in artificial intelligence 1036) (pp.17-46) Berlin: Springer, 1995. 6. Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design Patterns: Elements of Reusable Object-Oriented Software. Addison Wesley, 1994. 7. Kantrowitz, M., Bates, J.: Integrated natural language generation systems. In R.Dale, E.Hovy, D. Rösner, and O. Stock, editors, Aspects of Automated Natural Language Generation, pages 13--28. Springer, Berlin, 1992. 8. Kempen, G., Hoenkamp, E.: An incremental procedural grammar for sentence formulation. Cognitive Science, 11, 201-258. 1987. 9. Reiter, E. “Has a consensus NL generation architecture appeared, and is it psychologically plausible?”, in: McDonald, D. and Meteer, M. (eds.) Proceedings of the 7th. International Workshop on Natural Language generation (INLGW ’94), pages 163-170, Kennebunkport, Maine, 1994.