Interfaces of the Dialogue Component in the VERBMOBIL ... - CiteSeerX

Interfaces of the Dialogue Component in the VERBMOBIL Demonstrator, dialogue-system 4 Jan Alexandersson DFKI GmbH

Technisches Dokument 23

Marz 1995

Marz 1995

Jan Alexandersson DFKI GmbH Stuhlsatzenhausweg 3 66123 Saarbrucken Tel.: (0681) 302 - 5347

e-mail: [email protected] Das diesem Bericht zugrundeliegende Forschungsvorhaben wurde mit Mitteln des Bundesministers fur Forschung und Technologie unter dem Forderkennzeichen 01 IV 101 K/1 gefordert. Die Verantwortung fur den Inhalt dieser Arbeit liegt bei den Autoren.

Abstract This paper presents the current interfaces of the dialogue component as implemented in the demonstrator february -95. The aim with this paper is twofold: 1. It should serve as a basis for writing software that accesses the dialogue component. 2. It should (together with [3]) also serve as a basis for discussion on future extensions { the readers are strongly encouraged to contact the dialogue group when the current version of the dialogue system not suces. Also, this document will be updated as more functionality is added to the dialogue component { look in the bibliography on the FTP-server in Saarbrucken to nd an upto-date version of this document.

1

Contents 1 Introduction

1.1 Reading this document

3 : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

3

2 A sketch of the Dialogue Component

4

3 Current interfaces

5

3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10

The Testbed The Semantic Evaluation Component (semaus) The Semantic Construction Component (semkon) The Keyword Spotter (KWS) The German Generator The Speech Recognizer (aufnahme) The Syntax Parser The German Synthesizer (synthesedt) Clari cation Dialogues Global Variables

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

4 Conclusions and Future Extensions

2

6 8 10 11 12 13 14 15 16 17

18

1 Introduction We present the interfaces of the dialogue component. It describes the interfaces as implemented in the demonstrator1 (february -95). It is shown how the system could be accessed (by means of storing and/or rewtrieving data). In the demonstrator, the communication was based on ICE [2]. Therefore this document is describing how to access the dialogue component via ICE 2. We start with a brief sketch of the dialogue component in section 2. In section 3 we then describe the dierent interfaces component by component.

1.1 Reading this document The interface consists of messages send to or from the dialogue component. Under a label of the form:

from ! to: where \from" is the sender of the message, and \to" is the receiver, you will nd messages described in the following format:

Label

from ! to (comment)

Syntax: \the syntax" (type description) Purpose: the purpose ...where \Label" should give a hint of what the message is for, \from" and \to" is the names of the components involved, and \the syntax" describes the syntax. The \(type description)" which is an optional comment, is the type of the message (string, integer, etc). The type description does not apply for predicates (see section 3.2). \the purpose" is a description of the eects, reason, and/or preconditions of why/when the message is send. Our module will be referred to as the version 4 . It is to be found on the FTP-server under FTPSERVER/vm-module/dialogue-system-4.tar.gz 2 The communication could for instance instead be based on TCP, sockets or similar tools. In fact our rst version of the dialogue system used sockets as basis for the communication. Sockets have the advantage that one can run dierent modules at dierent internet sites, a fact successfully explored when running our rst test together with the FLEX system in TU Berlin. 1

3

2 A sketch of the Dialogue Component Here we give a brief scetch of the dialogue component3. It consists of the following sub parts (see gure 1). A Statistical Module A statistics-based prediction algorithm for the next dialogue acts. A Finite State Machine A non-deterministic Finite State Machine that implements the dialogue model. A Plan Recognizer The plan recognizer is responsible for constructing the dialogue structure. A Dialogue Memory which currently contains the sequence of turns and an intentional structure. This will in the future also contain a thematical structure where the actual dates will be stored, as well as a referencial strucure. An interface to the outer world The interface provides the process de nition, initialization and the input/output functions for the dialogue module. A Surface module This module visualizes the current state of the dialogue module. Dialogue Act Determination

Finite State Machine Semantic Evaluation Key Word Prediction Dialogue Act Determination

Key Word Spotter

Statistic Component

Time Description Selection of Language Model

Statistical Repair

Speech Recognition

Planner / Plan Recognizer Generation of Referents

Construct Dialogue Memory

Dialogue Component

Dialogue Memory

Generation

Transfer Constraining Translation Equivalents

Figure 1: The architecture of verbmobil The dialogue module is imlemented in both Allegro (Common) Lisp and Clisp4, and will in the future also be ported to Lucid. It is developed in Franz Allegro 4.2 environment, and uses tcl/tk for visualization. The dialogue module runs in the package "DIALOGUE" To get more information the reader is encouraged read [1], [4], and [3] Norbert Reithinger ([email protected]) gladly answers Questions concerning the cLisp version. The Clisp version is a \full" version of the dialogue component (with exception of the ICE interface). 3 4

4

3 Current interfaces We divide the data going to and from the dialogue module into three types.

Discourse Information By this we mean actual data concerning storing/retrieving dis-

course information e.g. dialogue acts, propositional contents of an utterance, etc. Clari cation Request We are responsible for performing clari cation dialogues. Hence we oer other modules the possibility to call for a clari cation dialogue (for instance when a certain component has failed to solve its task). Control Information This information mostly concerns wether the dialogue component is active/inactive, a request to start the surface, etc. This chapter starts with a description of the dierent interfaces, and ends with a section concerning global variables that can be set to change the behaviour of the dialogue component. The software version 4 interfaces the following modules:

       

The testbed The semantic evaluation component The semantic construction component The keyword spotter The german synthesizer The german generator The speech recognizer The syntax component

5

3.1 The Testbed The communication with the testbed consists mainly of control information. We tell the testbed when we are ready, active, or inactive, etc. The testbed can reset the dialogue component, and switch the surface on or o. We communicate with the testbed by sending and receiving strings.

Testbed ! Dialogue: reset dialogue

testbed ! dialogue ICE

Syntax: \reset" (string) Purpose: Causes the dialogue to reset itself, and get prepared for a new dialogue.

This means that the dialogue component resets all its internal structures (i.e. all information concerning previous dialogues are thrown to the gc-monster). Omitting to send this message before a new dialogue is started, may have impact on the translation of the utterances. Therefore it is recommended to send a reset before starting a new dialogue.

guion

testbed ! dialogue ICE

Syntax: \guion" (string) Purpose: Causes the dialogue to switch on the surface.

guio

testbed ! dialogue ICE

Syntax: \guio" (string) Purpose: Causes the dialogue to switch o the surface.

exit dialogue

testbed ! dialogue ICE

Syntax: \exit" (string)

6

Purpose: Causes the dialogue to kill itself.

Dialogue ! Testbed: dialogue ready

dialogue ! testbed ICE

Syntax: \dialog ready" (string) Purpose: Informs the testbed that we are ready.

dialogue active

dialogue ! testbed ICE

Syntax: \dialog active" (string) Purpose: Informs the testbed that we are active.

dialog inactive

dialogue ! testbed ICE

Syntax: \dialog inactive" (string) Purpose: Informs the testbed that we are inactive.

7

3.2 The Semantic Evaluation Component (semaus) For this interface and for the demonstrator we developed a Lisp-Prolog protocol. Using this we wrote predicates in Prolog, which are described below. However, in future versions of the dialogue component, we will not support the writing of Prolog predicates { the user will have to access the dialogue component via ICE . An example of a rudimentary protocol is to be found in the appendix.

Semaus ! Dialogue: id/1

semaus ! dialogue (predicate)

Syntax: id(-Id) Purpose: Get a new turn-identi er

teilsh id/2

semaus ! dialogue (predicate)

Syntax: teilsh id(+Id, -Teilsh id) Purpose: Get a new sub-turn-identi er for a turn

sprechhandlung/2

semaus ! dialogue (predicate)

Syntax: sprechhandlung(+Id,+Typ) Purpose: Update the dialogue memory with a dialogue act for a certain turn

teil sh/3

semaus ! dialogue (predicate)

Syntax: teil sh(+Id, +TeilId, +Dialog akt) Purpose: Update the dialogue memory with a dialogue act for a certain sub turn

8

vorhersage/4

semaus ! dialogue (predicate)

Syntax: vorhersage(+Id, -Vorhersage1, -Vorhersage2, -Vorhersage3) Purpose: Retrieve the three most probable Top Down Predictions for the turn Id + 1. The Variables Vorhersagef1,2,3g are bound to a list with two atoms, the

rst is a dialogue act and the second is the score. The score is given as the score  1000, e.g. when the probability for a certain dialogue act is 0.123 the score given will be 123. If the +Id is not yet de ned (the dialogue does not recognize the turn id) the predicate fails.

9

3.3 The Semantic Construction Component (semkon) The interface to the semantic construction component consists of a clari cation request.

Semcon ! Dialogue: clari cation-dialogue-request

semkon ! dialogue ICE

Syntax: message (string) Purpose: Request for clari cation dialogue. semkon failed nth g


10

message 2 f




semkon failed 1st,

3.4 The Keyword Spotter (KWS) The communication between the dialogue component and the KWS is a bidirectional communication. The dialogue component is called with a name of a le where the most recent recognized keywords are written. For every turn the KWS is then updated with the 3 most probable dialogue acts to come.

Keyword Spotter ! Dialogue: lename

KWS ! dialogue ICE

Syntax: lename (string) Purpose: Give the dialogue the name of a le where the most recent recognized words


from the KWS are stored. If the string is a valid path description and the le exists, then the dialogue analyzes the content of the le and attributes a dialogue act to the utterance.

Dialogue ! Keyword Spotter: Update-kws

dialogue ! KWS ICE

Syntax: \d-a score d-a score d-a score" (string) Purpose: Update the KWS with the 3 most probable dialogue acts5 to look for, and their scores.

5 d-a

= dialogue act.

11

3.5 The German Generator Since the generated sentences only consist of prede ned utterances we send a number to the generator which correspond to a prestored structure. For eciency reasons we can instead redirect the clari cation message directly to the german synthesizer (see 3.8 3.9). To change the behaviour see 3.10.

Dialogue ! Generatordt: send-to-german-generator

dialogue ! generatordt ICE

Syntax: message (string) Purpose: message 2 f"8", "10", "30", "31", "32", "33"g


12

3.6 The Speech Recognizer (aufnahme) We receive two error messages from the recognizer. Either the input signal is to low, or it is to high.

Aufnahme ! Dialogue: clari cation-dialogue-request

aufnahme ! dialogue ICE

Syntax: message (string) Purpose: Request for clari cation dialogue. The message is on of the following





3.7 The Syntax Parser The interface to the syntax parser consists of a clari cation request.

Syntax ! Dialogue: clari cation-dialogue-request

syntax ! dialogue ICE

Syntax: message (string) Purpose: Request for clari cation dialogue. parser failed nth g


14

message 2 f




parser failed 1st,

3.8 The German Synthesizer (synthesedt) We can also (see 3.5 3.10) call the german synthesizer with strings.

Dialogue ! Synthesedt: send-to-german-synthesizer

dialogue ! synthesedt ICE

Syntax: message (string) Purpose: message 2 f"signal too high", "signal too low", "parser failed 1st", "parser failed nth", "semkon failed 1st", "semkon failed nth" g


15

3.9 Clari cation Dialogues For the demonstrator no complex clari cation dialogues have been implemented. The clari cation dialogues consist of single utterances informing the user that verbmobil could not process the utterance. The requests are either encoded to numbers which are sent to the german generator or redirected to the german synthesizer. The german generator triggers prestored input structures to be gernerated, while the german synthesizer synthesizes prestored strings (see 3.10). Figure 2 shows the encoding of the dierent requests to numbers and how they are nally realized. From Message Code Synthesized aufnahme signal too high 10 sprechen sie bitte etwas leiser aufnahme signal too low 8 sprechen sie bitte etwas lauter syntax parser failed 1st 30 die syntaktische Suche konnte im Worthypothesengraphen keinen Satz nden syntax parser failed nth 31 die syntaktische Suche konnte im Worthypothesengraphen keinen weiteren Satz nden semkon semkon failed 1st 32 die syntaktisch-semantische Verarbeitung konnte keine Loesung nden semkon semkon failed nth 33 die syntaktisch-semantische Verarbeitung konnte keine weitere Loesung nden Figure 2: Clari cation Request tabel

16

3.10 Global Variables In this section we describe global variables which has impact on the behaviour of the dialogue component.

*generator-dt-on*

variable

Syntax: *generator-dt-on* Purpose: When a component requests the dialogue component with a clari cation re-

quest, the dialogue component sends either a message direct to the german synthesizer, or to the german generator. When *generator-dt-on* is set to T the german generator will be called, otherwise the german synthesizer.

17

4 Conclusions and Future Extensions We have presented the interfaces to the dialogue component as implemented in the verbmobil demonstrator february -95. For information about future extensions, please read [3].

References [1] Jan Alexandersson, Elisabeth Maier, and Norbert Reithinger. A Robust and Ecient Three-Layered Dialog Component for a Speech-to-Speech Translation System. In Proceedings of the 7th Conference of the European Chapter of the ACL (EACL-95), Dublin, Ireland, 1995. also available as Verbmobil Report Nr. 50, DFKI GmbH, Dezember 1994. [2] Jan W. Amtrup. ICE: INTARC Communication Environment Users Guide and Reference Manual Version 1.2. Verbmobil Technisches Dokument 14, Universitat Hamburg, January 1995. [3] Elisabeth Maier, Jan Alexandersson, and Norbert Reithinger. Titel Unknown. Verbmobil report 63, DFKI Saarbrucken, 1995. forthcoming. [4] Norbert Reithinger. Some Experiments in Speech Act Prediction. In AAAI 95 Spring Symposium on Empirical Methods in Discourse Interpretation and Generation, 1995.

18

Appendix: An example of a Prolog-Lisp protocol To illustrate how a protocol could look like, we here give a simple example implementation in Prolog using ICE as communication basis. The protocol allows for sending strings and integers. On the Lisp side one has to specify a similar mechanism, that for each predicate return the corresponding parameters. The Prolog code below is just an illustration of a possible implementation. It is not intended to show the way to implement a robust LispProlog protocol { it is merely an illustration. The input parameters should for instance be checked for groundness etc. We rst de ne a predicate call lisp/2 which we later use to implement the appropriate predicate. Below get tag/2 binds its second arument to the appropriate ICE tag (for more detailed information see [2]). The protocol works like this. First the predicate name (for instance teil sh) is sent, followed by its input parameters (in case of teil sh a turn id). The dialogue then sends the corresponding output parameter(s) (for teil sh an unique identi er). %%%%%%%%%%%%%%%%%%%%%%%%%%%%% %call_lisp(+Command, -Result) call_lisp(Command,Result) :send(Command), receive(Result).

%send(+Command). send([]). send([H|T]) :atom(H), get_tag(dialog,Dest), ice_Send( Dest, ice_notag, 0, 0, 50, idl_str, H), send(T). send([H|T]) :integer(H), get_tag(dialog,Dest), ice_Send( Dest, ice_notag, 0, 0, 50, idl_int, H), send(T).

%receive(-Result). receive([]). receive([H|T]) :get_tag(dialog,Dest), ice_Receive( Dest, ice_any, _, ice_msg(_,_,_,_,H)),

19

receive(T).

Using call lisp/2 we can easily implement our prolog predicates for instance as shown below, the de nition of teilsh id/2. %%%%%%%%%%%%%%%%%%%%%%%%%% %teilsh_id(+ID, -Teilsh_id) % ID : Identifier % Teilsh_id: Teilidentifier teilsh_id(ID, Teilsh_id) :call_lisp([teilsh_id,ID],[Teilsh_id]).

20