interruptable transition networks - Association for Computational ...

INTERRUPTABLE TRANSITION NETWORKS Sergei N i r e n b u r g Colgate U n i v e r s i t y Chagit A t t i y a H e b r e w U n i v e r s i t y of J e r u s a l e m

ABSTRACT

l l a b e l x M), w h e r e l a h ~ l belongs to the vocabulary of c o n s t i t u e n t types. In our i m p l e m e n t a t i o n this set includes S, NP, VP, PP, NP& (the "virtual" NP), Del (the delimiter), etc. X and M are the left and the right indices of the b o u n d a r i e s of these c o n s t i t u e n t s in the input string. They m a r k the points at w h i c h p a r e n t h e s e s are to be opened (x) and closed (y) in the tree representation. The values x and y relate to positions of words in the initial input string. For example, the s e n t e n c e (i) w i l l be p r o c e s s e d at stage 2 into the string (2). The '?' in (2) stand for u n k n o w n c o o r d i n a t e s y.

A specialized transition network mechanism, the interruptable transition n e t w o r k (ITN) is used to p e r f o r m the last of three stages in a m u l t i p r o c e s s o r s y n t a c t i c parser. This approach can be seen as an exercise in implementing a parsing procedure of the active chart parser family. Most of the ATN parser implementations use the left-to-right top-down chronological backtracking control structure (cf. Bates, 1978 for discussion). The control strategies of the active chart type permit a blend of bottom-up and top-down parsing at the expense of time and space o v e r h e a d (cf. Kaplan, 1973). The e n v i r o n m e n t in w h i c h the interruptable t r a n s i t i o n n e t w o r k (ITN) has been implemented is not similar to 6hat of a t y p i c a l A T N model. Nor is it a straightforward implementation of an active chart. ITN is r e s p o n s i b l e for one stage in a multiprocessor parsing technique described in Lozinskii & Nirenburg, (1982a and b), where p a r s i n g is performed in e s s e n t i a l l y the bottom-up fashion in p a r a l l e l by a set of r e l a t i v e l y small and "dumb" p r o c e s s i n g units running identical software. The process involves three stages: (a) p r o d u c i n g the c a n d i d a t e strings of p r e t e r m i n a l category symbols; (b) determining the positions in this string at w h i c h h i g h e r - l e v e l constituents start and (c) d e t e r m i n i n g the closing b o u n d a r i e s of these constituents.

(i) The v e r y big b r i c k b u i l d i n g that sits 1 2 3 4 5 6 7 on the hill belongs to the university. 8 9 i0 ii 12 13 14 (2)

(s 1 ?)(np 1 ?)(s 6 ?)(np& 6 6) (vp 7 ?)(pp 8 ?)(np 9 ?)(vp ii ?) (pp 12 ?)(np 13 ?)

It is at this point that the interruptable transition network starts its w o r k of finding the unknown boundary c o o r d i n a t e s and thus d e t e r m i n i n g the upper levels of the parse tree. An input string ~ triads long w i l l be allocated n identical processors. Initially the chunk of e v e r y p a r t i c i p a t i n g processor will be one triad long. After these p r o c e s s o r s finish w i t h their chunks (either s u c c e e d i n g or failing to find the missing coordinate) a "change of levels" interrupt occurs: the size of the chunks is d o u b l e d and the number of active processors halved. T h e s e latter c o n t i n u e the s c a n n i n g of the ITN from the point they w e r e interrupted t a k i n g as input what was f o r m e r l y the chunk of their right neighbor. Note that all constituents already closed in that chunk are transparent to the c u r r e n t p r o c e s s o r and already closed in that chunk are transparent to the current p r o c e s s o r and are not rescanned. The number of active processors steadily reduces during parsing. The choice of processors that are to remain a c t i v e is made with the help of the Pyramid p r o t o c o l (cf. Uozinskii & Nirenburg, 1982). The p r o c e s s o r s released

Each of the p r o c e s s o r s allocated to the first stage obtains the set of all syntactic readings of one word in the input string. Using a table grammar, the processors then choose a subset of the word's readings to ensure c o m p a t i b i l i t y with similar subsets generated by this p r o c e s s o r ' s right and left neighbor. S t a g e 2 uses the results of stage 1 and a different tabular grammar to e s t a b l i s h the left ("opening") boundaries for c o m p o s i t e s e n t e n c e constituents, such as NP or PP. The output of this stage assumes the form of a string of triads

393

Since it is never known to any p r o c e s s o r w h e t h e r it w i l l be a c t i v e at the next stage, it is n e c e s s a r y that the information it o b t a i n e d be saved in a place w h e r e another p r o c e s s o r w i l l be able to find it. Unlike the standard ATN parsers (which r e t u r n the parse tree as the value of the p a r s i n g function), the I%N parser records the results in a special working area (see discussion below).

after each "layout" are returned to the system pool of a v a i l a b l e resources. At the top level in the pyramLd only one processor wL]l remain. The status of such a processor is d e c l a r e d final, and this trlggers the wrap-up o p e r a t i o n s and the c o n s t r u c t i o n of output. The w r a p - u p uses the original string of words and the a p p r o p r i a t e string of p r e t e r m i n a l symbols obtalned at stage 1 together with the results of stage 3 to build the parse tree.

impl~m~nLaLiun ITN can start processing at an arbltrary position in the input string, not n e c e s s a r i l y at the beginning of a sentence. Therefore, we introduce an a d d i t i o n a l subnetwork, "initial", used for handling control flow among the other subnetworks.

The ITN interpreter was i mp l e m e n t e d in YLISP, the d i a l e c t of LISP d e v e l o p e d at the H e b r e w University of Jerusalem. A special scheduler r o u t i n e for s i m u l a t i n g p a r a l l e l p r o c e s s e s on a VAX 11/780 was written by Jacob Levy. The interpreter also uses the pyramid p r o t o c o l p r o g r a m by S h m u e l Bahr.

The llst of "closed" constituents obtained through ITN-based parsing of string (2) can be found in (3), w h i l e (4) is the output of ITN p r o c e s s i n g of (3). (3)

In w h a t follows we w i l l d e s c r i b e the organization of the stack, the w o r k i n g area, and the p r o g r a m itself.

(s 1 [4)(np 1 10)(s 6 10)(np& 6 6) (vp 7 10)(pp 8 10)(np 9 10)(vp ll 14) (pp 12 14)(np 13 14)

(4)

(s(np(s(np&)(vp(pp(np)))))(vp(pp)))

3.

An

ITN

a) The stack. The item to be s t a c k e d must describe a p o s i t i o n in the network. An item is pushed onto the stack every time a PUSH or an INTR arc is traversed. E v e r y time a POP arc is t r a v e r s e d or a return from an interrupt occurs one item is popped. The s t a c k item consists of: I) names and values of the c u r r e n t n e t w o r k registers; 2) the r e m a i n d e r of the arcs in the state (after the PUSH or the INTR traversed); 3) the actions of the PUSH arc traversed; 4) the name of the current network (i.e. that of the latter's initial state); 5) the v a l u e of the input pointer (for the case of a PUSH failure).

Interpreter.

The interpreter was designed for a parallel processing system. This goal c o m p e l l e d us to use a p r o g r a m environment s o m e w h a t d i f f e r e n t from the usual p r a c t i c e of writing ATN interpreters. Our interpreter can, however, be used to interpret both ITNs and ATNs. A new type of arc was introduced: the interrupt arc INTR. The interrupt arc is a w a y out of a n e t w o r k state a d d i t i o n a l to the regular POP. It g i v e s the process the o p p o r t u n i t y to resume from the very point w h e r e the interrupt had been called, but at a later stage (this mechanlsm is rather similar to the d e t a c h - t y p e commands in p r o g r a m m i n g languages which support coroutines, such as, for instance, SIMULA). Thus, the interpreter must be able to suspend p r o c e s s i n g after trying to proceed t h r o u g h any arc in a state and to resume processing later in that very state, from the arc immediately following the interrupt arc. For example, if [NTR is the fourth of seven arcs in a state, the work resumes from the fifth arc in this state. This is implemented with a stack in w h i c h the t r a n s i t i o n s in the net are recorded. The PUSH and POP arcs are also implemented through this stack and not through the recursion handling m e c h a n i s m s built into Lisp.

The w o r k i n g area is used for two purposes: to support message passing b e t w e e n the p r o c e s s o r s and to hold the findings. The w o r k i n g area is o r g a n i z e d as an array, R, that holds a d o u b l y linked list used to c o n s t r u c t the o u t p u t tree. The actions d e f i n e d on the working area are: a) initialization (procedure init-input): e v e r y cell R[i] in R obtains a token from input, while the links Rill.[next-index] and R[i].[previous-index] obtain the values i+l and i-l, respectively; b) CLOSE, the tool for d e l i m i t i n g s u b t r e e s in the input string; The array R is used in p a r a l l e l by a number of processors. At e v e r y level of p r o c e s s i n g the active processors' chunks cover the array R. This a r r a n g e m e n t does not c o r r u p t the p a r a l l e l character of the process, since no p r o c e s s o r a c t u a l l y seeks i n f o r m a t i o n from the chunks other than its own.

394

For

The m a i n f u n c t i o n of the interpreter is called //,El. It obtains the s ta c k c o n t a i n i n g the h i s t o r y of processing. If an interrupt is encountered, the f u n c t i o n returns the s t a c k w i t h n e w history, to be used for invoking this f u n c t i o n again, by the p y r a m i d protocol.

1

.

The p a r a m e t e r s of ~ l o s e are i) the number of the t r i a d we w a n t to c l o s e and 2) the v a l u e for w h i c h the y in this triad is to be substituted. The d e f a u l t v a l u e for the second p a r a m e t e r is the value of the y in the triad c u r r e n t at the m o m e n t a call to ~ i o s ~ is made. When the processing is parallel, £1os~ is a p p l i e d m u l t i p l y at e v e r y level, which would mean that a higher level processor will obtain prefabricated subtrees as e l e m e n t a r y input tokens. This is a major source of the efficiency of multiprocessor parsing.

INTR arc has the following (INTR*).

The ABORT arc (ABORT).

input string

after the a c t i o n (close 3 i0) is p e r f o r m e d the input for further p r o c e s s i n g has the form:

At this stage we a l r e a d y know which state of w h i c h n e t w o r k f r a g m e n t we are in. Moreover, we even know the path through the states and f r a g m e n t s we t o o k in order to reach this state and the exact arc in this state from which we have to start processing. So, we e x e c u t e the test on the current arc. If the test s u c c e e d s we p e r f o r m b r a n c h i n g on the arc name.

The current state is stacked p r o c e d u r e is exited r e t u r n i n g the the value. was inserted p r e s e r v e the usual c o n v e n t i o n of the test in the t h i r d slot in an

if the



If a call to i t n is a r e t u r n from the interrupt status, then a s t a c k item is popped (it c o r r e s p o n d s to the last state entered d u r i n g the p r e v i o u s run). If the f unc t i o n call is the initial one, we start to scan the n e t w o r k from the first state of the "initial" subnetwork.

The syntax:

example,

The ITN in the c u r r e n t i m p l e m e n t a t i o n is relatively small. A broader i m p l e m e n t a t i o n w i l l be n e e d e d to s t u d y the properties of this parsing scheme, including the estimates for its time complexity, and the e x t e n d a b i l i t y of the grammar. A c o m p a r i s o n should also be made w i t h other m u l t i p r o c e s s o r p a r s i n g schemes, including t h o s e that are based not on organizing c o m m u n i c a t i o n among r e l a t i v e l y "dumb" processors running identical software but rather on interaction of highly specialized and "intelligent" processors -cf., e.g., the w o r d e x p e r t parser (Small, 1981).

and the s t a c k as s i m p l y to situating arc. syntax

W h e n we e n c o u n t e r an error and it becomes clear that the input string is illegal, we want to be able to stop processing immediately and print a d i a g n o s t i c message.

Acknowledgments. The authors thank E. L o z i n s k i i and Y. Ben A s h e r for the m a n y d i s c u s s i o n s of the ideas d e s c r i b e d in this paper.

The actions on the s t a c k involve the m o v e m e n t of an item to and f r o m the stack. The s t a c k item is the q u a n t u m value that can be p u s h e d and popped, that is no part of the item is a c c e s s e d separately from the rest of the values in it. The functions managing the stack are p u s h - o n - s t a c k and p o p - f r o m - s t a c k .

Bibliography Bates, M. (1978), The t h e o r y and practice of augmented transition network grammars. In: L. Bolc (ed.), Natural Language Communication w i t h Computers. Berlin: Springer.

The p u s h - o n - s t a c k is called whenever a PUSH or an INTR arc is t r a v e r s e d . The p o p - f r o m - s t a c k is called, first, w h e n the POP arc is t r a v e r s e d and, second, w h e n the process resumes after return from an interrupt.

Kaplan, R. M. (1973), A g e n e r a l s y n t a c t i c processor. In R. R u s t i n (ed.), N a t u r a l Language Processing. NY: Academic Press.

The closa action is p e r f o r m e d w h e n we find a b o u n d a r y for a c e r t a i n s u b t r e e for w h i c h the opposite boundary is a l r e a d y known (in our case the b o u n d a r y that is found is always the right boundary, y). QIo~@ performs two tasks: first, it inserts the numeric value for y and, second, it declares the newly built subtree a n e w t o k e n in the input string.

Loz~nskii, E.L. and S. N l r e n b u r g (1982a). Locality in N a t u r a l L a n g u a g e processing. In: R. T r a p p l (ed.), Cybernetics and Systems Research. Amsterdam: N o r t h Holland.

395

Lozinskii, (1982b), language. France.

(*nthcdr c u r r - a r c 3)))) (go loop)) ((eq a r c - n a m e 'abort) ; A B O R T (tpatom (*nth c u r r - a r c 2)) (return nil)) ((eq a r c - n a m e 'intr) ; INTeRrupt (push-on-stack) (return stack)) (t ; error (tpatom '"illegal arc") (return nil)) ( go loop ] ; try next arc

E.L. and S. N i r e n b u r g Parallel processing of natural Proceedings of ECAI, Orsay,

Small, S. (1981), Viewing word parsing as a linguistic Proceedings of IJCAI, Vancouver,

expert theory. B.C..

A p p e n d i x A. ITN: the main function of the interruptable transition network interpreter (def Itn (lambda ( stack ) ; stack - current processing stack (prog (regs c u r r - s t a t e - a r c s n e t - n a m e c u r t - a r c $ test arc-name) ; regs - current registers of the n e t w o r k ; curr-state-arcs list of arcs not yet ; processed zn current state ; n e t - n a m e - name of n e t w o r k being : processed ; c u r t - a r c - arc in p r o c e s s i n g ;(all these are pushed on stack when a ; 'push' arc occurs) ; $ - a special register. ; the functlon first checks if stack is ; nil; if not then this call is a return ; from interrupt previous values must be ; popped from the stack [cond (stack (seta ec pn nil) ;set e n d - c h u n k flag to nil ( p o p - f r o m - s t a c k t)) (t (set-net 'al] loop [ cond ((null curr-state-arcs) (cond((null (pop nil)) (return nil)] (set 'curt-arc (setcdr 'curt-state-arcs)) ( set 'test (*nth c u r r - a r c 3) ) ( cond ((eval test) ;test succeeds - t r a v e r s e the arc ( set 'arc-name (car curr-arc)) [cond ((eq arc-name 'push ) ; PUSH (evlist (*nth c u r r - a r c 4)) (push-on-stack) (set-net (cadr curr-arc)) (go loop)) ((eq arc-name 'pop ) ; POP (evlist (*nthcdr c u r r - a r c 3)) (cond ((null (pop(eval(cadr curr-arc)))) (return $))) (go loop)) ((eq arc-name 'jump ) ; JUMP (evlist (*nthcdr c u r r - a r c 3)) (set-state (*nth c u r t - a r c 2)) (go loop)) ((eq a r c - n a m e 'to) ; TO (evlist (*nthcdr c u r r - a r c 3)) (set-state (*nth curr-arc 2)) (get-input) (go loop)) ((eq arc-name 'cat) ; CAT (cond

L[eq (currlI~B)) (*nth curt-arc 2)) (evlist

396

A p p e n d ix B. A F r a g m e n t of an ITN n e t w o r k (the "initial" and the s e n t e n c e subnetworks) ;Note that "jump" and "to" can be either ;terminal actions on an arc or s e p a r a t e ;arcs (def-net '(s-place) '( (initial (pop t (end-of-sent) (close*)) (intr nil ( e n d - o f - c h u n k ) ( ( t o initial))) (push S (lab s) ((setr s-place (inp-pointer))) ((jump initial/DEL))) (push NP (lab np) nil ((to initial))) (push VP (lab vp) nil ((to initial))) (push PP (lab pp) nil ((to initial))) (cat np& t (to initial)) (cat del t (to initial))) (initial/DEL (cat del t (close* (getr s-place)) (to initial)) (to initial t] (def-net

,( (s

'( v p - p l a c e n o - p p p p - p l a c e np-place)

(pop t (is-def (Y))(close (to S/ t (setr no-pp 0)))

(inp-pointer)))

(S/ (intr nil ( e n d - o f - c h u n k ) ( ( t o S/))) (Bush PP (and (lab pp) (le (getr no-pp) 2)) ((and (gt (getr no-pp) 0) (close* (getr p p - p l a c e ) ) ) (setr p p - p l a c e (inp-pointer)) ) ((setr no-pp (addl (getr no-pp))) (jump S/))) (abort "more than 2 PPs in S" (lab pp) ) (cat np& t (to S/NP&)) ;(s (pp & pp) ..) (cat del (gt (getr no-pp) 0) (close* pp-place) (setr no-pp l) (to S/)) (abort "DEL cannot appear at b e g i n n i n g of sent" (lab del)) (jump S/NP& t] (S/NP& (intr nil ( e n d - o f - c h u n k ) ( ( t o S/NP&))) (push NP

;(s .. (np & np) ..) (cat del (lab del) (close" (getr np-place)) (to S/NP&)) (abort "too many NPs before a VP" (lab np]

(lab np) ((and (getr pp-place) (close* (getr pp-place))) (setr np-place (inp-pointer))) ((to S/NP))) ;here we can allow PPs after an NP! (push VP (lab vp) ((and (getr pp-place) (close* (getr pp-place)))) ((jump S/OUT))) (abort =no NP or VP in the input sentence" t) (jump S/NP t]

(s/vP (cat del (lab del) (close* (getr vp-place)) (jump S/VP/DEL)) (jump S/OUT t] (S/VP/DEL ;standing at 'del' and looking ahead (abort "del at EOS?" (ge (next-one (inp-pointer)) sent-len)) ; the above is a test for eos (intr nil (null (look-ahead i)) ((lump S/VP/DEL))) (to S/NP (eq (look-ahead l) 'vp)) (jump S/OUT t] ;exit: it must be an s (S/OUT (pop t (end-of-sent) (close*)) (pop t t]

(S/NP (abort "not enough VPs in S" (end-of-sent)) (intr nil (end-of-chunk)((to S/NP))) (push VP (lab vp) ((setr vp-place (inp-pointer)) ;if there is a del (close* (getr np-place))) ;close the preceding NP ;and everything in it ((jump S/VP)))

397