Dan Bailey Yuliya Lierler Benjamin Susman. University of Nebraska at Omaha. Bailey, Lierler, Susman (UNO). PP-Attachment Problem Revisited. IWCS 2015.
Prepositional Phrase Attachment Problem Revisited: How VERBNET Can Help Dan Bailey
Yuliya Lierler
Benjamin Susman
University of Nebraska at Omaha
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
1 / 26
Outline
1
Introduction to PP-attachment
2
Tackling the Problem
3
PPATTACH System Explanation
4
PPATTACH System Evaluation
5
Conclusions
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
2 / 26
Outline
1
Introduction to PP-attachment Syntactic Parse Structures Prior Work and Dataset
2
Tackling the Problem
3
PPATTACH System Explanation
4
PPATTACH System Evaluation
5
Conclusions
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
3 / 26
Syntactic Parse Structures
Resolving Dependency Structures of Prepositional Phrases PREP-POBJ DOBJ
eat
spaghetti
with
meatballs
PREP-POBJ
DOBJ
eat
spaghetti
with
chopsticks
Widely available syntactic parsers have not reached human performance on preposition phrase attachment
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
4 / 26
Outline
1
Introduction to PP-attachment Syntactic Parse Structures Prior Work and Dataset
2
Tackling the Problem
3
PPATTACH System Explanation
4
PPATTACH System Evaluation
5
Conclusions
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
5 / 26
Prior Work and Dataset Prior work Ratnaparkhi et al. (1994) introduces a dataset used for comparative purposes for subsequent research Stetina and Nagao (1997) introduce the highest performing system, achieving accuracy of 88.10% (Human benchmark of 88.20%)
Basics of the Ratnaparkhi dataset Consists of tuples: (verb, noun1 , preposition, noun2 ) from the Penn Treebank Example: (eat, spaghetti, with, meatballs) Total of 23898 tuples Preposition Total % of R
All
of
in
to
for
on
from
with
at
as
by
23898 100
6503 27.2
3973 16.6
3005 12.6
2522 10.6
1421 5.9
1059 4.4
1049 4.4
780 3.3
564 2.4
526 2.2
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
6 / 26
Outline
1
Introduction to PP-attachment
2
Tackling the Problem Use of VERBNET Use of Selectional Restrictions
3
PPATTACH System Explanation
4
PPATTACH System Evaluation
5
Conclusions
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
7 / 26
Tackling the Problem Incorporate semantic knowledge Use lexico-semantic ontologies. VERBNET
(University of Colorado Boulder)
Groups verbs that share usage patterns and have semantic similarity Composed of frame syntax which apply to verb classes Frame syntax for a hit-verb class follows: A G E N T intControl
hit-class
PATIENT
{with}
the dog
with
the bat
the dog
with
the collar
I N S T R U M E N T concrete
Matching Example: I
hit
Non-Matching Example: I
Bailey, Lierler, Susman (UNO)
hit
PP-Attachment Problem Revisited
IWCS 2015
8 / 26
Outline
1
Introduction to PP-attachment
2
Tackling the Problem Use of VERBNET Use of Selectional Restrictions
3
PPATTACH System Explanation
4
PPATTACH System Evaluation
5
Conclusions
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
9 / 26
Tackling the Problem Selectional Restrictions (Katz and Fodor, 1963) Semantic common-sense restrictions that words impose Represented as a tuple [w, t, r , p] w is a word t is a thematic role that w allows r is a restriction on the role t p is a set of necessary prepositions for realizing t
Selectional Restrictions for hit frame Frame A G E N T intControl
hit-class P A T I E N T {with} I N S T R U M E N T concrete
Selectional Restrictions (hit, AGENT, intControl, ∅) (hit, PATIENT, ∅, ∅) (hit, INSTRUMENT, concrete, {with}) Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
10 / 26
Outline 1
Introduction to PP-attachment
2
Tackling the Problem
3
PPATTACH System Explanation Decision Procedure Selection Algorithm Features Explanation
4
PPATTACH System Evaluation
5
Conclusions
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
11 / 26
PPATTACH Decision Procedure
Goal Procedure Input “Ratnaparkhi”-style tuple Output Verb or Noun ⇒ Classification Problem
Given Annotated Data Ratnaparkhi dataset ⇒ Supervised Machine Learning
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
12 / 26
PPATTACH Decision Procedure
Weka (University of Waikato) Tools for data preprocessing, clustering, and classification We utilized the Logistic Regression Classifier Ratnaparkhi Tuples
Annotation
(hit, dog, with, bat) (hit, dog, with, collar) tuple3 ... tuplen
Verb Noun annotation3
Bailey, Lierler, Susman (UNO)
annotationn
⇒
Feature Vector
Annotation
[f1 , f2 , ..., fn ]1 [f1 , f2 , ..., fn ]2 feature-vector3 ... feature-vectorn
Verb Noun annotation3
PP-Attachment Problem Revisited
⇒
Weka Classifier
annotationn
IWCS 2015
13 / 26
Outline 1
Introduction to PP-attachment
2
Tackling the Problem
3
PPATTACH System Explanation Decision Procedure Selection Algorithm Features Explanation
4
PPATTACH System Evaluation
5
Conclusions
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
14 / 26
PP-Attachment Selection Algorithm
Features 1
Preposition: Outputs the preposition from the Ratnahparkhi tuple.
2
Verbclass: Outputs all verb classes associated with the verb using
3
VERBNET [noun1 , noun2 ]
4
VERBNET [noun2 ]
5
Nominalization
VERBNET
Feature Vector Tuple (hit, dog, with, bat) (hit, dog, with, collar)
Preposition
Verbclass
VERBNET [noun1 , noun2 ]
VERBNET [noun2 ]
Nominalization
with with
hit-18.1 hit-18.1
V 0
0 0
0 0
...
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
15 / 26
Outline 1
Introduction to PP-attachment
2
Tackling the Problem
3
PPATTACH System Explanation Decision Procedure Selection Algorithm Features Explanation
4
PPATTACH System Evaluation
5
Conclusions
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
16 / 26
Features Explanation VERBNET [noun1 , noun2 ] Search all verb-classes that include verb From these classes, extract all frame syntax of the form T H E M R O L E verb-class T H E M R O L E 1restriction1 {prep} T H E M R O L E 2restriction2
Extract selectional restrictions from the frame syntax of the form (verb, T H E M R O L E 1 , restriction1 , ∅) (verb, T H E M R O L E 2 , restriction2 , {prep})
Verify selectional restrictions
Example hit dog with bat Extracts selectional restrictions: (hit, P A T I E N T , concrete, ∅) (hit, I N S T R U M E N T , concrete, {with}) Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
17 / 26
Features Explanation VERBNET [noun2 ] Search all verb-classes that include verb From these classes, extract all frame syntax of the form T H E M R O L E verb-class
{prep} T H E M R O L E 2restriction2
Verify selectional restrictions (verb, T H E M R O L E 2 , restriction2 , {prep})
Example sell cars to buyers Extracts selectional restrictions: (sell, R E C I P I E N T , animate|organization, {to})
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
18 / 26
Features Explanation Nominalization Check if noun is derived from a verb using
WORDNET
Search all verb-classes that include noun1 ’s related verb From these classes, extract all frame syntax of the form T H E M R O L E verb-class
{prep} T H E M R O L E 2restriction2
Verify selectional restrictions (verb, T H E M R O L E 2 , restriction2 , {prep})
Example held talks with partners talks is derived from the verb talk Extracts selectional restrictions: (talk , C O -A G E N T , animate|organization, {with})
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
19 / 26
Outline
1
Introduction to PP-attachment
2
Tackling the Problem
3
PPATTACH System Explanation
4
PPATTACH System Evaluation PPATTACH PPATTACH+ with Specific Features
5
Conclusions
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
20 / 26
Evaluation Developed System: PPATTACH Multiple metrics for analyzing system performance: Baseline Consists of only the Preposition feature. Chooses the most likely attachment depending on the on preposition. PPATTACH- The Preposition feature and GENERIC
VERBNET / WORDNET
Features
The Preposition and Verbclass Features
PPATTACH The Preposition, Verbclass,
VERBNET / WORDNET
Features
Preposition
All
in
to
for
on
from
with
at
as
by
Baseline PPATTACH-
74.6 79.3 79.0 79.3
54.6 64.6 64.7 64.7
80.1 87.8 87.8 88.0
51.2 66.6 67.0 66.9
53.8 68.5 68.2 69.6
68.6 75.5 76.3 75.4
64.4 70.9 69.7 70.7
80.4 81.8 82.9 81.9
81.2 79.8 79.8 78.5
72.2 80.0 82.3 81.7
GENERIC
PPATTACH
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
21 / 26
Outline
1
Introduction to PP-attachment
2
Tackling the Problem
3
PPATTACH System Explanation
4
PPATTACH System Evaluation PPATTACH PPATTACH+ with Specific Features
5
Conclusions
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
22 / 26
Evaluation with additional with Specific Features
Developed System: PPATTACH PPATTACH+ PPATTACH + 3 with Specific Features 1 2 3
Instrumentality Adverbial Use Relational Noun
Preposition
All
in
to
for
on
from
with
at
as
by
Baseline PPATTACH-
74.6 79.3 79.0 79.3
54.6 64.6 64.7 64.7
80.1 87.8 87.8 88.0
51.2 66.6 67.0 66.9
53.8 68.5 68.2 69.6
68.6 75.5 76.3 75.4
64.4 70.9 69.7 70.7 72.0
80.4 81.8 82.9 81.9
81.2 79.8 79.8 78.5
72.2 80.0 82.3 81.7
GENERIC
PPATTACH PPATTACH+
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
23 / 26
Outline
1
Introduction to PP-attachment
2
Tackling the Problem
3
PPATTACH System Explanation
4
PPATTACH System Evaluation
5
Conclusions
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
24 / 26
Conclusions
Significance Full automated Incorporated lexico-semantic ontologies for prepositional phrase attachment
Future Work Add more preposition-specific features Integrate other ontologies in new features Integrate into syntactic parser
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
25 / 26
Conclusions
PPATTACH is Available http://www.unomaha.edu/nlpkr/software/ppattach/ Entire PPATTACH system Tutorial for Setup and Running PPATTACH IWCS 2015 Paper
Thank you for your attention Any Questions?
Bailey, Lierler, Susman (UNO)
PP-Attachment Problem Revisited
IWCS 2015
26 / 26
