Using Propositional Graphs for Soft Information Fusion Michael Prentice and Stuart C. Shapiro Department of Computer Science and Engineering, Center for Cognitive Science, Center for Multisource Information Fusion University at Buffalo, Buffalo, NY 14260 {mjp44,shapiro}@buffalo.edu
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
1 / 26
Outline 1
Overview
2
Syntax
3
Frame Semantics
4
Propositional Graphs
5
Propositionalizer
6
Fusing Propositional Graphs
7
Conclusions, Future Work, Acknowledgments
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
2 / 26
Overview
Soft Information Fusion Combine information from multiple natural language messages. Counterinsurgency domain. Message 12: 01/13/07 — Cell phone call from unidentified male in Adhamiya to unidentified male in Ramadi lasted just five seconds with the words “my brother sends greetings” spoken by originator of call. Message 14 extract: 01/14/07 — Originator of 1/13/07 cell phone call to Ramadi from Adhamiya has now been identified as Sufian Mashhadan. The recipient has been identified as Ziyad al-Obeidi. Some problems: Embedded in many messages. Same cell phone calls? from vs. originator to vs. recipient M. Prentice & S. C. Shapiro (UB)
Fusion 2011
3 / 26
Overview
Tractor
1
Syntactic Processing Produce dependency graph
2
Propositionalizer Map dependency graph to propositional graph
3
Contextual Enhancement Add relevant ontological and other background information
4
Fuse contextually enhanced propositional graphs
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
4 / 26
Syntax
Dependency Structure of Sentences Clauses Head: Verb Dependents: Adverbs Noun phrases Prepositional phrases
Prepositional Phrases Head: Preposition Dependents: Noun phrase Noun Phrases Head: Noun Dependents: Determiner Adjectives Nouns Prepositional phrases M. Prentice & S. C. Shapiro (UB)
Fusion 2011
5 / 26
Syntax
Dependency Parse for Message 12
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
6 / 26
Frame Semantics
Caseframes for Semantics
Based on “The Case for Case” [Fillmore, 1968] and The Berkeley FrameNet Project [Baker, Fillmore, & Lowe, 1998] Frame schematic representation of a situation with a set of participants and conceptual roles.
Eliminates syntactic differences. E.g. Sufian called Ziyad. Ziyad was called by Sufian. a call from Sufian to Ziyad
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
7 / 26
Frame Semantics
Structure of Frames
Frame set of frame elements semantic roles filled by entities of certain types.
Filler can be another frame. Core Frame Elements: “conceptually necessary component”
Non-Core (Peripheral) Frame Elements: Does not “introduce additional, independent or distinct events”
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
8 / 26
Frame Semantics
Contacting Frame Core Elements Communicator, type: Sentient Addressee, type: Sentient Communication ...
Non-Core Elements Medium Time, type: Time ...
E.g. Time: 2007-01-13 Medium: cell phone Communicator: Sufian Mashhadan Addressee: Ziyad al-Obeidi Communication: “My brother sends greetings.” M. Prentice & S. C. Shapiro (UB)
Fusion 2011
9 / 26
Frame Semantics
Frame for Message 12
Time: 2007-01-13 Communicator: Some Sentient Message:
Duration: 5 seconds Entity: Medium: Cell phone Communicator: Some male in Adhamiya Addressee: Some male in Ramadi Communication: “My brother sends greetings.”
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
10 / 26
Propositional Graphs
Propositional Graphs A Knowledge Representation (KR) labeled directed acyclic graph with formal syntax and formal semantics
Atomic Node Corresponds to individual constant No outgoing arcs Denotes entity in domain
Molecular Node Corresponds to functional term Outgoing arcs labeled with argument position (role) Compositional semantics Denotes Entity in domain Possibly a proposition M. Prentice & S. C. Shapiro (UB)
Fusion 2011
11 / 26
Propositional Graphs
Basic Principles
Comprehensiveness Every entity, person, category, property, value, etc. Every proposition, belief, fact, etc. is represented by a node.
Uniqueness Principle No two nodes with same ID. No two molecular nodes with same labeled arcs to same nodes. No two nodes representing the (obviously) same domain entity. Base case for fusing propositional graphs
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
12 / 26
Propositional Graphs
SNePS KR System
Logic-based Frame-based Propositional graph-based Each view supports a different style of inference (not covered in this talk) Uses Arbitrary and Indefinite terms [Shapiro, 2004] E.g., (some ind1 () (Instance ind1 Male) (Location ind1 Ramadi))
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
13 / 26
Propositional Graphs
SNePS Graph::Frame::Logic
Atomic node :: atomic symbol :: individual constant Node ID = atomic symbol
Molecular node :: frame :: functional term Node ID = wfti [!]
Arc label :: frame slot :: argument position Uniqueness Principle No two nodes with same ID No two molecular nodes with same labeled arcs to same nodes Base case for fusing SNePS propositional graphs
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
14 / 26
Propositional Graphs
Lowest Parts of Propositional Graph for Message 12
some Male in Ramadi; some Male in Adhamiya; some Sentient; 5 seconds
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
15 / 26
Propositional Graphs
Contacting in Propositional Graph for Message 12
Some Male in Ramadi communicated “my brother sends greetings” to some Male in Adhamiya. Note 3-ary relation.
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
16 / 26
Propositional Graphs
Cell Phone in Propositional Graph for Message 12
The contacting was via cell phone and lasted 5 seconds. Note nesting of functional terms.
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
17 / 26
Propositional Graphs
Propositional Graph for Sender of Message 12
Some Sentient said that it lasted 5 seconds. Note pedigree information. M. Prentice & S. C. Shapiro (UB)
Fusion 2011
18 / 26
Propositional Graphs
Final Propositional Graph for Message 12
Message 12 was sent on 2007-01-13 M. Prentice & S. C. Shapiro (UB)
Fusion 2011
19 / 26
Propositionalizer
Propositionalizer
FrameNet DB
Dependencies & Named Entities
FrameNet Tagger
Syntax-Semantics Mapper
Syntax-Semantics Translation Rules
Propositional Graph
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
20 / 26
Propositionalizer
Example Syntax-Semantics Mappings
Sample mappings from dependency graph to propositional graph. M. Prentice & S. C. Shapiro (UB)
Fusion 2011
21 / 26
Fusing Propositional Graphs
Propositional Graph for Message 14
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
22 / 26
Fusing Propositional Graphs
Fused Propositional Graph
Given common Medium, Time, Locations, Data Association should fuse calls, communicators, and addressees.
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
23 / 26
Conclusions, Future Work, Acknowledgments
Conclusions
Tractor analyzes an English message builds a dependency graph and then a propositional graph.
Propositional Graphs Are a Knowledge Representation Based on FrameNet frames Can represent n-ary relations Can represent meta-information Can represent pedigree Support Data Association by graph matching
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
24 / 26
Conclusions, Future Work, Acknowledgments
Future Work
Automating tagging of words with FrameNet frames Automating Syntax-Semantics Mapper Designing test and evaluation measures
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
25 / 26
Conclusions, Future Work, Acknowledgments
Acknowledgements
Multidisciplinary University Research Initiative (MURI) Grant (Number W911NF-09-1-0392) for “Unified Research on Network-based Hard/Soft Information Fusion” Issued by the US Army Research Office (ARO) under the program management of Dr. John Lavery.
Center for Multisource Information Fusion Dr. James Llinas, Executive Director
Contact information Michael Prentice (email:
[email protected]) Dr. Stuart C. Shapiro (email:
[email protected])
M. Prentice & S. C. Shapiro (UB)
Fusion 2011
26 / 26
