done during the analysis of the first one (the pot on the floor), positioning. a g a i n t h e p o t i n a c c o r d a n c e t o t.he new i n f o r m a t i o n . REFERENCES.
SOME BASIC MECHANISMS FOR COMMON SENSE REASONING ABOUT STORIES ENVIRONMENTS*
Giovanni A d o r n i , Mauro Di Manzo and Fausto G i u n c h i g l i a I n s t i t u t e of E l e c t r o t e c h n i c s , U n i v e r s i t y of Genoa V i a l e F. Causa 13, 16145 Genoa, I t a l y ABSTRACT This work is concerned w i t h the problem of r e a soning about p h y s i c a l o b j e c t s and t h e i r p o s i t i o n s in the space. The main goal is to develop a system which is able to "imagine" the scene in which a s t o r y takes p l a c e , i n f e r r i n g by means of s u i t a b l e r u l e s the items which are not e x p l i c i t l y named, in order to v e r i f y the relevance of knowledge about the p h y s i c a l environment in the process of s t o r i e s understanding. The s p e c i f i c t o p i c covered by t h i s work is the problem of o b j e c t s i n s t a n t i a t i o n and a set of r u l e s f o r i n f e r r i n g the existence of an o b j e c t is proposed. I INTRODUCTION The problem of processing v i s u a l i n f o r m a t i o n can be approached under several p o i n t s of view and w i t h d i f f e r e n t g o a l s . F i r s t l y , the concept i t s e l f o f " v i s u a l i n f o r m a t i o n " can cover a wide range of i n f o r m a t i o n types, from elementary v i s u a l patterns to the knowledge about p h y s i c a l o b j e c t s and t h e i r o r g a n i z a t i o n in complex scenes, possibly i n v o l v i n g the i n t e r p r e t a t i o n of actors goals from t h e i r movements. Secondly such a knowledge can be h a n d l ed, f o r i n s t a n c e , w i t h the aim of supporting a r e c o g n i t i o n process or w i t h the goal of v e r i f y i n g the r o l e of imagination in the process of s t o r i e s understanding. This second goal d i d not receive in the past the same a t t e n t i o n deserved to the f i r s t one. Basic works are due to (Boggess, 1979), (Heskovits, 1980), (Keirsey, 1978), (Lehnert, 1978), (Sondheimer, 1976), (Waltz, 1980), and to the "gestaltisten" ( K a n i t z a , Legrenzi and Meazzini, 1975), but up to now the problem has been only scratched. In t h i s work we are faced w i t h high l e v e l v i s u a l i n f o r m a t i o n , t h a t is the r e p r e s e n t a t i o n and manipul a t i o n o f knowledge about p h y s i c a l o b j e c t s , t h e i r p o s i t i o n s in a given environment, a c t i o n s i n v o l v i n g t h e i r use and so on. Our main long term goal is to analyze how the i n f e r e n t i a l a c t i v i t i e s r e q u i r e d to understand a s t o r y are a f f e c t e d by the knowledge about the physical c h a r a c t e r i s t i c s of the e n v i r o n ment of the s t o r y i t s e l f , what s p e c i f i c reasoning r u l e s are used to handle p h y s i c a l o b j e c t s and how * Work supported by M . P . I , under Grant 27430/81.
such a reasoning c a p a b i l i t i e s can be implemented in an "anthropomorphic" r o b o t . The f i r s t step t o wards the i n t e g r a t i o n of what we c a l l " v i s u a l knowl edge" i n t o a general purpose i n f e r e n t i a l system c o n s i s t s of b u i l d i n g the scene sketched in a s t o r y . This generation a c t i v i t y requires a basic c a p a b i l i ty of i n s t a n t i a t i n g and p o s i t i o n i n g o b j e c t s , which can seem a t r i v i a l problem but needs a s u r p r i s i n g l y large amount of knowledge, due to the f a c t t h a t most of the scene is u s u a l l y not d i r e c t l y described i n the s t o r y . In the f o l l o w i n g we w i l l discuss some basic i n f e r e n t i a l r u l e s r e s t r i c t e d t o the case o f s t a t i c scenes and d e a l i n g w i t h " l o c a l r e a s o n i n g " , t h a t is reasoning based on knowledge which can be e x t r a c t e d from a s i n g l e c o n c e p t u a l i z a t i o n or a s i n g l e spatialr e l a t i o n s h i p between two o b j e c t s . This set of r u l e s is under implementation in a conversational system, c a l l e d NAUSICA, which uses I t a l i a n language as communication mean and a conceptual dependency l i k e formalism f o r i t s i n t e r n a l conceptual represen t a t i o n ( A d o r n i , A n s a l d i , Di Manzo and S t r i n g a , 1981), ( A d o r n i , Boccalatte and Di Manzo, 1982), (Schank, 1975). II PART-OF INSTANTIATION MECHANISM The PART-OF r e l a t i o n is used to express a f u n c t i o n a l connection between o b j e c t s , as, f o r instance " l e g PART-OF t a b l e " , "branch PART-OF t r e e " , " l o c k PART-OF door". The existence of a PART-OF r e l a t i o n does not always imply a p h y s i c a l connection between the involved o b j e c t s ; however such a connection is necessary in order use the o b j e c t , which is PART-OF, in accordance to i t s normal f u n c t i o n a l c h a r a c t e r i s t i c s , and t h i s gives r i s e to the expecta t i o n of a physical, connection, if there are no s p e c i f i c reasons t o r e j e c t t h i s hypothesis. I n the f o l l o w i n g we present some examples of c o n d i t i o n s t h a t have been proved to be u s e f u l in order to accept or r e j e c t the i m p l i c i t i n s t a n t i a t i o n . From here on, if "A is PART-OF B", A w i l l be r e f e r r e d to as the sub-part o b j e c t and B as the whole o b j e c t . A. Consistency checks When the whole is i n s t a n t i a t e d , the p o s i t i o n of the sub-part w i t h respect to the whole can be i n f e r r e d from the s t r u c t u r a l d e s c r i p t i o n of the whole i t s e l f , even w i t h i n approximate c o n s t r a i n t s . The
G. Adorni et al.
p o s i t i o n of the whole must be in t u r n i n f e r r e d from knowledge about its typical locations in a given (or assumed) environment. Thus, f o r instance, the sentence "the branch on the r o o f " allows us to assume an outdoor environment because of the on r e l a t i o n and the roof o b j e c t , while the branch can be considered PART-OF a t r e e . A t r e e , in an outdoor environment, suggests something l i k e a garden (a garden can be i n f e r r e d from knowledge about the l i v ing environment of a t r e e ) , where i t s t y p i c a l p o s i t i o n is v e r t i c a l and in physical contact w i t h the ground. From the structural description of a t r e e we obtain a possible p o s i t i o n of the branch which is c o n s i s t e n t w i t h the r e l a t i o n "on the roof", given t h a t a roof is PART-OF a house, and a garden can be around a house. We w i l l c a l l t h i s kind of consistency check a POSITION check. Other consistency checks are r e l a t e d to object DIMENSIONS and SUPPORTING c a p a b i l i t i e s . B. Containers When the sub-part object is in x, where x is any object having the basic f u n c t i o n of s t o r i n g t h i n g s , the i n s t a n t i a t i o n of the whole object is o f t e n u n c e r t a i n . L e t ' s consider the sentence "the lock in the store-room": even if an i n s t a n t i a t e d door can be put inside a store-room, without f a i l ing the previously mentioned consistency checks, t h i s is not the scene commonly imagined; it is much more l i k e l y t h a t the l i s t e n e r thinks simply of a stored l o c k . C. Supports If we hear that "the engines were r o a r i n g on the t r a c k " , we probably i n s t a n t i a t e a number of cars r a c i n g or ready to s t a r t ; on the c o n t r a r y , i t i s u n l i k e l y t h a t the sentence "the engine i s r o a r i n g on the bench" suggests a car on the bench. In the second sentence the sub-part object is on an object having the f u n c t i o n o f g i v i n g support; i f there are not s p e c i f i c reasons to do the opposite, a d i r e c t support is supposed and the whole object i s not i n s t a n t i a t e d . 1ll
CONTAINED INSTANTIATION MECHANISM
There is a class of objects which usually need to be contained by some s p e c i f i c container. We w i l l r e f e r t o objects o f t h i s class a s " f l u i d " o b j e c t s . Liquids are a t y p i c a l example of f l u i d o b j e c t s , but also s o l i d objects made of a large number of s m a l l , disconnected parts e x i b i t s i m i l a r p r o p e r t i e s , at l e a s t from the p o i n t of view of the use of cont a i n e r s , and t h e r e f o r e they can c l a s s i f i e d as f l u i d s . When a f l u i d object is addressed,some s p e c i f i c containers must be i m p l i c i t l y i n s t a n t i a t e d , i f there are no e x p l i c i t references to them. The kind of con t a i n e r may depend on the type of f l u i d or on the assumed environment or both . There are however some c o n s t r a i n t s to f u l f i l l in order to make reasonable the i m p l i c i t i n s t a n t i a t i o n of a container.
73
A. Consistency checks Unlike what happens w i t h the PART-OF i n s t a n t i a t i o n mechanism, i f the f l u i d i s the subject of a s p a t i a l r e l a t i o n s h i p , then the supposed container usually inherits such a relationship; therefore consistency can be checked using those i n t e r p r e t a t i o n r u l e s which are t y p i c a l o f the i n volved r e l a t i o n . For instance, the phrase 'the beer on the t a b l e " implies a r e l a t i o n l i k e "a glass on the t a b l e " , which can be accepted, while in the phrase "the sugar in the l o c k " , the i n t e r p r e t a t i o n r u l e s of the in r e l a t i o n s h i p r e j e c t the hypothesis of "a sugar-basin in the l o c k " . B. Locations The i n s t a n t i a t i o n of a container is usually r e j e c t e d i f i t s i n f e r r e d l o c a t i o n i s very unusual, in our everyday experience. I f , f o r instance, we say "the coffee on the w a l l " , probably the l i s t e n e r w i l l not imagine the coffee inside a c o f f e e - p o t hung up on the w a l l ; it is much more l i k e l y to suppose t h a t , f o r some accident, coffee splashed up to the w a l 1 . C. Object c h a r a c t e r i s t i c s Some kind of f l u i d objects u s u a l l y do not lead to the i n s t a n t i a t i o n of s p e c i f i c c o n t a i n e r s . They are t y p i c a l l y " n a t u r a l f l u i d s " which e x i s t i n outdoor environments in a free s t a t e , or the r e mainder of some physical or manufacturing process ("the sand on the f l o o r " ) . The assumed environment can play a relevant r o l e . Some f l u i d s , f o r i n stance, are much more l i k e l y to r e q u i r e a s p e c i f i c container in an indoor than in an outdoor e n v i r o n ment ("the water on the t a b l e " against "the water on the s t r e e t " ) . IV STATE CONDITIONS Some objects require the presence of other obj e c t s , o f t e n depending on the environment, in order to mantain a good physical s t a t e . Such s t a t e c o n d i t i o n s are t y p i c a l of l i v i n g o b j e c t s ; so, if we say "the f i s h on the t a b l e " , some water must be i n s t a n t i a t e d in order to create a s u i t a b l e l i v i n g environment f o r the f i s h . In most cases the i m p l i c it i n s t a n t i a t i o n is based on the assumption t h a t the l i v i n g object is in a good physical s t a t e ; t h i s assumption is acceptable if there is no contrary evidence. Some environments, however, can lead to r e j e c t i o n of such an assumption, and t h e r e f o r e no i m p l i c i t i n s t a n t i a t i o n is r e q u i r e d : an example can be "the f i s h in the fishmonger's shop". State condi t i o n s can hold also f o r not l i v i n g o b j e c t s . The most common case is r e l a t e d to not-embedding assump t i o n s ; hence, if we say "the plane is hidden in the mountain", a cave is i n s t a n t i a t e d . V OTHER INSTANTIATION MECHANISMS be
There is a set of basic mechanisms which can used both to d i r e c t l y i n s t a n t i a t e objects and
74
G. Adorni et al.
to check
i n s t a n t i a t i o n s made by means of PART-OF or
CONTAINED
relations.
b r i e f l y discuss
In
this
t h r e e of them,
section
namely
we
will
t h e SUB-PART,
as well
t h e SUPPORT and t h e CONTENT.
because earth
When of
a
the
geometric
class
of
structure
objects
is
of
the
described,
prototype
usually
only
every
instantiation;
a
so
ations
must,
into
instantiated do
it.
for
a
if
table,
it
is in
for
"the
it
we
in
should
is
ed
the
are say
assumed since
can
contain
ber
of objects"
another i f one
in
the
the the
it. w i l l be i n s t a n t i a t -
the
(a
is
first
one
when
a
supporting
inferred;
lation
is
this
action
usually
asserted
or
assumed.
p o r t h y p o t h e s i s a number succeed, ject, and
involving
material so
on.
One
capability physical a
fly
of
on
t u r e on
the the
wall
intermediate (a n a i l C.
in
accept
on
results during
pot
the
the supported ob-
the
has has
support
object
to
supporter;
this
capability,
not. must
man t a i n
for
In
the
be
while
second
inferred,
t h e case o f t h e p i c t u r e o n
Many o b j e c t s , (living
expectations to
say
view
"living with
a
case,
instance, other
room"
suggests
typical
in
the w a l l ) .
which
a
"the
implicit sofa, is
so
on)
may cause
content.For
example,
particular
pieces
as
the
of armchairs, object
of
indoor
furniture.
p l a n t in
the
living
instantiation,
smal1
commonly
for
t a b l e s and e v e r in
such
a
room.
We
will
conclude
coordinate
use
of
the
of
implementation.
Let
us
room". nism, its
system
typical
the
in
one
simple
proposed at
example
mechanisms
the
current
of and
stage
phrase:
the
through
of
The
in the l i v i n g
t h e CONTENT mecha-
instantiation
accessories. position
"a plant
of a
structure
door,
windows
room
with
of the
room
and
s o on)
language
used a s
to
instantiate
another
table,
exists;hence
the
analysis
floor), t.he new
of
positioning information.
tive
models
COL ING.
for
Case
and D i Manzo.M.
computer v i s i o n "
in
"Cogni-
Proc.
9th.
i n f e r p r e t a t ion o f
P r o p o s i f i o n s " , T . R . 7 5 , Un i v e r s i t.y
Jllinois,Urbana,111inois,February
llep.kov i t.s, A.
"On
sitions"
Proc.
1980,
,L.
Analyzed".Riv.1nf.
"Computational
Engl i s h S p a t i al
4.
Stringa
Prague,Czechos J o v a k I a , J u 1 y , 1 9 8 ? , p p . 7 - 1 ? .
3. Boggess,L.0. of
and
Understanding System;
39-88.
Adorni,G.,Boccalatte,A.
in
the s p a t i a l 18th.
1 979.
uses of t h e p r e p o ACL.
Philadelphia,
pp.1-5.
D. Kani t s a , G . , L e g r e n z i , P . and Nleazzm i , P . C o g n i t i v e Processes
(Italian).
6. Keirsey,D.M. November'
o f Cal i f o r n i a , I r v i n e ,
"Representing physical
memory",T.R.131,Yale Conceptual
Amsterdam:
1975. T.R.
California
(1976)
in
1978.
Information Processing
"Spatial
machine
objects
University,Yale,May
North Holland P u b l i s h i n g ,
9. Sondheimer,N.K.
.
1975.
r e f e r e n c e and n a t u r a l
control",Int.J.Man-Mach.Stud.
329-336.
10.Waltz,D.L.
" U n d e r s t a n d i n g scene d e s c r i p t i o n s a s
event s i m u l a t i o n s " phia,
Mulino,
1978.
7.Lehnert,W.G.
language
Bologna:II
"A representation of space",
125,University
8,
the
l i v i n g room,
results
(dimensions,
giving
capabilities
consider The
of
table)
Manzo , M .
1anguage
3 (1981)
8.Schank,R.
VI CONCLUDING REMARKS
a
?.
an
if possible
Italian
(Mi I a n )
a pic
they d e f i n e e n v i r o n -
and
specific
sentence
results
if
garden
some
some
Therefore, room"
expecially
room, for
the
a
instance,
t h e same
mantained,
i n r e m o v i n g one o f
the
the pot in accordance to
"NAUSICA:NAtura1
sup-
t h e s u p p o r t i n g one
on
1. A d o r n i , G . , A n s a 1 d i , W . , Di
re-
Content
ments
y
of of
supported
with
wall
an
t h i s check i s c o n c e r n e d w i t h t h e
the
contact
To
weight
structure
of
two o b j e c t s
when
is
already
done (the
follows,as,
REFERENCES
o f c o n s i s t e n c y c h e c k s must
the
and
between
happens
objects
simply
The SUPPORT i n s t a n t i a t i o n mechanism may be a c t i is
(a
possibly
i n s t a n c e o f such sentence
object,
is on
The p r i n c i p l e o f "minimum num-
and
inferences
room)
followed,avoiding
plant,
new
relation
living
in-
in an
location
is on the t a b l e " ,
the
reference.
in an
( a b i g pot.
default
spatial
" t h e plant,
Support
vated
some
characteristics,
container
whose
another
Earth
fluid
proper
environment
it
again
B.
if
local
other-
ed.
a
new
describ-
is
requires
t.he e a r t h .
has
environment),
floor,
infer a
embedded
drawer
requires
instance,
we
container,
it
indoor for
spa-
the p o i n t of view
is generated.The plant-,
conditions,
environment
the
a
if
STATE
to
only
fixed
l i v i n g room a p l a n t i s supposed a l i v e ) ;
drawer
there
make
w e must
pencil
prototype,
(a
example,
if an optional
table
to
table",
the
not.
reasons
however,
For
vari-
are
instantiated
inferred
the
suppose
therefore,
which
sub-part
t a b l e usable as a
table; in
be
reasonable. is
optional
specific
mentioned;
must
pencil
sub-part of the wise
are
optional
instance)
explicitly which
relation
that
an
of
account,
there
Generally,
cases tial
unless
number
of
(in a
is
scene
s i t i o n s o f t h e p l a n t and door
taken
the whole
STATE c o n d i t i o n s g i v e a l s o t h e e x p e c t e d r e l a t i v e p o
few e l e m e n t s can b e assumed t o b e a l w a y s p r e s e n t i n be
i n t h e same way
under which
A.
Sub-part
a s t h e p o s i t i o n o f i t s c o n t e n t i s chosen b y
default;
1980,
in Proc.
pp.7-11.
18th.
ACL.
Philadel_
