Learning RDBNs for Activity Recognition
Cristina Manfredotti Department of Computer Science University of Regina Regina, Saskatchewan S4S 0A2, Canada
[email protected] Sandra Zilles Department of Computer Science University of Regina Regina, Saskatchewan S4S 0A2, Canada
[email protected]
Howard Hamilton Department of Computer Science University of Regina Regina, Saskatchewan S4S 0A2, Canada
[email protected]
1 Introduction Informally, activity recognition refers to the identification of an activity involving one or more objects across time. Activities often involve multiple objects, and the relation (or interaction) between them is of predominant importance for recognizing their activity. Our approach to activity recognition of multiple interacting objects is based on a model composed of: (1) a transition part that models the dynamic aspects and allows to predict the state and (2) a relational part that models the interaction between objects. Previous researchers have examined the problem of learning either a model for dynamic interacting objects, where the interaction is constrained to be of particular types [1], or a probabilistic relational model, where neither objects nor their relations can change during time [2]. To the best of our knowledge, this is the first approach to learning a model for both the dynamic aspects of interacting objects and the relations between them. We do so by learning Relational Dynamic Bayesian Networks (RDBNs), which are probabilistic models that extend Probabilistic Relational Models [2] to model dynamic objects [3]. In this paper, we address the problem of learning RDBNs from lowlevel data about individual objects in the environment. We assume relations between objects are unknown; our approach first detects these relations, then learns a model to recognize them and finally uses the acquired knowledge to learn a model of the activities. We do so by introducing a multi-layer approach that considers different levels of abstraction: attribute values, single-object activities and relations between objects, atomic activities and complex activities.
2 Our approach Sensors only provide sequences of the objects’ attribute values, however we need to learn models from them that represent individual as well as interactive behaviour of objects. To do that, we define single-object activities as the class of changes in attribute values concerning an object alone and relations as the degree of similarity between two (or more) objects’ attribute values at a given time step. Then, we group together single-object activities and relations that involve the same set of objects into atomic activities, defined as activities of only related objects. Finally, we define complex activities as classes of sequences of atomic activities. The state of the world is the vector of the attribute values of the objects in the world, their complex activities and the “sub-activities” they are involved in. For each of these sub-activities we learn two probabilistic models: one that gives the probability of the sub-activity given the elements it abstracts from and another that gives the transition probability of the elements given the sub-activity. These 1
probabilistic models together with the networks shown in Fig. 11 form an RDBN that can be used to make (online) activity recognition.
(a) Subsequent levels of abstraction from data to complex activity models and the probabilistic models learned at each layer.
(b) The Bayesian network that encodes the overall transition model.
Figure 1: The two Relational Bayesian Networks together define an RDBN.
3 Experiments and conclusion The abstraction presented before is independent of the actual algorithm used for leaning. In our implementation, we used a method based on clustering: at each layer we learn a mixture of Gaussians (MOGs) from the classes obtained clustering the elements at the previous layer: e.g., to learn a model for atomic activities we cluster the data classified into relations and single-object activities and then we fit the clusters obtained with an MOG. In order to learn the transition probability of atomic activities given a complex activity and the probability of a complex activity given a sequence of atomic activities, we learn a Markov Chain (MC) for each of the complex activities and build a system of parallel MCs. Experiments in the domain of sea navigation show that our approach successfully learns a model for the complex interactions between vessels that can be used to recognize their activities. In particular, we compared the performance of our system for the recognition of the “rendezvous” between two ships, with the performance of a Hidden Markov Model (HMM) that has been defined using domain knowledge. Our multi-layer approach distinguishes between positive (“rendezvous”) and negative examples much better than an HMM (HMM F-measure .6824 and our method .8718). Direction for future work include an extension of our approach to situations in which data come from noisy sensors.
References [1] Galata, A., Cohn, A.G., Magee, D.R., Hogg, D.: Modeling interaction using learnt qualitative spatiotemporal relations and variable length markov models. In van Harmelen, F., ed.: ECAI, IOS Press (2002) 741–745 [2] Friedman, N., Getoor, L., Koller, D., Pfeffer, A.: Learning probabilistic relational models. In Dean, T., ed.: IJCAI, Morgan Kaufmann (1999) 1300–1309 [3] Manfredotti, C.E.: Modeling and inference with relational dynamic bayesian networks. In Gao, Y., Japkowicz, N., eds.: Canadian Conference on AI. Volume 5549 of Lecture Notes in Computer Science., Springer (2009) 287–290 1 In Figure 1 st is the set of attribute values at time step t, r(Oi , t) is the set of the relations, {e(oi , t)}oi ∈O is the set of the single-object activities, {b(Oi , t)}Oi ∈O represents the atomic activities and a(Oi , t) the complex activity in the world.
2
