2012 15th International IEEE Conference on Intelligent Transportation Systems Anchorage, Alaska, USA, September 16-19, 2012
A System for Storing and Retrieving Huge Amount of Trajectory Data, Allowing Spatio-Temporal Dynamic Queries Antonio d’Acierno1 , Marco Leone2 , Alessia Saggese
2
and Mario Vento
2
Abstract— In the framework of intelligent traffic surveillance, we propose a system for efficiently storing and retrieving moving objects’ trajectories extracted from surveillance cameras. We index spatio-temporal data by using a method based on off-the-shelf (widely available) bi-dimensional indexes and enhanced by a segmentation stage. The proposed system does not restrict the choice of the parameters of range queries at query time, unlike clustering and similarity-based methods do. The experimental results, obtained on a standard PC-based system both on a well known real-world trajectory dataset and on synthetic data, confirm the efficiency of the proposed approach.
I. I NTRODUCTION In the last decades we have witnessed a growing need for security in many public environments and particular attention has been devoted to traffic scenarios, which are daily interested by accidents, traffic queues, highway code violations, driving in the wrong lane or on the wrong side, and so on. In order to collect traffic data, cameras represent a suitable solution for their relative low cost of maintenance, the possibility of installing them virtually everywhere and, finally, the capability of analyzing more complex events. In the context of camera-based traffic surveillance tools, we are developing a system for extracting, storing and querying moving objects’ trajectories from surveillance cameras. The system is composed by three main components (Figure 1): the Video Analytics Engine (VAE), starting from the acquired video stream, detects the objects in the scene and then extracts the objects’ trajectories together with their appearance information; as the tracking algorithm is outside of the scope of this paper, the reader can refer to [1] for a detailed description of our approach. The Storage Engine (SE) is in charge of storing the extracted data by means of suited indexing strategies while a Retrieval Engine (RE) allows to retrieve salient data for visualization and statistical purposes on the basis of the specific queries submitted by the user through a Graphical User Interface (GUI). The scientific literature is recently enriching with papers dealing with the above mentioned phases or, in several cases, proposing whole architectures implementing a video-based analytics system. For example, in order to extract the motion trajectories of vehicles and pedestrians, different tracking
Fig. 1: An overview of the architecture of the proposed system.
algorithms have been proposed, as in [2] and [3]; many of these approaches focus on vehicles tracking and allow the traffic flow analysis in relation to the type of vehicle, properly using an object classification stage [4] [5]. Despite the relatively high number of papers on trajectory extraction and vehicle and pedestrian classification, it is worth pointing out that only a modest attention has been devoted to storing and retrieving systems able to cope with very large amount of trajectory data and sufficiently general to deal with the needs of different application domains. This is an important and not negligible feature, especially when considering crowded real world scenarios (like highway intersections, city crossroads and important junctions). In these cases it is required that billions of trajectories must be stored and that, on this wide database, the user must be able to submit complex queries involving geometric and temporal data. An example of these systems is [6], which proposes
1 Institute of Food Sciences, National Research Council of Italy, Avellino, Italy -
[email protected] 2 Department of Electronic and Information Engineering (DIEII), University of Salerno, Italy -
{mleone,asaggese,mvento}@unisa.it 978-1-4673-3063-3/12/$31.00 ©2012 IEEE
989
a learning-based framework for capturing, modeling and indexing the spatio-temporal relations of the moving objects. Optimal route planning is dealt in [7], presenting a web interface which supports shortest path or shortest time queries by use of the underlying geometrical structure. Kumar et al. [8] present a rule-based framework for activity detection in traffic videos and propose a novel Bayesian network approach for robust classification; they also handle with behavior recognition, analyzing the interactions between the moving objects and the scenario. Other approaches aim at building activity models by applying clustering algorithms [9][10].
II. S YSTEM A RCHITECTURE The tracking algorithm [1] produces the set of objects moving in the scene, each one represented as the triple: Ok =< I k , T k , Ak > where I k is the object’s identifier, T k is its 3D trajectory and Ak is the vector of characteristics representing additional information about object’s appearance (as color, type, size, etc). T k is in turn defined as a sequence of spatio-temporal points: T k =< P1k , P2k , ..., PNk >
The systems above cited are good examples of the potentiality of spatio-temporal queries in a given domain, but they suffer from the fact that the parameters characterizing the queries are mostly pre-determined and cannot be chosen by the user at query time. Their rationale is to have a system architecture devised and optimized for supporting a bunch of queries, each one referring to a given spatial area; this is sufficient to solve the corresponding retrieving problem, but the capability of choosing at query time (i.e. exactly when the query is thought) the area in which we are interested in is neglected.
where the generic point Pik = (xki , yik , tki ) represents the spatial location (xki , yik ) of an object at the time instant tki . From now on, we will use the line segments model [12], each segment being the line connecting two consecutive points. A DST query allows to detect all those objects passing through a given spatial area in a given time interval. In the contest of a traffic management system, DST queries can be used for different retrieval purposes. Typical uses are: ”verify whether a given vehicle was crossing a given intersection between 4.30 pm and 5.30 pm yesterday”, or ”find all the vehicles passing by an area within San Peter’s Square on 14th March 2011 in the morning”. More specific information can be obtained by augmenting DST queries with additional information about objects’ appearance: ”find the number of red trucks leaving highway I59 from exit 14 running across the middle lane between 9-12 am yesterday”, or ”verify whether a given motorcycle was walking through a level crossing between 6-8 pm yesterday”. A key concept to be stressed is that the area and the time interval of interest are assumed to be defined at query time. Here, we think to the area as a rectangle with coordinates (xs , ys ) and (xe , ye ) while [ts , te ] are the starting and final time instants. According to this assumption, each DST query can be associated to a query box B (Figure 2a). Another query type, hereinafter named as Flow-DST (FDST), is well suited for analyzing the traffic flow in the observed scene; it allows to retrieve information of the type ”find the number of vehicles passing by a given area (dynamically defined at query time) on highway S14 each hour from 8 am to 6 pm yesterday”, or ”find the number of vehicles passing by a given toll-house each six hours during the last two days”. From a geometric point of view, a FDST can be seen as the application of various DST queries so as to obtain results at fixed time intervals, as shown in Figure 2b. A further type of query, called Multi-DST (M-DST), has been defined to retrieve more complex and structured information; examples of this query type are ”find the number of vehicles that have passed by a given intersection and then have exited by a given offramp two days ago” and ”find all the yellow vehicles passing by highway I-55 and then by Interstate 60 from 2 to 4 pm today”. As Figure 2c shows, this query typology can be seen as the application of two (or more) DST queries, typically having temporally successive query boxes.
While it is simple to imagine how much the flexibility degree of such a type of system can increase, it is not likewise to design a system architecture having these potentiality. The main contributions to this aim can be obtained by browsing the literature coming from the database field; for example, a widely adopted solution for bi-dimensional spatial indexing is based on R-trees [11], which hierarchically organize geometric bi-dimensional data representing each object using its Minimum Bounding Rectangle (MBR). Starting from Guttman’s pioneering paper, many other indexing schemes able to handle with spatio-temporal data have been proposed for many application contexts, most of which are optimizations of R-trees [12], [13], [14], [15]. However, the cited approaches, even presenting efficient solutions from different perspectives, are typically not widely supported, both in commercial and freely available products. For instance, PostGIS [16], the well known extension of PostgreSQL DBMS for storing spatial data, while supporting three (and even four)-dimensional data, does not support three-dimensional intersection and indexing operations. As a consequence, there is a strong interest in those methods which, even using off-the-shelf solutions, allow to solve the problem in the three-dimensional space [17] [18]. In this paper we present an efficient implementation of the SE which uses off-the-shelf solutions; in order to optimize the indexes’ efficiency, we also introduce a segmentation algorithm performed at loading time (Figure 1). Once data have been collected and properly stored, our system allows to efficiently solve, by means of the RE, Dynamic SpatioTemporal (DST) queries, e.g. queries finding all the trajectories passing through an area defined directly within the query (i.e. at query time). As it will be shown in the next section, other interesting kind of queries can be formulated in terms of specializations of DST queries. 990
(a)
(b)
(c)
Fig. 2: Geometric interpretation of different types of query: DST (a), F-DST (b) and M-DST (c).
tively xt and yt) plane the 3D MBR of T k . It is worth noting that the redundancy introduced by the three MBR projections is not dependent on the number of points in the trajectory and, therefore, has only a marginal impact on the spatial complexity, since it only requires the storage of six pairs of points. Assuming such a scheme, on each 2D plane we find the trajectories intersecting the corresponding 2D query box in a very efficient manner by using one of the available 2D indexes. Let Γxy , Γxt and Γyt be three sets of trajectories, each one defined as:
From a geometric perspective, in order to solve a DST query, we have to verify, for each trajectory, if at least one of its segments intersects the query box. The intersection between a segment and a box can be verified by using a clipping algorithm, a well-known family of algorithms widely used for identifying the portion of an image which is either outside or inside a picture. One of the most efficient ones is the Cohen-Sutherland Line Clipping Algorithm [19] which works, in its 3D formulation, by subdividing the plane into 27 regions by extending the faces of the query box. Despite its simplicity, the use of a clipping algorithm is not suited for handling large datasets: in fact, in the worst case, arising when a trajectory does not intersect the query box, all the trajectory segments must be processed, making this approach unfeasible for a large amount of trajectory data. It means that, in real applications, it is necessary to make use of more efficient approaches, as the ones using suitable indexing strategies, known as spatial indexes. Spatial indexes allow to efficiently perform queries involving geometry data types such as points, lines and polygons; a query in this case represents a spatial relationship among these geometric entities. Available commercial databases make use of very efficient spatial indexing techniques that, unfortunately, are typically restricted to deal with 2D data; therefore, it is worth representing an actual 3D problem in terms of (one or more) 2D sub-problems, so as to exploit at the best the available 2D indexing schemes. In the 3D space, given a trajectory T k and a query box B, it is straightforward to observe that, if T k intersects B, then the projection of T k on each coordinate plane also intersects the correspondent query box projection; this of course represents a necessary but not sufficient condition, as the opposite is clearly not true. Thus, if all projections of T k intersect the correspondent query box projections, we suggest to consider T k as a candidate to be clipped in the 3D space. According to the above considerations, for a given trajectory T k , we can also maintain [18] three different bi-dimensional MBRs: M BRxy (T k ), M BRxt (T k ) and M BRyt (T k ). M BRxy (T k ) (respectively M BRxt (T k ) and M BRyt (T k )) is obtained by projecting on the xy (respec-
Γsz = {T : M BRsz (T ) ∩ Bsz 6= ∅}
(1)
where Bxy , Bxt and Byt are the projections of the 3D query box B. The set Θ of the trajectories candidate to be clipped in the 3D space is therefore defined as: Θ = Γxy ∩ Γxt ∩ Γyt
(2)
It should be clear at this point that the entire system performance will strongly depend on the indexing phase and, as a consequence, on the capability to reduce the number of trajectories to be clipped in the 3D space. At a more detailed analysis, the selectivity of the indexes in each plane is related to the area of the corresponding MBR which, in turn, only depends on the trajectory geometry, so being (apparently) fixed. For this reason we introduced a segmentation stage aimed at increasing the selectivity of the indexes. Segmentation aims at subdividing each trajectory into consecutive smaller units, which we will refer to as trajectory units. The proposed algorithm aims at exploiting the characteristics of the available bi-dimensional indexes by decreasing the area of the projected MBRs of each trajectory unit by recursively working. Initially (that is at iteration 0), it assumes that the trajectory T k is composed by a single unit 0 U1k , that is split into a set of m consecutive smaller k units {1 U1k , . . . ,1 Um }; each of the 1 Uik is in turn inspected and, if the stop criteria are not satisfied, it is further split. Let us analyze how a generic unit (i−1) Uj = {P1 , . . . , Pm } is split into {i U1 , . . . ,i Un }; we first choose a split-dimension (sd) and a split-value (s∗ ). Assume, as an example and without loss of generality, that x has been 991
chosen as the split-dimension and let x∗ be the split-value. In addition, assume that x1 < x∗ . According to these hypotheses, i U1 is the set of the consecutive points lying on the left side of x∗ : i
U1 = {P1 , . . . , Pk }
(3) ∗
where Pk is the first point such that xk ≥ x . Then, the second unit will be formed by the sequence of consecutive points lying on the right side of x∗ : i
U2 = {Pk+1 , . . . , Pl }
(4)
where Pl is the first point such that xk ≤ x∗ . The inspection of (i−1) U ends when Pm is reached. According to the above considerations, the criteria for the choice of sd and s∗ play a crucial role. Since we aim at optimizing the indexing strategy, the proposed segmentation algorithm is based on the occupancy percentage on each 2D coordinate plane. Thus, with reference to the generic unit i Uj to be segmented, we calculate the three occupancy percentage values Oxy , Oxt and Oyt of as follows1 : Osz =
A(M BRsz (i Uj )) A(Vsz )
Fig. 3: A snapshot of the GUI.
∀ (k, i) ∈ ΘU { if k ∈ /
