a parallel model for multimedia retrieval based on ... - CiteSeerX

International Workshop on Systems, Signals, and Image Processing (IWSSIP98), Zagreb, Croatia, June 3-5, 1998. For other papers of the PDC group at GWU, goto http://www.seas.gwu.edu/seas/eecs/Research/Parallel-Distributed/

A PARALLEL MODEL FOR MULTIMEDIA RETRIEVAL BASED ON MULTIDIMENSIONAL SIGNAL STRUCTURE Sanan Srakaew, Nikitas A. Alexandridis, Punpiti Piamsa-nga, and George Blankenship Department of Electrical Engineering and Computer Science, George Washington University, Washington DC, 20052 USA G. Papakonstantinou, P. Tsanakas, and S. Tzafestas Department of Electrical and Computer Engineering, National Technology University of Athens, Athens, Greece Abstract: In this paper, we propose a parallel, unified model for the indexing and retrieval by content of multimedia data using multiresolution processing. Each multimedia data type can be represented by k-dimensional (k-d) signals in the spatio-temporal domain. [1] These k-d signals are separated into small blocks and then formed into a multidimensional tree structure, called a k-tree. Using the k-tree structure, the retrieval time improves while the retrieval accuracy remains relatively constant. Moreover, since we can realize all types of multimedia data using the same k-tree data structure, the data indexing and retrieval algorithms are uniform. We demonstrate a parallel algorithm that can be used with this uniform model. The parallel processing algorithm improves the retrieval time. We evaluated the performance results of multimedia database queries using a Beowulf-class cluster of workstations. The experimental results indicate that the retrieval accuracy improves with the utilized tree depth and has a minimal impact on the retrieval time. Keywords: k-tree model, multidimensional signals, similarity search, multimedia data retrieval, and parallel processing.

1. INTRODUCTION Multimedia data, such as audio, images, and video clips, are composed of a variety of characteristics and data structures. Audio is a one-dimensional signal of a fixed-frequency sampling in temporal domain. An audio signal in digital format is represented as an encoding of the analog signal. Image is a two-dimensional signal, where both dimensions of the data are in the spatial domain. The resolution of an image depends on the sampling rate. Video data is a multidimensional composite signal in the temporal domain. It is a temporal synchronized signal of motion picture and its associated audio. Data processing of the different types of multimedia signals exhibits the familiar trade-offs; one must decide whether data quality, storage requirement, or computation speed plays the crucial role. In this paper, we exploit a parallel, unified model with an indexing and retrieval algorithm that is extensible for multiple data types. [2] The indexing algorithm is based upon the transformation of a feature by use of numerical computation. The data transformation is performed using a mathematical discrete transform, such as DCT, Wavelet, Fourier, [3] or histograms. [4] This same model can be applied to different features, different media, and different mathematical transformations. Data searching of a multimedia database is an exhaustive time-consuming task. We overcome the time complexity by exploiting parallel searching using a cluster of workstations. We apply spatial parallelism in a single-feature searching operation. We evaluated our parallel model on a Beowulf-class cluster of workstations. The experimental results show that the k-dimensional data structure allows users to adjust linearly between the precision of results and the retrieval time. The model achieves a linear speedup.

2. CONTENT-BASED MULTIDIMENSIONAL SIGNAL RETRIEVAL ALGORITHM Content-based retrieval of multidimensional signals is done by comparing features extracted from the input query with features extracted from every record in the database. The features of a multidimensional signal are subjective information. They are characteristics that are used to distinguish one signal from others. A 2-dimensional signal, such as an image, is characterized by features such as color, texture, and intensity. The basic algorithms for the searching of data in each of the different domains are quite similar. A matching search requires that the index key (defining feature) be unique and matched to the query. Exactly matched searching requires exhaustive comparisons that are inefficient and unsuitable for multidimensional signals; similarity searching is more appropriate. A similarity-search re-orders the database by distance between each record and the query; the result is selected from the ranking. Figure 1 shows a block diagram of a similarity search.

Extracted Feature of Query (q1)

d(q1,rec1)

D recN

d(q1,rec2) Sorting

rec2

Distance Computation

rec1*

Ranking Results

d(qN,recN) * reci = feature of image i

Figure 1. The distance computation between query and records in the database

3. THE K-TREE PARALLEL MODEL 3.1 The k-tree model A k-tree is a directed graph; each node has 2k incoming edges and one outgoing edge with a balanced structure. [2] A k-tree is a binary tree for 1-dimensional data and a quadtree for 2-dimensional data. Exploiting a k-tree brings three main benefits. First, the k-tree holds the information of spatio-temporal data on the tree structure itself. It reduces distance computation time to a comparison between two tree nodes. Second, a k-tree can accelerate multiresolution processing by calculating small, global information first and then large, local information when precise resolution is needed. Third, the data on a k-tree is unified since only the degree of the tree changes, while the processing algorithm and data structure remain invariant. Therefore, an algorithm for a particular type of feature can be reused for a feature of another media type.

3.2 Parallelism in the search In Figure 2 , we show a parallel search algorithm using a single feature. Prior to use the database is distributed to the workstations in the cluster. At the first stage (T1), each workstation performs a feature based comparison between the query and the data in its assigned records. The output from each processor is a list of distances. At the second stage (T2), the results are sorted in parallel to create the overall ranking of the database records. The following algorithm is executed on each processor in parallel. Each processor works on different portion of the database. Then, the sorted results are combined to the final result. Algorithm Search (IN Query, IN FeatureInDatabase[N], OUT record(distances[N],FeatureNumber[N])) Begin 1. Extract the interest Feature from the Query. 2. For each record i in [1..N] of FeatureInDatabase do 3. Find the distances[i] between Feature and FeatureInDatabase[i] 4. FeatureNumber[N]=i 5. End For 6. Sort all the records of (distance[N],feature[N]) in ascending order of distance[N] End

Algorithm 1. Algorithm to find ranked results To exploit the k-tree model, we apply a histogram-based approach to the extracted feature of images. [5] The histogram value is the index of the input images. In this paper, we examine two types of histograms: color and textures. The color histogram is constructed by counting the number of pixels of each color in a particular area. The k-tree is a quadtree of color histograms. The histogram of textures is constructed by assigning areas of the image a texture index. Each index is a 14-dimensional tuple of means and variances generated by a wavelet sub-band (2 iterations, 7 subbands). For example, an image of size N x N pixels can be evenly divided into M = q2 blocks of size p x p pixels, where p = N/q; therefore, the image is represented by a set of M indices. The quadtree representation contains M leave nodes and made up of log4 (M) levels.

Spatial

Temporal T1

T2 List of Distances

P1 D1

F1

Extracted Feature of Query

List of Distances

P2

P1

P2

Ranking Results

D2 List of Distances

PN

PN

Database: D = D1+D2+…+DN

DN

Processors: P1, P2, …, PN

Parallel Sorting and Merging

Distance Computation

Feature: F1

Figure 2. A parallel ranking of a single-feature search

4. EXPERIMENTS AND RESULTS 4.1 Machines Our architecture is based on the concept used in the NASA Beowulf project. [6] The Beowulf project attempts to exploit parallelism using a network of lost-cost workstations. The network consists of 7 Pentium II processor-based PCs with the Linux operating system connected through a 100Mbps Ethernet. [7] The Message Passing Interface (MPI) library is used as the interconnection mechanism. [8]

4.2 The searching results We compare the results of querying images by two features: color and texture. The extracted features are derived from database images of 128x128 pixels; each is evenly divided into 64 blocks. The quadtree of histogram for each image is made up of 3 levels; 64 leaf nodes. The results shown in Figure 3(a) depict the response time of the system as a function of the number of processors used to perform the ranking; the selection features are color and texture. Figure 3(b) shows speedups of the system when color and texture are used as features. The computation time decreases significantly as the number of processors used to perform the computation increases. The achieved speedups are superlinear because each processor has more system resources available at the run-time; the smaller number of processors, the more frequent trashing occurred. Figure 4(a) and (b) depict the top-twenty output images on the sorted list when color and texture are used as selection features, respectively. 26

1400

Color

Texture

Color

Texture

21 1000

Speedup

Response Time (second)

1200

800

600

16

11

400 6 200

1

0 1

2

3

4

5

Number of Processors

(a) Response Times

6

7

1

2

3

4

5

Number of Processors

(b) Speedups

Figure 3. Response Times and Speedups; the selection features are color and texture.

6

7

Input Query

Input Query

Output

Output

(a) Search result using color as a selection feature

(b) Search result using texture as a selection feature

Figure 4. Perception Outputs; the selection features are color and texture.

5. CONCLUSION We introduce a parallel model for multimedia data retrieval by content using a multidimensional, multiresolution structure. The model allows the extension of the system for the new types of data, new techniques, and new types of interest contents with less effort. The experimental results show that multiresolution processing can reduce retrieval time, maintain the accuracy, and exploit parallelism.

REFERENCES [1] Z. Kemp, Multimedia and spatial information systems, IEEE Multimedia, 2(4), 1995. [2] P. Piamsa-nga, N. Alexandridis, G. Blankenship, G. Papakonstantinou, P. Tsanakas, and S. Tzafestas, A Unified Model for Multimedia Retrieval by Content, International Conference on Computer and Their Application (CATA98), 1998. [3] S. Subramanya, R. Simha, B. Narahari, and A. Youssef, Transform-based indexing of audio Data for multimedia databases, International Conference on Multimedia Computing System, 1997. [4] J. R. Smith and S.-F. Chang, SaFe: A General Framework for Integrated Spatial and Feature Image Search, IEEE Workshop on Multimedia Signal Processing, 1997. [5] P. Piamsa-nga, N. Alexandridis, S. Srakaew, G. Blankenship, G. Papakonstantinou, P. Tsanakas, and S. Tzafestas, “Muti-feature Content-based Image Retrieval, IASTED International Conference on Computer Graphics and Imaging (CGIM98), 1998. [6] T. Sterling, D. Becker, D. Savarese, M. Berry, C. Reschke, Achieving a Balanced Low-Cost Architecture for Mass Storage Management through Multiple Fast Ethernet Channels on the Beowulf Parallel Workstation, Proceedings of International Parallel Processing Symposium, 1996. [7] P. Chalermwat, N. Alexandridis, P. Piamsa-nga, and M. O'Connell, Parallel image processing on heterogeneous computing network systems, International Conference on Image Processing, 1996. [8] T. El-Ghazawi, P. Chalermwat, P. Piamsa-nga, A. Ozkaya, N. Speciale, and D. Wilson, PACET: PC-parallel architecture for cost-efficient telemetry processing, IEEE Aerospace Conference, 1998.