High Performance Multimedia Database System Support For Image ...

High Performance Multimedia Database System Support For Image Processing F. Andres, K. Ono

A. Makinouchi, K. Kaneko

R&D NACSIS

Department of Intelligent Systems

Tokyo, Japan

Kyushu University, Japan

{ andres,ono} @rd.nacsis.ac.jp

{ akifumi,kaneko) @is.kyushu-u.ac.jp

satisfy the requirements of interactive application. Image applications are a good example to point out the vertical application’s requirements (data type, operation definition, storage management, query processing). The image processing can also be easily mapped into a parallel execution model. OODBMS[lO] have been identified as the prime vehicle to fulfill these tough demands due to the short comings of Relational DBMS. Traditional extended relational DBMSs suffer most from the impedance mismatch and fail to deliver flexibility and performance in application domains as image video or hypermedia management systems. In recent years, several commercial ORDBMSs[ 191 have entered the market place. In spite of the success of image applications, ORDBMSs have still many limitations in terms of customizability and generic vertical image processing which must be overcome before the benefit of using ORDBMSs can be fully realized. The current technological solutions to static image processing include the manipulation of several compression standard formats such as TIF, GIF or P E G . ORDBMSs, however, present significant problems (dynamic optimization, parellel processing), which means they can be only used in a restricted way by image applications. An imaging DBMS typically integrates in a vertical way the following six layers: 1) an image data type definition, 2 ) a set of image operations, 3) a query model, 4) a physical image data structure, 5) query optimization tuned for image manipulation, and 6) an image-based execution model.

Abstract This paper describes the design concept and the implementation of the imaging database OnLine Image Delivery System (OLIDS). We propose a new framework to support vertically parallel image processing in multimedia database systems. We present a generic and customizable framework for specifying image query execution plan that permits the DBMS to optimize and to execute parallel image processing. Moreover, it is important to provide to the multimedia applications the capability to customize the information server according to their specific requirements. The capability to customize easily in a vertical way the multimedia information server according to new applications’ requirements (i.e. new kinds of retrieval methods with uncertainty factor, new kinds of compression algorithms) also provides both challenges and opportunities in building high performance multimedia and image systems. Applications such as (Active hypermedia project) or MODOS (Museum project between LRMF(France), NMWA and NACSIS(Japan)) demonstrate the usefulness of the customizable feature of the Application-Oriented DBMS Phasme, a microkernel database system tested and extended inside NACSIS.

1 Introduction Multimedia and image information are the prototypical database applications ([SI,[ 171) that require the database system to support complex and evolving data structures. Image queries often involve hierarchical-traversals. Furthermore, they also have to be executed with high performance to

In this paper, we present the framework to support a customisable image processing. This framework provides a means of vertical customizability to meet image processing inside the database kernel.

761 0-8186-8147.0197 $10.00 0 1997 IEEE

2.2 The Image Data Type Definition

Moreover, the implementation of the framework has been done inside the Phasme DBMS[2] as an imaging plug-ins. Phasme is an emerging DBMS tested and extended at NACSIS (National Center for Science Information Systems) based on the application orientation paradigm. Phasme has already demonstrated achievements and high performance in multimedia management[4] and in intelligent webserver management[I].

The image data type definition level is constituted of classes, each having a corresponding class extension. The image data definition and the corresponding physical representation are available for the runtime dynamic optimizer. Regarding to inheritance, subclasses (for example 2D-color-image) of a class c (e.g. image class) inherit the attributes and methods of this class. In other words, all image classes share common image attribute and methods. This is the key to integrate user-customized image classes. They can also redefine attributes already defined for the root class: The method's calls are optimized in lower levels at runtime. The use of a template class layer enables the programmer to introduce easily new kinds of images. The image data definition, given in Fig. 1, represents the subpart of the implementation as a basis for the sample image processing presented inside the paper. Image applications require various kinds of pixel representations. To provide a compact data structure, the traditional pixel class has been split in the pixelstorage class which supports the way of the pixel representation and in the pixel-sort class which defines the type of the pixel.

In the next section, we introduce the vertical framework implemented inside the OLIDS platform. Section 3 shows how the query are translated into the physical image scheme. Section 4 introduces the optimization mechanisms of image processing. Then we compare our work to related researches. Section 6 concludes.

2 A Vertical Framework for Image Processing In this section, we propose a vertical framework for image processing using an architecture based on the concept of many sorted algebra.

2.1 Many-sorted Algebra Frame work The core spine of our vertical framework is based on the general concept of many-sorted algebra@]. A many sorted algebra is a set of collections of functions between these sets. This concept enables that each kind of application has its own vertical plug-ins inside each of the six layers: data type definition, operation definition, query model, physical structure, query optimization, and execution model. For the image type, algebra operators can be defined as it is shown in the following examples: operator-1: image x image -> image, operator-2: image x image -> boolean. The list of operators is given in [ 3 ] . Furthermore, using a many-sorted algebra, DBMS evolution is also a key feature. From a vertical point of view, the different layers are still independent. The addition of new kinds of images or new image operations do not impact the execution model which is customized and tuned for previous kinds of images.

762

2.3 The Image Manipulation Functions Image operations needed by the image application are classified in three categories: VO functions, manipulation operations of images and binary operations which can process both image-itself and pixels of images. The customizability feature relies on the plug-ins concept. The image operations and the pixel operations are embedded inside customizable plug-ins which can be dynamically loaded or unloaded at run-time according to the application’s needs. The implementation of this set of operations was done using methods associated to image classes defined in the previous subsection.

presented in Fig. 2. The JTIP standard(IS0 10918-3) avoids the limitation of memory and improves the pyramidal manipulation andalso image transfers between the server and the clients. The Pyramid data structure is implemented as a plug-ins of the Phasme DBMS.

2.4 The Image Query Model The query representation that we use in the OLIDS is based on the data flow graph presented in [12]. The rationals of this choice are: (1) the nature of image processing is a data flow graph, ( 2 ) from our vertical approach, the dataflow graph is a vehicle which supports execution plan to be run on parallel machines, and finally (3) pipelining between image operations is easily implemented using data flow. The query graph combines image or pixel operations as nodes and directed data flow arcs connecting each other these images or pixel operations. Depending on the complexity of the query, each operation node has one or more input sub-graphs or sub-trees (ST,,ST2, * -ST,) and one output graph or subtree (Output ST).

Fig. 2: Pyramid Index In the case of hypermedia applications, pieces of different images can be manipulated separately. The pyramidal data structure allows clustering management. More details about the details of the implementation are given in [3].

3.2 Query Translation based on the physical schema

-

The main objectives of a query optimizer is to find the optimal execution plan (i.e. optimal operation permutation, optimal data flow) for a given query. To reach this goal, the optimization process investigates the space of equivalent plans for this query. The same framework can be used for the Imaging database systems. The translation based on a multithread process scans the physical schema and transform the input query graph in order to perform the translation of the image query according to the physical schema. This enables to combine the features of the image physical image schema and the semantic of the image query graph. The representation of a sample query is expressed as shown Fig. 3. In this graph, the nodes refer class (e.g. 2Dcolor-image, 3D-color-image), the full arcs denote the implicit joins, and the dashed arcs

3 Translating the query into the physical image scheme The input of the image query optimization is expressed on the data definition layer while the result is a parallel query execution plan.

3.1 A sample DB physical model The physical model implemented inside OLIDS is based on the JTIP standard (JPEG Titled Image Pyramid). This standard is an evolution of the JPEG standard in order to propose zooming facilities and image decomposition. The general structure is

763

denote explicit joins. pcontains operations are denoted by arcs around a single means for recognizing the entities of the physical image schema useful for the query execution plan. For the customizability point of view, the physical schema and the conceptual schema differ due to the image application's needs. It is important and necessary to combine the matching between the two levels of data.

algebraic graph In our approach, the customizability of the static base of optimization is a rule-based optimizer using a neural network-based cost evaluator. The static optimization transforms an imagebased query into a query execution plan (named SOQEP) statically optimized as it is shown in Fig. 4.

Imphclt ]omWhCIt Jon

Pa"..matcinng ( Z D - c o l o r _ ~ , 3D_ool~-mp)

A Simplifier

b p c Transformations

Algo selector

Cwtomizable-

cost evaluaum

Fig. 4:Image-based static optimization process

Fig. 3: Query Graph

4 The Query Optimization for Image

4.2 Dynamic optimization at runtime

Processing

Dynamic optimization at runtime enables to parallelize the query execution plan according to the real workload on the systems. Dynamic optimization includes multi-thread load balancing and data placement. In term of parallelization, the key point is to determine "heuristically" and dynamically the most suitable degrees of inter-operation and intraoperation parallelism of the statically-optimized QEPs. In detail, the degree of inter-operation parallelism enables data operators of the statically- optimized QEPs to run in parallel data set. The degree of intra-operation parallelism enables data operators to process in parallel sets of data. Using such a framework, it represents a good query execution model to process image-based queries upon fragmented image data. The bucket orientation of the physical image schema enables an easy mapping to parallel bucket-based operations to use

Our vertical approach also extends the openness of the optimization level to take into account the application's needs in a better way. The key point is the efficiency of the optimization according to the image application workload. The optimization of image processing is split into two parts: static optimization at compiletime and dynamic optimization at runtime. The rationale of this architecture is to perform static compile-time optimization, and runtime while keeping the image optimization framework customizable and hopefully efficient. The vehicle of the image query processing is an algebraic data flow graph using the vertical feature of the many-sorted algebra. query execution plan is represented as a graph in general cases.

4.1 Static optimization of operations on

764

efficiently registers and following the work of [ 181.

cache

memory

This introduces the fact that the data model can be moved to the client application. The customisability introduces the capabilities to tailor the database according to the application. The data model highly influences the capability of a DBMS to provide both performance and functionality to support image processing. Image processing capabilities depend also on the customizability of the DBMS. Built-in support is not sufficient according to the requirements of multimedia applications. Customizability should include new physical structures, optimization, and execution model as it is shown in [lo]. Unlike [7], we do not consider image only as a physical object without semantics. However, image accesses according to the image representation proposed in [7] are mainly based on color distribution indexes. Our image definition framework differs from [6] in the granularity of the approach: ours is a vertical approach compared to Chang’s one which is a macroscopic overview.

5 Related works The area of imaging DBMS is a very active research area. Numerous research prototypes, products and applications based on different architectural approaches have been proposed within the last few years. The most prominent research issues of this area include querying and content retrieval(r71, [ 14]), index management([ 131, [ 19]), storage management for imaging database system[8], supporting image data types, and support of multi-user imaging database systems[9]. To our knowledge, design concept and implementation of a vertical customisable architecture to support image processing has not been proposed before. We introduce a revisited taxonomy of the imaging database as it is shown in Fig. 5.

6 Conclusion

I Built-in support I I No built-in support I Extensible

Limited extensible

Traditional Taxonomy

e Data model

No data model

Customizability

Customizability

Revisited Taxonomy

In this paper, we presented a vertical bottom-up framework to support parallel image processing. First, we proposed a vertical architecture that is customizable according to image types. Then, we showed how to translate image queries into a physical representation supporting multiclass clustering and path indices. The vehicle of image query execution plan is processing trees. The main contributions of the OLIDS are the following: -> A vertical approach to customize image processing as processing trees. It contributes to help the optimizer to find efficient query execution plan to retrieve image. -> A vertical definition of an image plug-ins supported by the micro-kernel PHASME (application-oriented DBMS). -> The vertical approach enables to take into account of the application requirements (functionality, performance, execution). Furthermore, the framework for image processing proposed have been implemented inside

the

Online

Image

Delivery

System(0LIDS) can be extended to support image queries on views. We think that this framework is relevant to

Fig. 5 : Imaging DBMS Taxonomies

765

specifiying data models, query processing and optimisation, in Proc. ACM SIGMOD, May 1993. [9] Kato T., Database architecture for content-based image retrieval, in SPIE-Image Storage and Retrieval Systems, 1992. [lo] Khoshafian S., and Baker A.B., Multimedia and Imaging Databases, Morgan Kaufmn Publishers, 1996. [ I l l Kim W., Modern Database Systems: The object model, Interoperability and beyond, Addison Wesley, 1994. I121 Lanzelotte R., Valduriez P. _,and h i t M. , On the effectiveness of Optimization Search Strategies for parallel execution spaces” In. Proc. Int. Con$ on Very Large Databases, pages 493-504, 1993. [13] L u n g C.H.C, and Zheng Z.J., Image Data Modelling for efficient Content Indexing, In Proc. International Workshop on Multimedia Database Management Systems, New York, 1995, pp 143-150. [14] Liu Z.Q., and Sun J. P. “Retrieving Structured Images Using fuzzy Similarity Measures”, In Roc. Pacific DMS96, Hong Kong, 1996. [15] Mechkour M., Emir an Extended Model for Image Retrieval, In Proc of DEXA 95, England, Sept. 95, pp 395-404. [16] Naqvi W., and Panyiotou S., Applied Active Databases for Evolving Image Processing Algorithms, In Proc. of DEXA 95, England, Sept. 95, pp 197-205. [17] Sakauchi M., Next Generation Multimedia Database and Conceptual Information Processing, In Workshop on Multimedia Information Systems and Hypermedia, 1995, PPI-6. [18] Shatdal A., Kant C., and Naughton J.F. Cache conscious algorithms for relational query processing, in Proc. of 2Uh Int’l Conference on Very Large Databases, Chile, Sept 1994, pp 510-521. [19] Smith J. R, and Shih-Fu Chang, Quad-Tree Segmentation for Texture-based Image Query, In Proc. of ACM Multimedia94, pages 279-286, San Francisco, California, October 1994. [20] Stonebraker M., Object-relational DBMSs The Next Great Wave, Morgan Kaufmann publishers, inc. California, 1996

process other kinds of multimedia objects inside OLIDS such as video which represents a list of images. The different layers of our vertical approach are a basis for extension for new video or multimedia applications. Comparing to image processing where image creation or lookup, video requires at the same time, creation or update and lookup. Moreover, video applications is a time dependent media. So the customizability is also required to be introduced in an easy way for the time management. From the video query processing point of view, we also expect to increase the efficiency of video applications (i.e. parallel algorithm for multi-resolution management) using this vertical approach.

Acknowledgments We would like to thank NACSIS for their encouragement and support. We also thank Jean Francis Barda (Netimage, France) for his technical support during the implementation.

References [ l ] Andres F., Boulos J., Ono K, Accessing Active Application-oriented DBMS from the World Wide Web, In Proc of Int. Symp. of CODAS, Kyoto, Japan, 1996, pp 232-234. [2] Andres F., Ono K., Phasme : A High Performance Parallel Application-oriented DBMS, to appear in Informatica, Special Issue on Parallel and Distributed Database Systems, 1997. [3] Andres F.. Ono K, The OLDS Database System. NACSIS report, 1997. [4] Boulos J., Andres F.. and Ono K, Architecture and Storage Management of Multimedia Caching Server for Online Satellite Data Delivery System, in Proc. of Asia-Pacific on Information and Symp. Telecommunication Technologies, APSITT97, Vietnam, March 97. [ 5 ] Budford John F., Multimedia Systems, AddisonWesley, ACM Press.1944. [6] Chang S.K., and Hsu A., Image Information Systems: where do we go from here?, in IEEE Transactions on Knowledge and Data Engineering, vol. 4, No5, October 1992. [7] Flickner. M., and al. Query by image and video content: The QBIC System, IEEE Computer 28(9/Sepr), 1995. [8] Guting R.H.. “Second Order Signature* A tool for

766