Image database indexing and retrieval using the Fractal Transform ...

Image Database Indexing and Retrieval Using the Fractal Transform Jean Michel Marie-Julie, Hassane Essafi LETI(CEA-Technologies Avanc6es) DEIN - CEA Saclay F91191 Gif sur Yvette Cedex France E-mail: {jmmjulie, hessafi}@gaap.saclay.cea.fr Abstract. Accessing to large image databases is a huge challenge because of

the large amount of data required by images. Therefore automatic and efficient indexing is needed for fast content based retrieval, it alleviates the drawback of any manual annotating. We propose a method for pattern matching into large image databases based on the Fractal Transform. A mathematical representation is associated to the images of the database. This representation is a set of function parameters resulting from a dedicated fractal compression scheme, and used as an index by a retrieval algorithm. It works entirely in the Fractal transform parameter space of both image and pattern, to obtain performances compatible with an interactive search. The research engine uses both textures and edges of the pattern. The pattern can be present in the image with different orientations and/or scales by using a multi-compression Fractal representation of the pattern. This method allows to retrieve in 3 seconds a 64 x 64 pixels pattern in an 100 images (512 x 512 pixels) database, on a SUN Sparc 20 workstation. It can be combined with other indexing and retrieval techniques.

1.

Introduction

In order to obtain fast access to large image repositories, specific indexing and retrieval techniques are needed. Indexing is required to avoid the performing o f lowlevel processing on images (edge, texture or colour segmentation) or of high level recognition processing at query time. Textual annotation is a very common technique, it allows to index the image by specifying the objects within it, then high level queries can be performed, such as "find images showing a cat". The generation of the automatic text description is hard because it involves unresolved machine vision problems. Regarding the manual generation o f the text, it is subjective and influenced by requests, the database is dedicated to a particular problem, and have to be re-annotated for each new application. Different indexing techniques [4]allow to associate to the image a description issued from a segmentation (by colour, texture) or analysis (eigen-representation for instance) (see table [l J). These methods are query-dependant. In other words, to process complex queries, inVolving several aspects, several indexes have to be

170

constructed. Some of these techniques need a pre-processing step, consisting in the edge detection and object localisation. This task is very complex and the human interaction could be needed. For queries based on pattern matching, traditional indexing/retrieval methods does not allow to reduce enough the retrieval time. The correlation method, even coupled with a wavelet decomposition is not as fast as needed (on scalar workstation) for interactive search process. The method presented in this paper proposes an efficient indexing to obtain small query processing times. The indexing step uses a dedicated Fractal compression scheme, this offers an image representation preserving the semantic. This representation contains the image compressed data, allowing to reconstruct the image by decompressing, and a compact index used during the search process. The search engine compares this index with the compressed data of the pattern. Furthermore, thanks to a multi-compression or multiresolution representation [15] of the pattern, it is possible to retrieve this pattern even if its orientation or scale in the image is different. The multiresolution compression scheme is presented in [15]. The advantage of the search process is that it uses directly compressed data, without decompressing them, and avoid using low-level processes, so it allows to reduce considerably the query processing time (see the result section). The indexing and retrieval scheme can be applied to any application field, it does not require any knowledge about image content (no pre-processing required).

Shapes Textures

Eigen representation[5] Moments[6] [8] [9] Wold decomposition[6]

Multifractal dimension[ 12] [ 13] Colour Colour histogram [7][101111] Pattern search Multiresolution correlation[ 14] Retrieval based on Fractal Transform

Yes Yes Yes

-

Yes Yes

Yes

-

Yes Yes Yes

Yes

Yes Yes

Table [1] : Request types and some examples of associated methods (non-exhaustive list).

171

In a first part, the Fractal compression is exposed, then the image indexing, which uses a particular compression scheme, is described. The retrieval is presented in a third part, followed by some results and a conclusion.

2.

Image fractal compression

The basic idea of the Fractal compression [1][2][3] is that the image is the attractor of a contractive map. This map is an PIFS (Partitioned Iterated Function System) and can be described by its parameters set. The aim of compression is to determine these parameters and therefore the PIFS map. First, we consider the image f as a function defined on [0,1Ix[0,1 ]=12 such as: f : l 2 --+1

(x, y) -~ z = f ( x , y ) The image f i s the attractor of a map W: f = [WI = lim W on = WoWoWo... o W n --~oo

W is a set of n maps wi, each wi works on a compact part of the image, called domain block Dj and transforms it into another compact part, called range block Ri, such as w i ( D j ) = Ri Ri, i ~ [0;N], is a element of a partition of the image. Dj, j e [0;N], is a compact of

12" wi can be written as: ra i

Wi

--

di

0

Y + fi

0

SiJkZ j

(x, y, z) belonging to the domain Dj.

~OiJ

wi contains a geometric part (isometric and scaling) and a massic part applied to image pixels. si and oi are respectively the contrast scaling and luminosity offset parameters, ai, bi, ci, di parameters are used for the isometric (rotation, reflection or scaling), ei and f-

define the translation. The contractivity of W assumes the uniqueness of the fixed point, here the image f. 2.1 I m a g e encoding To encode an image f, we need to find the transformation set ws, we..... wu with U W -'- U w i andf=/W]. In other words, f is the fixed point of W: i=l

f = W ( f ) -- w t ( f ) k.) w 2 (f)L_J...t..)wN ( f )

172

In reality we are looking for: f ~ f ' = W ( f ' ) ~ W ( f ) = w0 ( f ) u . . . u w N ( f )

To determine the wi transformations, we calculate for each range Ri, the domain O i which is the most similar, considering an isometrie, a contrast scaling and a luminosity offset. The similarity is measured by the root mean square distance, to adjust s/and oi. P

R =~.~(s.a i + o - b i ) 2 i=l ai and bi are respectively the grey levels of a domain block (after isometric) and of a range block.

We store the maps w as vectors Wi:

Wi =(~i

xi Yi txi tyi si oi)

isometrie code; xi, Yi domain position; txi, tyi, translation coordinates of wi; si, oi contrast scaling and luminosity offset

In the following parts, we note ~ f a set of vectors Wi, describing the image f. 2.2 Image decoding To decompress the image, the fixed point or attractor of W is calculated by iterating W (seefigure [12]).

3.

Image database indexing

The Fractal image compression gives a set of parameter vectors, noted ~'~f describing a PIFS (Partitioned Iterated Functions System). The idea is to index the image by using this set. The index must be efficient and small to allow fast retrieval. In order to achieve that, we have to adapt the compression scheme by reducing the domain search space during the compression. Then, we apply a filtering to the index to make it as compact as possible. The indexing is fully integrated to the compression process, they can be done simultaneously.

173

3.1 Adapted c o m p r e s s i o n scheme 3.1.1

Reduction of the d o m a i n search space

The modification of the compression scheme is motivated by the fact that the Fractal compression is a global process. This process can be seen as a vector quantization, where the dictionary contains all the domain blocks. For each range block, we search which domain block is the most similar. This implies that the encoding of a given image part, if we consider a global dictionary (all the domain blocks) is different compared to the encoding of the same part, if the dictionary only contains domain blocks of this part (seefigure [1]). To alleviate this drawback, the research area of each range block is reduced to its neighbourhood (domain blocks belonging to a restricted search space). The size of this neighbourhood is an important parameter, the pattern size will not be able be smaller than it.

Sam~e

Sam~e

Image

Image

Figure [1] : In the first case (top), wl associates Dt to R~, if the sub-imagef' is compressed independently, the transformation wi' associates Dk' to Ri. The transformations wl and w~' are different. In the second case, a restricted search space transformations w~' and w~ are identical.

3.1.2

is used, the

Partitions choice

Generally, during the compression process, a domain partition is built. But nothing assumes that the image and a part of this image independently considered, have the same range partition and domain blocks (seefigt~re [2]). During the request processing, this implies to compress several times the pattern with different partitions to cover all possibilities.

174

Figure [2] : Two different square partitions of an image part. The solution is to consider all domain blocks of constant size in the image. 3.1=] Relevant feature extraction

To make the index more compact, the vectors belonging to g21 are classified. Given a class, Ck of PIFS, **; such as wi(Di)=Ri, belongs to Ck if q(Ri)>sk. The q0 function allows to measure the information quantity available in the range block Ri, q0 could be the standard deviation of the pixel distribution of R i. The sk values are the index thresholds. We can organise the index as shown infigure [3]. 3.1.4 General image index structure Part corresponding to the standard deviation threshold sl

Image index

L

Image compressed data

Part corresponding to the standard deviation threshold So

Figure [3] : The index is partitioned into classes according to the standard deviation.

175

4.

Retrieval stage

4.1 Search procedure The search algorithm allows to retrieve an image by giving a pattern. It compares the image PIFS (index) with several PIFS of the pattern (seefigure [6]). In order to allow the retrieval of the image, for any size or rotation of the pattern, the pattern is multi-compressed or compressed with a multiresoludon scheme [15]. The multi-compression scheme allows both scale and rotation invariance for predefmed scales and orientations (nsi~ predefined scales and nrot~tionpredef'med rotations). Thanks to a suited data structure, the comparison between the image and sample PIFS is very fast. Pattern

Ima~les

1

Indexing based on Fractal compression [,

Compressed images Storage

I Fractalcompression [

[

~

I

~v

Images containing [

F Compressedimages

[.................................................. Image .................................................... OataBse t ]

Retrieval

Figure [6] : General synopsis. The pattern compression contains several steps depending of the type of search. Multi-compression is used to retrieve a pattern in an image with any size or orientation (see table[2]). The multi-compression scheme consists in compressing the pattern with different sizes and rotations, but the compression process is time consuming. To avoid multiple compression processes, the multiresolution representation can be used [ 15]. Both compression schemes generate several sets of parameter vectors s i denotes the different sets of parameter vectors involved in a multi-compression scheme and t denotes the different translations applied to the pattern partition, to alleviate the partition problem exhibited in part 3.1.2. For each -Qi.t, we calculate a distance using an adapted nearest neighbour search technique, to evaluate the distance between the sample and the candidate images. This gives us a ratio distribution. The ratio or score associated to the image is the maximum of the distribution.

176

Table [2]: Number of PIFS used and invariance properties of different methods for the

pattern and/or image compression, n~ze and nrot~ao, are respectively the number of different sizes and the number of different rotations for the pattern partition 4.2 Distance calculation Distance calculation is an important part of the search process. The query time is very dependant on the complexity of the distance. The distance calculation is based on the PIFS parameters. We consider s

and s

i=O ..... n as the set of parameter vectors of the pattern PIFS

j=O,..,m, as the set of parameter vectors of the image PIFS.

Let Ui be a parameter vector of the pattern sample, Ui can be written as: Ui = (~ti Yi Yi txi tYi si oi ) We search among ~ e parameter vectors of Lhe image P!FS. a vector ~,(i. such as rr; and Vj are comparable (same isometrie code and translation coordinates). For each Ui, we build O(Ui) the set of vector Vj belonging to the image PIFS, which are comparable to U,-.

Vj -~ (T/j xj yj txj ~ j sj oj) Vj belongs to O(Ui) and verifies: ]/i=Tj,

txi=txj,

l)'i-~ly j

The distance d~ between U i and V i is:

d v (U i ,V i ) -- ~/(s i - sj )2 d- (o i - oj

)2

The distance d between the pattern PIFS and image PIFS is

d(ifsE,ifsim) = ~_,e(Ui,O(Ui)) u,e~s~ with e(U i , f~(U i )) = 1 if 3 V i E O ( U i ) such as d v (U i ,V i )