Pattern Anal Applic (2007) 10:345–360 DOI 10.1007/s10044-007-0072-z
THEORETICAL ADVANCES
Binary object extraction using bi-directional self-organizing neural network (BDSONN) architecture with fuzzy context sensitive thresholding Siddhartha Bhattacharyya Æ Paramartha Dutta Æ Ujjwal Maulik
Received: 4 February 2006 / Accepted: 8 March 2007 / Published online: 5 May 2007 Springer-Verlag London Limited 2007
Abstract A novel neural network architecture suitable for image processing applications and comprising three interconnected fuzzy layers of neurons and devoid of any back-propagation algorithm for weight adjustment is proposed in this article. The fuzzy layers of neurons represent the fuzzy membership information of the image scene to be processed. One of the fuzzy layers of neurons acts as an input layer of the network. The two remaining layers viz. the intermediate layer and the output layer are counterpropagating fuzzy layers of neurons. These layers are meant for processing the input image information available from the input layer. The constituent neurons within each layer of the network architecture are fully connected to each other. The intermediate layer neurons are also connected to the corresponding neurons and to a set of neighbors in the input layer. The neurons at the intermediate layer and the output layer are also connected to each other and to the respective neighbors of the corresponding other layer following a neighborhood based connectivity. The proposed architecture uses fuzzy membership based
weight assignment and subsequent updating procedure. Some fuzzy cardinality based image context sensitive information are used for deciding the thresholding capabilities of the network. The network self organizes the input image information by counter-propagation of the fuzzy network states between the intermediate and the output layers of the network. The attainment of stability of the fuzzy neighborhood hostility measures at the output layer of the network or the corresponding fuzzy entropy measures determine the convergence of the network operation. An application of the proposed architecture for the extraction of binary objects from various degrees of noisy backgrounds is demonstrated using a synthetic and a real life image. Keywords Binary object extraction Bi-directional self-organizing neural network Fuzzy context sensitive thresholding System transfer index
1 Introduction S. Bhattacharyya (&) Department of Computer Science and Information Technology, University Institute of Technology, The University of Burdwan, Burdwan 713 104, India e-mail:
[email protected] P. Dutta Department of Computer Science and Engineering, Kalyani Government Engineering College, Kalyani 741 235, India e-mail:
[email protected] U. Maulik Department of Computer Science and Engineering, Jadavpur University, Kolkata 700 032, India e-mail:
[email protected]
The primary objective of object extraction from a noisy background is the segmentation of an image scene into foreground regions corresponding to different localized homogeneous object regions and background regions corresponding to non-object regions. This localization task, which centers around the classification of the image pixels based on some local features such as color, texture and position, is followed by merging the localized regions together. Typical applications include removal of noise artifacts, deblurring and segmentation/clustering of image data. The fields of defense, GIS and remote sensing rely heavily on these types of image processing applications. However, the problems of detection and extraction of ob-
123
346
jects assume greater complexity in respect of real-time applications. Several classical approaches exist in the literature to deal with the task of extraction of object centric features from an image scene [1, 2]. These approaches rely on identifying 4 or 8-connected components in an image scene, merging and labeling them appropriately as the object or background class. Though these approaches exhibit linear time complexity with respect to the number of image pixels under consideration, yet they end up at merging each and every connected regions in the image scene, irrespective of the noisy pixels therein. Other approaches in this direction focus on filtering out regions composed by less than a given number of pixels, assuming these pixels to be a part of the inherent noise content [3, 4]. These approaches still suffer from the lack of generalization in as much as they do not discriminate between object and noisy pixels. Moreover, the main problem regarding this filtering procedure lies in the fact that some a priori knowledge about the nature and degree of noise is a prerequisite for the design of an appropriate filter. Among other approaches, techniques involving standard morphological operations to retrieve object regions out of an image scene can also be found in the literature [3, 5–7]. Neural networks have often been employed by researchers for dealing with the daunting tasks of extraction [8–11], classification [12, 13, 14, 16] of relevant object specific information from redundant image information bases, segmentation of image data [15] and identification and recognition of objects from an image scene [16, 17]. Several neural network architectures, both self-supervised and unsupervised, are reported in the literature, which have been evolved to produce outputs in real time. Kohonen’s self-organizing feature map [18, 19] is centered on preserving the topology in the input data by subjecting the output data units to certain neighborhood constraints. The Hopfield’s network [20, 21] proposed in 1982, is a fully connected network architecture with capabilities of auto-association. The bi-directional associative memory (BAM) model [22, 23], introduced by Kosko manifests similar network dynamics as the Hopfield model. The multilayer self-organizing neural network (MLSONN) architecture [24], which operates like the multilayer perceptron (MLP) [25, 26], is a feedforward network architecture and resorts to some fuzzy measures of the image information so as to derive the system errors therein. Chiu et al. applied artificial neural network architectures for the processing of photogrammetric targets [27]. However, the inherent problems with these network architectures lie in the adjustment and reassignment of the interconnection weight matrices involved in the processing task. The backpropagation algorithm used in most of these approaches, for updating of the interconnection weights, increases the turnaround time of these approaches.
123
Pattern Anal Applic (2007) 10:345–360
Genetic algorithm based optimization techniques also find use in supplementing the neural network architectures. Zamparelli [28] resorted to genetic algorithms for the purpose of training of cellular neural networks. In [29], Munshi et al. used a fully connected cellular neural network architecture for the extraction of binary objects from a noisy image scene. The proposed technique uses some fuzzy measures of the input image scene for deriving an optimized template for the purpose of a convolution operation required in the object extraction task. However, the use of genetic algorithm based optimization procedures reduces the time efficiency of these techniques. Real time operation of neural networks can be achieved if the computational burden is reduced by resorting to such network structures, which do not use any time complex back-propagation or genetic algorithm based weight assignment/adjustment procedures. The resultant architecture, besides being faster in terms of processing speed, will also be simple to implement in hardware. In this article, a novel neural network architecture, efficient for object extraction tasks, is proposed. The architecture is a bi-directional self-organizing architecture comprising an input layer, an intermediate layer and an output layer representing fuzzy information of an input image scene. The architecture employs a feedforward mode of propagation of input information vis-a-vis counterpropagation of network states to self-organize the input information into extracted object regions. The transfer characteristics of the proposed architecture are guided by the standard sigmoidal activation function with image context sensitive thresholding values, derived from the fuzzy image information content through the fuzzy cardinality estimates of image pixel neighborhoods. The novelty of the architecture lies in the methodology adopted in the assignment and adjustment of network interconnection weights, which does not require the time complex backpropagation algorithm, thereby inducing hardware implementability of the same. An application of the proposed network architecture for the extraction of binary objects from a noisy environment is demonstrated with several noisy versions of one synthetic image and one real life image. The performance of the proposed architecture as regards to time efficiency and image quality of the extracted objects, as compared to that of the standard multilayer selforganizing neural network employing back-propagation based weight adjustment [24], is reported using several noisy versions of the images under consideration. Various degrees of uniform noise and Gaussian noise with zero mean and several standard deviation values are used in this treatment. The extraction efficiency of the network is also substantiated from a systematic point of view in quantitative terms, with a proposed system transfer index so as to reflect the noise immunity of the proposed architecture. The paper
Pattern Anal Applic (2007) 10:345–360
347
is organized as follows. Section 2 provides an overview of fuzzy sets and fuzzy measures. The architecture and operational dynamics of the proposed bi-directional self-organizing neural network (BDSONN) is presented in Sect. 3. Section 4 illustrates the results of application of the proposed BDSONN architecture for the extraction of binary objects from noisy backgrounds. Section 5 concludes the paper with future directions of research.
2 Fuzzy sets and fuzzy set theoretic operations In this section, fuzzy set theoretic concepts, fuzzy cardinality and fuzzy entropy measure relevant to the article are discussed. A fuzzy hostility index reflective of image context sensitive information, is also introduced in this section. 2.1 Fuzzy set concepts A fuzzy set [30, 31] is a collection of elements, A = {x1, x2, x3,..., xn}, characterized by a membership function lA(xi), where xi refers to the ith element in the set. This membership function, which is indicative of the fuzziness in the set, lies in [0, 1]. A membership value of 1 indicates strict containment of an element within the set, while a membership value of 0 indicates weak containment. According to Zadeh’s notation [30, 31], a fuzzy set A can be represented as A¼
X l ðxi Þ A ; xi i
i ¼ 1; 2; 3; :::; n
where, n is the number of elements in the set and resents a collection of elements.
ð1Þ P
rep-
i
2.2 Fuzzy set theoretic operations Similar to the crisp set theory, the following set theoretic operations of union, intersection and complement [31] are defined on two fuzzy sets A, B for an element x in the universe of discourse X. Union: lA[B ðxÞ ¼ maxðlA ðxÞ; lB ðxÞÞ
ð2Þ
Intersection: lA\B ðxÞ ¼ minðlA ðxÞ; lB ðxÞÞ
ð3Þ
Complement: lA ðxÞ ¼ 1 lA ðxÞ
ð4Þ
2.3 Fuzzy cardinality The scalar cardinality of a fuzzy set A is the sum of the membership values of all the elements in the set. Mathematically is defined as [31]
nA ¼
n X
lA ðxi Þ
ð5Þ
i¼1
For a finite number of elements, this cardinality is referred to as the fuzzy cardinality. It is evident from this definition that higher is the degree of containment of the elements in the fuzzy set, the higher is the fuzzy cardinality. Similarly, a lower fuzzy cardinality results when the elements are weakly contained in the fuzzy set. Thus, the fuzzy cardinality value of a fuzzy set indicates the overall degree of containment of the constituent elements in the fuzzy set. 2.4 Fuzzy entropy The entropy EA of a fuzzy set A, characterized by the membership function lA(xi) is a measure of the degree of fuzziness in the fuzzy set. For a fuzzy set comprising n elements, it is given by [32] EA ¼
n 1 X lA ðxi ÞlnðlA ðxi ÞÞ nln2 i¼1
f1 lA ðxi Þglnf1 lA ðxi Þg
ð6Þ
The fuzzy entropy measure reflects the amount of ambiguity that corresponds to the randomness/disorder in an observation. 2.5 Fuzzy hostility index An image represents a map of pixel intensity values. The ambiguity in the intensity distribution in an image can be described by representing the image as a fuzzy set of brightness/darkness, with the individual pixel intensities being the representative fuzzy membership values in that fuzzy set of brightness/darkness. Considering the image as a collection of pixel neighborhood regions, where the image pixels are surrounded by a number of neighboring pixels, such a fuzzy set of brightness/darkness is thus a superset of several neighborhood fuzzy subsets. Each candidate element in a particular neighborhood fuzzy subset is surrounded by several orders of neighboring entities, starting from a first order neighborhood (comprising four immediate neighbors) or a second order neighborhood (comprising eight neighbors) to some other higher order neighborhoods. The degree of ambiguity in these fuzzy subsets of brightness/darkness (neighborhood fuzzy subsets) is indicative of the degree of homogeneity/ heterogeneity in that neighborhood. The closer are the representative membership values in a neighborhood fuzzy subset, the higher is the homogeneity in that neighborhood and less is a candidate element hostile to its neighbors. On
123
348
Pattern Anal Applic (2007) 10:345–360
the contrary, a heterogeneous neighborhood fuzzy subset arises out of sharp contrasting membership values of the elements in the neighborhood fuzzy subset and hence such a neighborhood is a hostile one. This homogeneity/heterogeneity in a second order neighborhood fuzzy subset can be accounted for by a fuzzy hostility index defined over the fuzzy neighborhood as, f¼
8 jlp lq i j 3X 8 i¼1 jlp þ 1j þ jlq i þ 1j
ð7Þ
where, lp is the membership value of the candidate element and lqi , i = 1, 2, 3, ..., 8 are the membership values of its fuzzy neighbors in a second order neighborhood fuzzy subset. The value of the fuzzy hostility index (f) lies in [0, 1].
3 Bi-directional self-organizing neural network architecture The BDSONN architecture, as the name suggests, is a three-layer network structure assisted by bi-directional propagation of network states for self-supervised organization of input information. The network architecture consists of an input layer for accepting external world inputs and two competing self-organizing network layers, viz. an intermediate layer and an output layer. The number of neurons in each of the network layers corresponds to the number of pixels in the input image scene. The input layer of the network accepts fuzzy membership values of the constituent pixels in the input image scene. This fuzzy input information is propagated to the other network layers for further processing. Thus the network layers resemble fuzzy layers of neurons, guided by the fuzzy membership information. The neurons in each layer of the network are connected to each other within the same layer following a cellular network structure. The strengths of these intralayer interconnections are fixed and full and equal to 1. However, each neuron in a particular layer of the network is connected to the corresponding neuron and to its second order neighbors of the previous layer following a neighborhood-based topology through forward path inter-layer interconnections. In addition, the output layer neurons are similarly connected to the intermediate layer neurons via the backward path inter-layer interconnection paths. The strengths of these inter-layer interconnections between the input layer and the intermediate layer, the intermediate layer and the output layer and between the output layer and the intermediate layer neurons are decided by the relative measures of the membership values at the individual neurons of the different layers. Figure 1 shows a schematic of the proposed BDSONN architecture using fully connected
123
network layers and second order neighborhood based interlayer interconnections. 3.1 Network dynamics The input layer of the network architecture acts as a switching layer of network inputs. This layer accepts the fuzzy membership values of the input image information and switches them to the intermediate layer for further processing. The fuzzy cardinality values corresponding to the different neighborhood fuzzy subsets in the input layer are accumulated at the central candidate neurons of the different fuzzy neighborhoods through the intra-layer interconnections. These fuzzy cardinality values are reflective of the neighborhood fuzzy subset context information and indicate the membership distribution within a neighborhood. These values are used for defining the context sensitive thresholding information necessary for the characteristic transfer function of the processing neurons. The inter-layer interconnections serve twofold purposes. The connections between the corresponding neurons in the different layers of the network are meant for the propagation of the context sensitive thresholding information from the neurons of one layer to those of the other layer. The second order inter-layer interconnections between the candidate neurons of one layer and the second order neighbors of the corresponding neuron in the other layer are meant for propagating the network states from one layer to the other. The strengths of these second order inter-layer interconnections are decided from the relative measures of the membership values at the constituent neurons. If lj is the membership value at the jth candidate neuron at one layer and lij is the membership value at the ith second order neighbor of the jth candidate neuron in the
Fig. 1 Bi-directional self-organizing neural network (BDSONN) architecture using a second order neighborhood based forward and backward path inter-layer interconnections for the propagation of network states (bold lines indicate path for propagation of fuzzy context sensitive thresholding information, not all intra-layer interconnections are shown for clarity)
Pattern Anal Applic (2007) 10:345–360
349
same layer, then the inter-layer interconnection strength, wijj’, between the corresponding other layer j¢th candidate neuron, and the ith second order neighbor of the other layer is decided by the fuzzy complement operator given in (4) as. wijj0 ¼ li lij
ð8Þ
Thus, the interconnection strength is a function of the fuzzy membership values at the neurons in a particular network layer. These types of neighborhood based interlayer interconnections exist between the input layer and the intermediate layer, the intermediate layer and the output layer in the forward direction and between the output layer and the intermediate layer in the backward direction.
Fig. 2 Original images a synthetic image, b spanner image
3.2 Network operation On reception of the fuzzy membership values of the input image information from the input layer of the network, the counter-propagating layers of the BDSONN architecture self-organize the input information into outputs by means of counter-propagation of the intermediate network states. The input layer neurons switch the network inputs to the intermediate layer via the inter-layer interconnections and the fuzzy context sensitive thresholding information accumulated therein via the intra-layer interconnections. If lIij is the fuzzy membership value of the ith neighbor of the jth input layer neuron in a neighborhood fuzzy subset, then the input Imj¢ at the j¢th corresponding intermediate layer neuron, which enjoys connectivity with this input layer neighborhood, is given as X Imj0 ¼ wijj0 lI ij ð9Þ i
where, wijj¢ is the inter-layer interconnection strength between the j¢th corresponding intermediate layer candidate neuron and the ith second order input layer neighbor of the jth candidate neuron of the input layer. The output produced by the j¢th intermediate layer neuron is given by Oj0 ¼ f ðImj0 Þ
ð10Þ
where, f is the standard sigmoidal activation function with fuzzy context sensitive thresholding given by 1
y ¼ f ðxÞ ¼ 1þe
kðxhC O 0 Þ
ð11Þ
j
hC Oj0 is the fuzzy context sensitive thresholding information [determined from the fuzzy cardinality estimates as given in (5)] for the j¢th intermediate layer neuron, prop-
Fig. 3 Noisy and extracted synthetic images a, b, c, d, e noisy images with Gaussian noise at r = 8, 10, 12, 14 and 16; a¢, b¢, c¢, d¢, e¢ BDSONN extracted images; and a¢¢, b¢¢, c¢¢, d¢¢, e¢¢ MLSONN extracted images
123
350
Pattern Anal Applic (2007) 10:345–360
Fig. 4 Noisy and extracted spanner images a, b, c, d, e noisy images with Gaussian noise at r = 8, 10, 12, 14 and 16; a¢, b¢, c¢, d¢, e¢ BDSONN extracted images; and a¢¢, b¢¢, c¢¢, d¢¢, e¢¢ MLSONN extracted images
Fig. 5 Noisy and extracted synthetic images a, b, c, d, e noisy images with uniform noise at 64, 100, 144, 196 and 256; a¢, b¢, c¢, d¢, e¢ BDSONN extracted images; and a¢¢, b¢¢, c¢¢, d¢¢, e¢¢ MLSONN extracted images
agated from the input layer of the network through the inter-layer interconnections between the corresponding input and intermediate layer neurons. The k parameter controls the slope of the function. In this way, the network input states are propagated from the input layer to the intermediate layer of the network and finally to the output layer of the network. The backward path inter-layer interconnections are similarly determined from the relative measures of the fuzzy membership values at the output layer neurons. Once
the inter-layer interconnections are assigned, the output layer network states and the corresponding output layer fuzzy neighborhood context sensitive thresholding information are propagated back to the intermediate layer. The intermediate layer, in turn, processes the incoming information and again propagates the processed information back to the output layer, after reassignment of the interlayer interconnection strengths and evaluation of the fuzzy context sensitive thresholding information. This back and forth propagation of the network states is continued until the inter-layer interconnection strengths in the forward
123
Pattern Anal Applic (2007) 10:345–360
351
network initialization phase, where the intra-layer interconnections between the constituent neurons within the different network layers are initialized to 1, (ii) an input phase, where the fuzzy membership values of the external world input noisy image scene pixels are fed at the input layer of the network, (iii) forward propagation phase, where the forward path inter-layer interconnections between the consecutive network layers are determined from the relative fuzzy membership values at the constituent neurons of the preceding network layer and the processed outputs of the preceding network layer are propagated to the following network layer i.e. from the input layer to the intermediate layer and from the intermediate layer to the output layer, and (iv) backward propagation phase, where the network output layer outputs are propagated to the previous network intermediate layer after the backward path inter-layer interconnections between the network output and intermediate layers are determined from the relative fuzzy membership values at the constituent neurons of the output layer. Each of these propagation phases incorporates the process of the determination of the fuzzy cardinality estimates of the different network layer neighborhood fuzzy subsets for computing the fuzzy context sensitive thresholding information required for the processing operation of the succeeding network layer. This self-organization procedure can be illustrated by the following algorithm [33]. 1
Begin Initialization phase
2
Initialize intra_conn[l], l=1, 2, 3
Remark: intra_conn[l] are the intra-layer interconnection matrices for the three l network layers. All intra-layer interconnections are set to unity. Input phase Fig. 6 Noisy and extracted spanner images a, b, c, d, e noisy images with uniform noise at 64, 100, 144, 196 and 256; a¢, b¢, c¢, d¢, e¢ BDSONN extracted images; and a¢¢, b¢¢, c¢¢, d¢¢, e¢¢ MLSONN extracted images
path of the network architecture or the fuzzy hostility indices at the output layer neighborhood fuzzy subsets stabilize. At this point, the input image information is selforganized into stable homogeneous object and background regions at the output layer of the network architecture. 3.3 Network self-organization algorithm The proposed BDSONN architecture operates on the principle of self-supervision and self-organizes input information into extracted output information. The selforganization procedure is carried out in four phases viz. (i)
3
Read pix[l][m][n]
Remark: pix[l][m][n] are the fuzzified image pixel information at row m and column n at the lth network layer, i.e. the fuzzy membership values of the pixel intensities in the image scene. pix[1][m][n] are the fuzzy membership information of the input image scene and are fed as inputs to the input layer of the network. pix[2][m][n] and pix[3][m][n] are the corresponding information at the intermediate and output layers. Forward propagation phase 4 5 6
CONSENT[l+1][q][r]=card[l][q][r] inter_conn[t][l][l+1][m][n]=1(pix[l][m][n] - pix[l][q][r]) pix[l+1][q][r]=SUM[fsig(pix[l][m][n] x inter_conn[t][l][l+1][m][n])]
123
352
Pattern Anal Applic (2007) 10:345–360
Fig. 7 Distribution of the neighborhood fuzzy hostility indices of the Gaussian noise affected synthetic images and their BDSONN extracted counterparts a at r = 8, b at r = 10, c at r = 12, d at r = 14 and e at r = 16
Remark: CONSENT[l+1][q][r] are the fuzzy context sensitive thresholding information accumulated at the (l + 1)th network layer’s [q, r]th candidate neuron. card[l][q][r] are the fuzzy cardinality estimates of the lth layer. fsig is the standard sigmoidal activation function. inter_conn[t][l][l+1][m][n] are the inter-layer interconnection weights between the lth layer [m, n]th neuron and (l + 1)th layer [q, r]th candidate neuron at a particular epoch (t), determined from the relative pix[l][m][n] values. SUM refers to the summation of the functional responses of the standard sigmoidal activation function over a neighborhood. The CONSENT values and the processed image information are propagated to the following layer (until the output layer is reached) using the inter-layer interconnections existing between the different network layers.
10
7 8
3.4 Stabilization of the network
Do Repeat steps 4, 5 and 6 with intermediate layer outputs Backward propagation phase
9
CONSENT[l-1][q][r]=card[l][q][r]
123
11
inter_conn[t][l][l-1][m][n]=1(pix[l][m][n] - pix[l][q][r]) pix[l-1][q][r]=SUM[fsig(pix[l][m][n]
x inter_conn[t][l][l-1][m][n])] Remark: Propagation of the CONSENT values and the processed information in the reverse direction from the network output layer to the network intermediate layer. 12
Loop Until [m][n]
((inter_conn[t][l][l+1]
inter_conn[t-1][l][l+1][m][n]) < epsilon) Remark: epsilon is the tolerable error. 13
End
The principle of the object extraction process is the localization of object centric features from a noisy background. The fuzziness in a noisy image scene comprising object information and background information is due to
Pattern Anal Applic (2007) 10:345–360
353
Fig. 8 Distribution of the neighborhood fuzzy hostility indices of the Gaussian noise affected spanner images and their BDSONN extracted counterparts a at r = 8, b at r = 10, c at r = 12, d at r = 14 and e at r = 16
the induced noise therein. The presence of noise results in a fair amount of heterogeneity (as regards to the intensity levels) between the individual image pixels in the different neighborhood fuzzy subsets in the image scene. This heterogeneity, manifested in the neighborhood fuzzy hostilities, is an indirect measure of the degree of noise in the image scene. The network system error (w) is thus a function of the neighborhood fuzzy hostility index (f) and can be represented as 8 jlp lq i j 3X w ¼ f ðfÞ ¼ f 8 i¼1 jlp þ 1j þ jlq i þ 1j
! ð12Þ
Recalling (8), w is thus a function of wij, the inter-layer interconnection weights between the ith and the jth layer neurons. Thus, w assumes a minimum value when f is minimum, i.e. when lp lqi ¼ 0. This implies a fuzzy hostility index value of f = 0. Thus, it can be inferred that the system attains stabilization, when the heterogeneity of the system is at a minimum. In addition, the fuzzy entropy measures of these fuzzy hostilities also reflect of the average
amount of ambiguity in the image scene. The task of object extraction is tantamount to the reduction in the fuzzy hostilities in the image neighborhood regions or localizing homogeneous object and background regions out of the heterogeneous noisy regions, thereby reducing the average fuzzy entropy measures of the fuzzy neighborhood hostilities. This implies that in an extracted image scene comprising homogeneous object and background regions, the fuzzy entropy measures of the fuzzy neighborhood hostilities are minimum and have attained stabilization. Therefore, the convergence of the network operation is determined by the stability achieved in the fuzzy entropy measures. Thus, the stabilization of the object extraction process by the network is decided by the stabilization of the fuzzy entropy measures.
4 Results The proposed BDSONN architecture has been applied for the extraction of binary objects from a noisy background. Comparative results of object extraction are reported using
123
354
Pattern Anal Applic (2007) 10:345–360
Fig. 9 Distribution of the neighborhood fuzzy hostility indices of the uniform noise affected synthetic images and their BDSONN extracted counterparts a at noise level of 64, b at noise level of 100, c at noise level of 144, d at noise level of 196 and e at noise level of 256
the proposed bi-directional network architecture and the multilayer self-organizing neural network (MLSONN) [24]. A MLSONN [24] architecture is a multilayer feedforward network architecture efficient for the extraction of binary objects from a binary noisy image scene. It comprises an input layer, any number of hidden layers and an output layer. It is characterized by the standard sigmoidal activation function with fixed and uniform thresholding. Inputs at the input layer of the network architecture are propagated to the inner layers of the network and finally these inputs are self-organized to extracted output objects. The self-organization procedure is effected by computation of system errors based on the linear indices of fuzziness of the output layer outputs and adjustment of the interconnection weights by means of the standard back-propagation algorithm. The output layer outputs are fed back to the input layer for further processing and the self-supervised learning algorithm is repeated until the system errors are reduced below tolerable limits.
123
In the present treatment, five different noisy versions of a synthetic image and a real life spanner image, each of dimensions 128 · 128, affected with five levels of uniform noise (noise levels of 64, 100, 144, 196 and 256) and Gaussian noise with zero mean and standard deviation values of r = 8, 10, 12, 14 and 16, are used. The original non-noisy images are shown in Fig. 2. The noisy Gaussian images and the respective extracted counterparts with BDSONN architecture are shown in Figs. 3 and 4. In addition, Figs. 3 and 4 also show the extracted images obtained using a MLSONN. Figures 5 and 6 illustrate the images extracted by the proposed BDSONN architecture and the MLSONN architecture when the images are affected by uniform noise. The proposed architecture is seen to maintain the shapes and boundaries of the images much better than the MLSONN architecture after extraction. Figures 7, 8 and 9, 10 show the distribution of the fuzzy hostility indices of the neighborhood fuzzy subsets in the noisy versions of the synthetic and spanner images along with the corresponding distributions in the BDSONN extracted counterparts for Gaussian and uniform noise,
Pattern Anal Applic (2007) 10:345–360
355
Fig. 10 Distribution of the neighborhood fuzzy hostility indices of the uniform noise affected spanner images and their BDSONN extracted counterparts a at noise level of 64, b at noise level of 100, c at noise level of 144, d at noise level of 196 and e at noise level of 256
Fig. 11 Variation of the coefficient of variation of the neighborhood fuzzy hostility indices during the extraction process for the Gaussian noise affected synthetic images at r = 8, 10, 12, 14, 16
respectively. Figures 11, 12 and 13, 14 show the variation of the coefficient of variation of the fuzzy hostility indices of the noisy and BDSONN extracted images during the object extraction process for Gaussian and uniform noise, respectively. Figures 15, 16 and 17, 18 indicate the corresponding variations in the fuzzy entropy measures of the
Fig. 12 Variation of the coefficient of variation of the neighborhood fuzzy hostility indices during the extraction process for the Gaussian noise affected spanner images at r = 8, 10, 12, 14, 16
fuzzy hostility indices during the extraction process for the Gaussian and uniform noise, respectively. From Figs. 7, 8, 9 and 10, it is evident that the gross neighborhood fuzzy hostility measures have been reduced after the object extraction process. The higher levels of
123
356
Fig. 13 Variation of the coefficient of variation of the neighborhood fuzzy hostility indices during the extraction process for the uniform noise affected synthetic images at noise levels of 64, 100, 144, 196, 256
Pattern Anal Applic (2007) 10:345–360
Fig. 16 Variation in the fuzzy entropy measures of the neighborhood fuzzy hostility indices for the Gaussian noise affected spanner images during the extraction process at r = 8, 10, 12, 14, 16
Fig. 17 Variation in the fuzzy entropy measures of the neighborhood fuzzy hostility indices for the uniform noise affected synthetic images during the extraction process at noise levels of 64, 100, 144, 196, 256 Fig. 14 Variation of the coefficient of variation of the neighborhood fuzzy hostility indices during the extraction process for the uniform noise affected spanner images at noise levels of 64, 100, 144, 196, 256
Fig. 18 Variation in the fuzzy entropy measures of the neighborhood fuzzy hostility indices for the uniform noise affected spanner images during the extraction process at noise levels of 64, 100, 144, 196, 256 Fig. 15 Variation in the fuzzy entropy measures of the neighborhood fuzzy hostility indices for the Gaussian noise affected synthetic images during the extraction process at r = 8, 10, 12, 14, 16
heterogeneity in the noisy versions of the image scene for the different levels of uniform and Gaussian noise are indicated by the dashed curves, where several different peaks of varying heights are present. Moreover, the smoothness of the curves also indicates the distributed
123
nature of the heterogeneity therein. This implies greater deviation among the fuzzy hostility measures in the different neighborhood fuzzy subsets. The continuous curves signify that a fair amount of reduction in the heterogeneity in the image scene is achieved after the object extraction process. This is reflected by the nature of the continuous curves with smaller number of well-demarcated comparable height peaks, which correspond to the
Pattern Anal Applic (2007) 10:345–360
357
Table 1 b values for the two images with Gaussian noise r
BDSONN
Table 6 Time in seconds for extracting images affected with uniform noise
MLSONN
Synthetic
Spanner
Synthetic
Spanner
8
1.00524
0.96182
1.197670
1.007658
10
1.05889
0.96959
1.432354
1.146836
12 14 16
1.10250 1.54134 1.74403
1.20619 1.23790 1.28930
1.593678 1.608356 1.771557
Noise
BDSONN
MLSONN
Synthetic
Spanner
Synthetic
Spanner
64
32
31
73
71
1.229339
100
32
30
72
74
1.238953
144
42
40
73
73
1.294104
196
42
50
73
115
256
61
93
116
157
Table 2 b values for the two images with uniform noise Noise
BDSONN
MLSONN
Synthetic
Spanner
Synthetic
Spanner
64
0.99385
0.97342
1.06297
1.06321
100
1.00176
0.99923
1.15593
1.07836
144
1.07850
1.02417
1.22951
1.16853
196
1.09673
1.06926
1.28872
1.21133
256
1.12036
1.10021
1.33993
1.28215
Table 3 pcc values for the two images with Gaussian noise r
BDSONN
MLSONN
Synthetic
Spanner
Synthetic
Spanner
8
99.94
99.63
99.39
98.08
10
99.55
98.98
97.78
96.52
12
98.96
97.76
96.28
94.78
14
98.27
96.93
95.79
94.42
16
96.86
95.72
95.04
93.38
Table 4 pcc values for the two images with uniform noise Noise
64
BDSONN
MLSONN
Synthetic
Spanner
Synthetic
Spanner
100
99.75
99.82
98.73
100
99.99
99.72
99.52
98.27
144
99.77
99.29
98.23
96.95
196
99.25
98.11
97.29
95.15
256
98.62
97.20
96.31
94.79
Table 5 Time in seconds for extracting images affected with Gaussian noise r
BDSONN Synthetic
MLSONN Spanner
Synthetic
Spanner
8
32
31
79
10
43
42
73
74 74
12
62
51
115
117
14
104
73
156
156
16
114
103
197
201
different extracted homogeneous regions in the image scene. The sharp isolation of the peaks from each other relates to the demarcation between the object and background regions. Figures 11 through 18 depict the stabilization characteristics of the network during the object extraction process. The curves of Figs. 11, 12, 13 and 14 represent the coefficients of variation of the neighborhood fuzzy hostility measures during the different intermediate phases of the object extraction process for the two images at different noise levels. It is seen that the coefficients of variation of the neighborhood fuzzy hostility measures exhibit an increasing trend in the initial stages of the extraction process. These coefficients, however, stabilize in the final stages of the extraction process. The stability achieved by the coefficients of variation of the fuzzy neighborhood hostility measures is thus an indication of the stability attained by the network. Figures 15, 16, 17 and 18 indicate the variation in the fuzzy entropy measures of the neighborhood fuzzy hostility indices during the extraction of the two images affected by different levels of noise. Higher initial values of the entropy measures as depicted in the curves are reflective of the higher degree of fuzziness in the image scene, which however, get reduced and settle to stable minima after the network converges indicating the stability achieved by the intensity distributions in the images. Thus, the degree of fuzziness in the image scene hostility indices is also a guiding factor in deciding the stability criteria of the network. 4.1 Extraction efficiency of the proposed BDSONN architecture The figure of merit of the object extraction procedure using the proposed BDSONN architecture can be judged from a systematic point of view. Considering the proposed architecture as a noise immune system, where noisy inputs are converted/transferred into non-noisy versions, the performance of the system can be represented as a system transfer index (b) from the noisy to the non-noisy domain.
123
358
Pattern Anal Applic (2007) 10:345–360
The system transfer index (b) of the architecture can be defined as the ability of the architecture to extract nonnoisy images from the degraded noisy versions of those images. It can be mathematically expressed as b ¼ qðO; NÞ qðN; EÞ
ð13Þ
where, q(O, N) refers to the ratio of the coefficients of variation of the neighborhood fuzzy subset hostility indices in the original and the noisy images and q(N, E) refers to the ratio of the coefficients of variation of the neighborhood fuzzy subset hostility indices in the noisy and the extracted counterparts. From the definition of the system transfer index, it is clear that closer the value of the index is to unity, the better extraction it reveals. Tables 1 and 2 show the b values for the two images for different levels of Gaussian and uniform noises using the proposed BDSONN architecture and the MLSONN architecture. From Tables 1 and 2, it is evident that the extraction performance of the network is better at lower noise levels and it degrades as the noise level goes up. Thus, the system transfer index (b) is an indirect measure of the noise immunity of the network architecture. Moreover, it is seen that the performance of the network is better for the extraction of the real life spanner image comparable to that of the synthetic image. This can be attributed to the relative proportion of the area of coverage of the image scene by the objects therein. It may also be summarized from the observation that the darker object regions are more vulnerable to noise than the brighter background regions. Moreover, the figures reflect the efficiency of the proposed architecture over the MLSONN architecture in terms of noise immunity. 4.2 Evaluation of the image quality of the extracted objects The quality of the extracted objects as regards to the degree of faithful extraction can be evaluated by means of the percentage of correct classification of pixels (pcc) [24] as object/background. It is given as pcc ¼
tcc 100 tnp
ð14Þ
where, tcc is the total number of correctly classified pixels and tnp is the total number of pixels in the image scene. Tables 3 and 4 list the pcc values for the two images for different levels of Gaussian and uniform noises using the BDSONN and the MLSONN architectures. Higher pcc values obtained indicate the better extraction capability of the proposed BDSONN architecture as compared to the MLSONN architecture.
123
4.3 Time efficiency of the proposed architecture The main objective of the proposed architecture is to attain self-organization without resorting to back-propagation based weight adjustment procedures. The selforganization is achieved by means of the counter-propagation of the network states and the adjustment of the network inter-layer interconnection weights is carried out deterministically using the relative fuzzy membership values of the constituent neurons at the different layers of the network. Thus, it is evident that the network is more time efficient as compared to other neural networking techniques, both supervised and unsupervised, which adapt the standard back-propagation algorithm based weight adjustment and error compensation methodologies. Tables 5 and 6 compare the operational time (in seconds) involved in the object extraction procedure using BDSONN and MLSONN, which uses the back-propagation based weight adjustment technique. It is clear from Tables 5 and 6 that the proposed BDSONN architecture outperforms the MLSONN counterpart in terms of the time taken in the extraction process.
5 Discussions and conclusion A three-layer BDSONN architecture applicable for realtime image processing applications is presented. The architecture uses counter-propagation of intermediate network states for self-organizing input information into outputs. The network dynamics and operation have been discussed. The network uses fuzzy context sensitive thresholding information for the processing task. The network interconnection weights are assigned and updated by the relative measures of the fuzzy membership values of the representative points in the image information, rather than through back-propagation strategies. The stability criteria of the network, which are decided by some fuzzy measures of the image scene, are also established through some empirical results on two image scenes with varying degrees of noise. The results of object extraction also show the shape restoring capabilities of the network in the extracted images. The efficiency of the proposed network architecture as regards to its immunity to different types of noise, the betterment of the quality of the extracted objects and the reduction in the time complexity of the object extraction process is also reported. The network in the present form is able to extract binary objects from a noisy image scene. However, the network dynamics can also be extended to extract multiscale objects from multiscale image scenes and pure/true color objects
Pattern Anal Applic (2007) 10:345–360
from color image scenes. Moreover, the avoidance of the standard back-propagation algorithm based weight adjustment procedure opens up the possibility of simple hardware implementation of the proposed network architecture. Methods also remain to be investigated for accomplishing moving object tracking from video sequences as well, using the proposed network architecture. The authors are currently engaged in these directions.
References 1. Suzuki K, Horiba I, Sugie N (2000) Fast connected-component labeling based on sequential local operations in the course of forward-raster scan followed by backward-raster scan. In: Proceedings of 15th international conference on pattern recognition, vol 2, pp 434–437 2. Chang F, Chen C-J, Lu C-J (2004) A linear-time componentlabeling algorithm using contour tracing technique. Comput Vis Image Underst 93(2):206–220 3. Gonzalez RC, Wintz P (1977) Digital image processing. Addison-Wesley, MA 4. Chen L-H, Chen J-R (2000) Object segmentation for video coding. In: Proceedings of 15th international conference on pattern recognition, vol 3, pp 383–386 5. Rosenfeld A, Kak AC (1982) Digital picture processing, vol 1. Academic, New York 6. Ekstrom MP (ed) (1984) Digital image processing techniques. Academic, New York 7. Meyer F, Beucher S (1990) Morphological Segmentation. J Vis Commun Image Represent 1(1):21–46 8. Hertz J, Krogh A, Palmer RG (1991) Introduction to the theory of neural computation. Addison-Wesley, Reading 9. Haykin S (1994) Neural networks: a comprehensive foundation. Macmillan College, New York 10. Lippmann RP (1987) An introduction to computing with neural nets. IEEE ASSP Magazine, 3–22 11. Forrest BM et al (1988) Neural network models. Parallel Comput 8:71–83 12. Pao YH (1989) Adaptive pattern recognition and neural networks. Addison-Wesley, New York 13. Chua LO, Yang L (1988) Cellular neural network: theory. IEEE Trans Circuits Syst 35:1257–1272 14. Chua LO, Yang L (1988) Cellular neural network: applications. IEEE Trans Circuits Syst 35:1273–1290 15. Bilbro GL, White M, Synder W (1988) Image segmentation with neurocomputers. In: Eckmiller R, Malsburg CVD (eds) Neural computers. Springer, New York 16. Abdallah MA, Samu TI, Grisson WA (1995) Automatic target identification using neural networks. SPIE Proc Intell Robots Comput Vis XIV 2588:556–565 17. Tang HW, Srinivasan V, Ong SH (1996) Invariant object recognition using a neural template classifier. Image Vis Comput 14(7):473–483 18. Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybern 43:59–69 19. Kohonen T (1984) Self-organization and associative memory. Springer, London 20. Hopfield JJ (1984) Neurons with graded response have collective computational properties like those of two state neurons. Proc Nat Acad Sci USA 81:3088–3092
359 21. Nasrabadi MN, Li W (1991) Object recognition by a Hopfield neural network. IEEE Trans Syst Man Cybern 21(6):1523–1535 22. Kosko B (1988) Bidirectional associative memories. IEEE Trans Syst Man Cybern 18(1):49–60 23. Kosko B (1992) Neural networks and fuzzy systems: a dynamical systems approach to machine intelligence. Prentice-Hall, Englewood Cliffs 24. Ghosh A, Pal NR, Pal SK (1993) Self-organization for object extraction using multilayer neural network and fuzziness measures. IEEE Trans Fuzzy Syst 1(1):54–68 25. Duda RO, Hart PE (1973) Pattern classification and scene analysis. Wiley, New York 26. Tou JT, Gonzalez RC (1974) Pattern recognition principles. Addison-Wesley, Reading 27. Chiu WC, Hines EL, Forno C, Hunt R, Oldfield S (1990) Artificial neural networks for photogrammetric target processing. SPIE Proc Close Range Photogramm Meets Mach Vis 1395(2):794–801 28. Zamparelli M (1997) Genetically trained cellular neural networks. Neural Netw 10(6):1143–1151 29. Datta AK, Munshi S, Bhattacharyya S (2000) Object extraction in artificial retina using cellular neural network optimized by genetic algorithm with fuzziness measure. In: Proceedings of international conference on fiber optics and photonics, vol 2, pp 723–725 30. Zadeh LA (1965) Fuzzy Sets. Inform Control 8:338–353 31. Ross TJ, Ross T (1995) Fuzzy logic with engineering applications. McGraw Hill College Div. 32. Deluca A, Termini S (1972) A definition of non probabilistic entropy in the setting of fuzzy set theory. Inform Control 20:301– 312 33. Bhattacharyya S, Dutta P, Maulik U (2006) A self supervised bi-directional neural network (BDSONN) architecture for object extraction guided by beta activation function and adaptive fuzzy context sensitive thresholding. Int J Intell Technol 1(4):345–365
Author Biographies Siddhartha Bhattacharyya did his Bachelors in Physics and Optics and Optoelectronics from Calcutta University, Kolkata, India in 1995 and 1998, respectively. Subsequently, he did his Masters in Optics and Optoelectronics in 2000 from Calcutta University, Kolkata, India. He is currently a Lecturer in the Department of Computer Science and Information Technology of University Institute of Technology, The University of Burdwan, Burdwan, India. He was a Lecturer in the Department of Information Technology of Kalyani Government Engineering College, Kalyani, India during 2001–2005. He has served as a Project Fellow in Bengal Engineering and Science University, Shibpore, India. He is a co-author of a book and about 30 research publications. His research interests include soft computing, pattern recognition and image processing. Mr. Bhattacharyya is a Fellow of OSI, India. He is a member of IAENG, Hong Kong.
123
360 Paramartha Dutta did his Bachelors and Masters in Statistics from Indian Statistical Institute, Kolkata, India in 1988 and 1990, respectively. Subsequently, he did his Masters in Computer Science in 1993 from Indian Statistical Institute, Kolkata, India. He did his Ph.D. in 2005 from Bengal Engineering and Science University, Shibpore, India. He is currently a Professor in the Department of Computer Science and Engineering of Kalyani Government Engineering College, Kalyani, India. He was an Assistant Professor and Head of the Department of Computer Science and Engineering of College of Engineering and Management, Kolaghat, India during 1998–2001. He has served as a Research Scholar in the Indian Statistical Institute, Kolkata, India and in Bengal Engineering and Science University, Shibpore, India. He is a co-author of four books and about 50 research publications. His research interests include evolutionary computing, soft computing, pattern recognition, multiobjective optimization and mobile computing. Dr. Dutta is a Fellow of OSI, India. He is the member of ISCA, CSI, IETE, India and IAENG, Hong Kong. Ujjwal Maulik did his Bachelors in Physics and Computer Science in 1986 and 1989, respectively. Subsequently, he did his Masters and
123
Pattern Anal Applic (2007) 10:345–360 Ph.D. in Computer Science in 1991 and 1997, respectively. He is a senior member of IEEE, USA. He is currently a Professor in the Department of Computer Science and Engineering of Jadavpur University, Kolkata, India. He was the Head of the Department of Computer Science and Engineering of Kalyani Government Engineering College, Kalyani, India during 1996– 1999. He has worked in the Center for Adaptive Systems Application, Los Alamos, NM, USA in 1997, University of New South Wales, Sydney, Australia in 1999, University of Texas at Arlington, USA in 2001, University of Maryland Baltimore County, USA in 2004 and Fraunhofer Institute AiS, St. Augustin, Germany in 2005. He has also visited many Institutes/Universities around the world for invited lectures and collaborative research. He is a coauthor of two books and about 100 research publications. He has been the Program Chair, Tutorial Chair and a Member of the program committee of many international conferences and workshops. His research interests include artificial intelligence and combinatorial optimization, soft computing, pattern recognition, data mining, bioinformatics, VLSI and distributed systems. Dr. Maulik is a Fellow of IETE, India. He is the recipient of the Govt. of India BOYSCAST fellowship in 2001.
