Hypergraph-based Image Representation Alain Bretto1 and Luc Gillibert1 Universit´e de Caen, GREYC CNRS UMR-6072, Campus II, Bd Marechal Juin BP 5186, 14032 Caen cedex, France.
[email protected],
[email protected]
Abstract. An appropriate image representation induces some good image treatment algorithms. Hypergraph theory is a theory of finite combinatorial sets, modeling a lot of problems of operational research and combinatorial optimization. Hypergraphs are now used in many domains such as chemistry, engineering and image processing. We present an overview of a hypergraph-based picture representation giving much application in picture manipulation, analysis and restoration: the Image Adaptive Neighborhood Hypergraph (IANH). With the IANH it is possible to build powerful noise detection an elimination algorithm, but also to make some edges detection or some image segmentation. IANH has various applications and this paper presents a survey of them.
Keywords: Image Processing, Image Model, Segmentation, Edge Detection, Noise Cancellation, Hypergraph, Graph, Neighborhood Hypergraph.
1
Introduction
Graphs are very powerful tools for describing many problems and structures in computer sciences but also in physic and mathematics. But graphs only describe some binary relations and are not always sufficient for modeling some complex problems or data. Hypergraph theory, originally developed by C. Berge [8] in 1960, is a generalization of graph theory. The idea consists in considering sets as generalized edges and then calling a hypergraph the family of these edges. This concept models more general types of relations than graph theory do. In the last decades, the theory of hypergraphs has proved to be of a major interest in applications to real-world problems. These mathematical frameworks can be used to model networks, data structures, process scheduling, computations, and a variety of other systems where complex relations between the objects in the system play a dominant role. To any digital image, a hypergraph, the Image Adaptive Neighborhood Hypergraph (IANH), can be associated and used for image processing. Many publications were written about this hypergraph model and many applications were found [1, 2, 4, 6]. This paper is a survey of different methods about image analysis and treatment based on this adaptive neighborhood hypergraph model. First, we give basic definitions about hypergraphs and the definition of the IANH. Then we present an algorithm building the IANH and we study its properties
and its complexity. Finally we illustrate some applications of the IANH to the image segmentation, the edge detection and the noise cancellation, three of the most important low level image processings. We give some powerful algorithms always based on the adaptive neighborhood hypergraph associated to an image. A set of examples is shown to illustrate the effectiveness of the algorithms.
2
Definitions
The general terminology concerning graphs and hypergraphs is similar to [8, 7]. All graphs in this paper are, finite, undirected, connected with no isolated vertex and simple, i. e. graphs with no loops or multiple edges. We denote a graph G = (V ; E). Given a graph G, we denote by Γ (x) the neighborhood of a vertex x, i. e. the set consisting of all vertices adjacent to x which is defined by Γ (x) = {y ∈ V, {x, y} ∈ E}. A hypergraph H on a set X is a family (Ei )i∈I of non-empty subsets of X called hyperedges with; [ Ei = X, I = {1, 2, . . . , n}, n ∈ N i∈I
Let us note H = (S; (Ei )i∈I ). For x ∈ S, a star of H (with center x) is the set of hyperedges which contains x, and is called H(x). The degree of x is the cardinality of the star H(x) denoted by dx = Card(H(x)). Let H = (S; E = (Ei )i∈I ) be a hypergraph, the dual hypergraph H ∗ is the hypergraph such that the set of vertices is the set of hyperedges, and the set of hyperedges is the set of stars of H. We can represent a hypergraph as in figure 1-(a). A hyperedge Ei is isolated if and only if: ∀j ∈ I, j 6= i, if Ei ∩ Ej 6= ∅ then Ej ⊆ Ei An important structure from a hypergraph is the notion of intersecting family. A family of hyperedges is an intersecting family if the hyperedges from this family intersect two by two. We can distinguish two types of intersecting families: – Intersecting families with an empty intersection. – Intersecting families with an non empty intersection. A hypergraph has the Helly property if each family of hyperedges intersecting two by two (intersecting family) has a non empty intersection (belongs to a star). As example in figure 1-(a) the hypergraph has the Helly property. Figure 1-(b) shows these two types of intersecting hyperedges. To each graph one can associate a hypergraph. Indeed, let G = (X, E) be a graph, the hypergraph having the vertices of G as vertices and the neighborhood of these vertices as hyperedges (including these vertices) is called the neighborhood hypergraph of G and is denoted by: HG = (X, (Ex = {x} ∪ Γ (x)))
(a)
(b)
Fig. 1. (a) Example of hypergraph, the set of vertices is {x1 , x2 , . . . , x1 3} and the set of hyperedges is {E1 , E2 , E3 , E4 }. (b) We have two types of intersecting families the first is the star the second has an empty intersection.
3
Image Adaptive Hypergraph Model
First we recall some definitions about digital images. A distance d0 on X defines a grid (a graph connected, regular, without both loop and multi-edge). A digital image (on a grid) is a two-dimensional discrete function that has been digitized both in spatial coordinates and in magnitude feature value. Throughout this paper a digital image will be represented by the application I : X ⊆ Z 2 → C ⊆ Z n with n ≥ 1, where C identifies the feature intensity level and X identifies a set of points called image points. The couple (x, I(x)) is called a pixel. Let d be a distance on C, we have a neighborhood relation on an image defined by: ∀x ∈ X, Γα,β (x) = {x ∈ X, x = 6 x | d(I(x), I(x )) < α and d0 (x, x0 ) ≤ β} 0
0
0
(1)
The neighborhood of x on the grid will be denoted by Γβ (x). So to each image we can associate a hypergraph called Image Adaptive Neighborhood Hypergraph (IANH): Hα,β = X, ({x} ∪ Γα,β (x))x∈X . The attribute α can be computed in an adaptive way depending on local properties of the image. If α is constant the hypergraph is called the Image Neighborhood Hypergraph (INH). Throughout this paper α will be estimated by the standard deviation of the pixels {x}∪Γβ (x). Algorithm: Image Adaptive Neighborhood Hypergraph Construction of the hypergraph Hα,β . Data: Image I of size mx × my , and neighborhood order β X=∅; For each pixel x of I, do ; α = the standard deviation of the pixels {x} ∪ Γβ (x); Γα,β (x) = ∅;
For each pixel y of Γβ (x), do if d(I(x), I(y)) ≤ α then Γα,β (x) = Γα,β (x) ∪ {y}; end if end for X = X ∪ {x}; Eα,β (x) = {Γα,β (x) ∪ {x}}; end for Hα,β = (X, (Eα,β (x))x∈X ); End Data structures used: For each x, Γα,β (x) is a table of booleans, so Eα,β is a mx × my table of tables. The set X is a mx × my table of booleans. Proposition 1. Given β, the algorithm converges to a unique solution. Its complexity is in O(n) (n standing for the pixel number of the image). (For the proof report to [2]).
4
Detection of Impulsive Noise
A common type of corruption that occurs in image data is corruption by an impulsive noise process. Attenuation of noise and preservation of details are usually two contradictory aspects of image. Various noise reduction algorithms make various assumptions, depending on the type of imagery and the goals of the restoration [9],[10]. In this section, we present a noise cancellation algorithm that exploits a lack of homogeneity criterion. We consider that the global homogeneity characterizes regions, local homogeneity characterizes edges, no homogeneity characterizes a noise. A noise reduction algorithm is based on the following criterion: binary classification of hyperedge of image (H0 noisy hyperedge and H1 no noisy hyperedge) and filtering the noisy parts. Noise definition - We will call disjoined chain a ordered succession of hyperedges disconnected two by two and build on some adjacent pixels. A disjoined chain is thin if the cardinality of each hyperedge is equal to 1. To model a noise we propose the following definition: We say that Eα,β (x) is a noise hyperedge if it verifies one of the two conditions: – The cardinality of Eα,β (x) is equal to 1 and Eα,β (x) is not contained in disjoined thin chain having five elements at least. – Eα,β (x) is an isolated hyperedge and there exists an element y belonging to the open neighborhood of Eα,β (x) on the grid, such that Eα,β (y) is isolated. (i.e. Eα,β is isolated and it has an isolated hyperedge in its neighborhood on the grid). This definition allows a good discrimination between edge pixels and noisy pixels. The lemma below shows that a noisy hyperedge must be isolated.
Lemma 1. If the cardinality of a hyperedge is equal to one, then this hyperedge is isolated. (For the proof report to [6]). With the noise definition above, a noise detection algorithm is simple. The IANH is built and all the hyperedges satisfying the conditions of the noise definition are selected. This selection is separated in two step, first the detection of the isolated hyperedges and then, in the set of the isolated hyperedges, the detection of the noisy hyperedges. Algorithm: Noise detection Data: Image I of size mx × my , IANH Hα,β . Determination of isolated hyperedges of Hα,β For each vertex x of Hα,β , do ; S 0 Ex = y∈Eα,β (x) Ey ; 0
If Ex == Ex , (Ex is an isolated hyperedge) then If Cardinality of Ex is equal to one then ISO[x] = Ex ; Else IS[x] = Ex ; end if end if end for Detection of noise hyperedges of Hα,β For each Ex of ISO[], do For each Ey of ISO[], and x 6= y do For each Ez of ISO[], and x 6= z and y 6= z do If y, z 6∈ Γβ (x) then N H[x] = Ex ; end if end for end for end for For each Ex of IS[], do If there is y ∈ Γ o (Ex ) such as Eα,β (y) ∈ ISO[] ∪ IS[], then N H[x] = Ex ; end if end for End Data structures used: The data structures used for the IANH and its hyperedges are the same that in the IANH construction algorithm. The sets ISO, ES and N H are some mx × my tables of hyperedges. The complexity of this algorithm is in O(n3 ). This algorithm has been tested on several images in order to show how effective our method is. This method has a great advantage over the class of linear filters; it preserves the edges, so
the additional complexity provides some additional results. Some experimental results can be found in [6]. Some visual examples are shown in figure 2. Natural image
Image corrupted
Noise detection
Fig. 2. Example of IANH-based noise detection and cancellation.
5
Segmentation
One of the first major step of low level vision is segmentation. Segmentation is the process which consists in partitioning an image into some non-intersecting regions such that each region is homogeneous and that the union of two adjacent regions is never homogeneous. The algorithm below will give us the segmentation of an image. This algorithm is based on the detection of stars in the hypergraph model. The algorithm process can be divided into in two main parts. In the first part, a covering of the image by a minimal set of stars is computed. In the second part, selected stars are aggregated to obtain the regions. This regions are the segments of the image. Algorithm: Covering and selection for segmentation Data: Image I of size mx × my , IANH Hα,β . Choosing a cover of the image by a minimal set of stars. Chose a minimal cover of the image, E = {H(x1 ), H(x2 ), . . . H(xn )}, such that any pixel of the image belongs to at most one hyperedge of at least one star of the set E. Building aggregate areas. For each H(xi ) in E, do I(xi ) = grey level of the center of the star H(xi ); Aggi = ∅ (initialization on a new aggregation area) For each H(xj ) intersecting with the star H(xi ), do If I(xj ) in [I(xi ) − α, I(xi ) + α], then Aggi = Aggi ∪ (vertices of H(xj ) ∪ H(xi )); end if end for end for
Reducing the number of areas. For each aggregate area Aggi , do gi = center of gravity of Aggi ; mini = minimum grey level of the centers of the stars of Aggi ; maxi = maximum grey level of the centers of the stars of Aggi ; medi = medium grey level of the centers of the stars of Aggi ; end for For each aggregate area Aggi , do For each area Aggj intersecting with Aggi , do If gi or gj is in Aggi ∪ Aggj , and mini in [minj − α, minj + α], and maxi in [maxj − α, maxj + α], and medi in [medj − α, medj + α], then Aggregate Aggi and Aggj ; end if end for end for Assigning each star to an aggregation area to obtain a partition. Chose the area Aggi containing the greatest number of stars For each star H in Aggi , do Assign the star H to Aggi ; For each aggregate area Aggj 6= Aggi , do Remove H from Aggj ; end for end for Repeat chose until all the stars have been assigned. Assigning the pixels generating edges For each pixel x in I, do If x in several stars, then assign x to the area of the star center whose grey level is the closest end if end for Finally, the pixels lefts are assigned to the area of the neighboring pixel which has already been assigned, and whose grey level is the closest. Data structures used: The sets E is a mx × my tables of hyperedges structured as the IANH in the other algorithms. Each aggregate area Aggi is a list. The set of all the aggregate areas is also a list. Visual example can be see in figure 3. Experimental results can be found in [1].
6
Edge detection
Another important aspect of image analysis is the edge detection. In a grey level image containing homogeneous objects, an edge is a boundary between two
Natural image
Segmentation
Fig. 3. Example of IANH-based image segmentation.
regions of different constant grey levels. If the difference is clear-cut between regions we have ideal edges. But there are several factors that degrade edges, such as noise or irregularities of the surface. An effective method for detecting edges must be based on an adequate definition of these edges. Let H be a hypergraph, we will call triangle an intersecting family of three hyperedges without common vertex. A triangle intersecting family (TIF) is an intersecting family which is not contained in a star. This means that the set of hyperedges family of a TIF do not share a common vertex. Theorem 1. Let H be a hypergraph and let IF = {E1 , E2 , . . . Ek } be an intersecting family. This family is a TIF if and only if IF contains a triangle. (For the proof report to [2]). Proposition 2. Let I be an image and Hα,β = X, Ex = ({x} ∪ Γα,β (x))x∈X ∗ be its INAH. Hα,β is isomorphic to Hα,β . (For the proof report to [2]). We will said that a pixel (x, I(x)) belongs to edges if it belongs to a triangle of Hα,β . So we can define an algorithm for detecting edges in an image. This one is based on three steps: (1) Construction of the line-graph of Hα,β with β = 1 (Algorithm: Line-graph). (2) 3-cliques detection in the line-graph. This step gives us the intersecting family of hyperedges in the image hypergraph (Algorithm: Detection of 3-cliques). (3) Test if the 3-cliques stand for a triangle in Hα,β (Algorithm: Hypergraph triangle test). All the pixels that belong to triangles in the third step of the algorithm correspond to the image edges. The following algorithm constructs the linegraph of the Image Adaptive Neighborhood Hypergraph. It easy to see that its complexity is O(n3 ) where n is the pixel’s number. Algorithm: Line-graph. Data: Image I, Image Adaptive Neighborhood Hypergraph Hα,β . Construction of the vertice’s set of L(Hα,β ) For each hyperedge Ex of Hα,β , do V (x) = ex ; end for
Construction of the edge’s set of L(Hα,β ). For each hyperedge Ex of Hα,β , do For each pixel y of Ex , do For each pixel z 6= y of Ex , do If BE[yz] = false, then E[xy] = {ex ; ey }; BE[xy] = true; end if end for end for end for End Then we must detect the 3-cliques. The edge number of L(Hα,β ) is in O(n) and there are n vertices, so the complexity of this algorithm is in O(n2 ). Algorithm: Detection of 3-cliques. Data: Line-graph L(Hα,β ). For each edge E[xy] of L(Hα,β ), do For each vertex z of L(Hα,β ), and z 6= x, y do If BE[xz] = true and BE[yz] = true, then T [xyz] = {ex , ey , ez }; end if end for end for End Finally we have to test if the 3-cliques stand for a triangle in Hα,β . The triangle number of L(Hα,β ) is in O(n3 ), so the complexity of this algorithm is in O(n3 ). Algorithm: Hypergraph triangle test. Data: Image Adaptive Neighborhood Hypergraph Hα,β and set of triangle T []. For each triangle T [xyz], do If Ex ∩ Ey ∩ Ez = ∅ then edges[xyz] = {x} ∪ Γα (x) ∪ {y} ∪ Γα (y) ∪ {z} ∪ Γα (z); end if end for End The method has been tested on a set of images. The results of our algorithm have beeen compared with the classical Canny-Deriche filter. Our method gives more meaningful edge maps and looks promising and more robust. Some visual examples are shown in figure 4.
7
Conclusion
Hypergraph-based image representation, as shown in this paper and in a lot of publications [1, 2, 4, 6], can be used for low-level treatment and gives some good results, only counterbalanced by the high complexity of the corresponding algorithms. Such representation is not limited to segmentation, edge detection and
Natural image
Edge detection
Fig. 4. Example of IANH-based edge detection.
noise cancellation. Very wide perspectives are opened by the use of hypergraphsbased models. By example, loseless and lossy picture compression can be performed with an appropriate hypergraph-based picture representation, as shown in [3]. In fact hypergraphs are very promising tools and can be applied in many other images process. Further works will be focused on: – Conception of algorithms in high level image processing using hypergraphs (by example for shape recognition). – Hypergraph-based compression for 3D images. – Hypergraph-based entropy on 2D and 3D images. – Generalisations of the IANH for 3D images, generalisation inducing some segmentation, edge detection and noise cancellation algorithms on 3D images.
References 1. A. Bretto, J. Azema, H. Cherifi, and B. Laget. Combinatoric and image processing. Computer Vision, Graphic and Image Processing (Graphical Model and Image Processing), 59(5):128–132, September 1997. 2. A. Bretto. Introduction to Hypergraph Theory and its Applications to Image Processing. Mongraphy in: Advances in Imaging and Electron Physics. Academic Press Elsevier, vol. 131, march 2004. 3. A. Bretto and L. Gillibert. Hypergraph Lossless Image Compression, preprint, submited to ITCC. 4. S. Chastel, P. Colantoni and A. Bretto. Displaying Image Neighborhood Hypergraph. 10th International Conference on Discrete Geometry for Computer Imagery (DGCI’2002). 5. A. Bretto and B. Laget. Helly property associated with the discrete planes. Southwest journal of pure and applied mathematics, 1:56–63, 1998. 6. S. Rital, A. Bretto, H. Cherifi and D. Aboutajdine. Application of Adaptive Hypergraph Model to Impulsive Noise Detection. CAIP’2001, Springer-Verlag, 2001, Warsaw, Poland, September 5-7, Lecture Note in Computer Sciences, 34–42. 7. C. Berge. Graphe. Dunod, 1983. 8. C. Berge. Hypergraphs. North-Holland Mathematical Library, 1989. 9. T. Chen and H. R. Wu. Adaptive Impulse Detection Using Center Weighted Median Filters. IEEE Signal Processing Letters, series 8, 2001, 1–3. 10. T. Chen and H. R. Wu. Application of Partition-Based Median Type Filters for Suppressing Noise in Images. IEEE Transactions on Image Processing, 2001.
