SEMI-AUTOMATIC EXTRACTION AND VECTORISATION OF MULTICOLOURED CARTOGRAPHIC OBJECTS V. Bucha *, S. Ablameyko* and T. Pridmore † *United Institute of Informatics Problems of the National Academy of Sciences Surganova, 6, 220012 Minsk, Belarus E-mail
[email protected] † School of Computer Science & IT, The University of Nottingham, Nottingham, UK
Keywords: interactive, semi-automatic, vectorisation, colour, map.
Abstract We describe a semi-automatic cartographic image interpretation system in which techniques previously applied to monochrome images are generalised to colour and extended to support digitization of the full range of multicoloured cartographic objects. A snapping tool eases the selection of seed points, speeding interpretation. Line and area objects are identified automatically and a suitable target representation (skeleton or contour) produced for each. This simplifies interaction as the user does not have to vary his/her behaviour with object type. Multi-coloured objects can be constructed from their components with ease and correction tools such as snapping and undo both save time and decrease error rates.
1 Introduction There is a longstanding and continuing need to convert paperbased cartographical information to digital form, primarily for use in geographic information systems After scanning to produce a colour image the map interpretation process can be broadly divided into three main stages: colour separation, raster-to-vector transformation to obtain a structural representation, and recognition of cartographic objects. Most systems aim to be fully automatic and are global in nature, applying the same processing to each image location [1-5]. Raster-to-vector transformation usually requires a binary image to be input. The colour separation stage must therefore convert the original colour image to a set of binary images, each depicting objects of one class (roads, rivers, buildings etc.). In many cases each object type is rendered in a single colour, making automatic colour separation tractable. Objects that significantly change colour or contain multicoloured subparts, can however become corrupted and may not appear fully in a single binary image. This generates errors at the vectorisation and object recognition stages that then need to be corrected, usually by hand. It is now widely accepted that fully automatic interpretation of cartographic images will not be achieved in the short to
medium term, and may indeed never be achieved. As a compromise, and to reduce the amount of manual errorcorrection needed following automatic interpretation, attention has turned to interactive, semi-automatic solutions. Interactive vectorisation methods [e.g. 6] have been developed which exploit local object models to interpret the image. Similar methods are implemented in almost every commercial raster-to-vector conversion system (ArcScan, EasyTrace, VPMax, I/Vector, etc). The operator typically indicates a starting point on a line segment and the system then attempts to trace an object. If a graphical ambiguity arises which the automatic component cannot resolve a message is returned to the operator. The user should have the ability to change the systems’ decisions at any time and easily correct its results. Semi-automatic techniques offer an opportunity to improve the interpretation of variable and multi-coloured objects. Existing semi-automatic methods, however, deal only with monochrome (grayscale or binary) images. These systems are typically capable of tracing only relatively simple constructs, often only lines, using a limited set of tools. Cartographic entities vary significantly, but can be broadly classified as either area or line objects. Any truly useful cartographic image interpretation system must be able to deal with entities of both types. In what follows we describe a semi-automatic cartographic image interpretation system in which techniques previously applied to monochrome images are integrated, generalised to colour and extended to support digitization of the full range of multi-coloured cartographic objects. A snapping tool eases the selection of seed points, speeding interpretation. Line and area objects are identified automatically and a suitable target representation produced for each, simplifying interaction. Line objects are represented by their skeletons, area objects by their boundaries. Multi-coloured objects can be constructed from their sub-components with ease and correction tools such as snapping and undo both save time and decrease error rates. The difficulties associated with the extraction of multicoloured cartographic objects are discussed in Section 2. The proposed system is overviewed in Section 3. Section 4
describes the colour distance and similarity measures employed, while line and area object detection and tracing are considered in Sections 5 and 6 respectively. Results are presented in Section 7 and conclusions drawn in Section 8.
Note the rich interaction between the different coloured segments. The dark, vertical, near-straight line to the right is broken into three pieces, two of which are separated by only a very small number of pink pixels. Note also the variation in colours near the boundaries between segments.
2 Multi-Coloured Cartographic Objects Figure 1a shows an example of a multi-coloured area object, which should be extracted as a single entity (Figure 1b). Note first that, although the original map is professionally printed, colour variation within e.g. the green region is clearly visible in the scanned image. More importantly, the target area contains sub-regions of widely different colour and shape. Some of the key components are very small (the dark circles) others, while large, are very narrow (the brown lines). These must not be left out of the final interpretation. This is particularly important near the boundaries of the construct, as contour-based representations are required for area objects. As Figure 1a demonstrates, multi-coloured sub-components often intersect the object boundary, rendering standard gradient and colour difference measures unable to locate the correct contour.
a)
b)
Fig. 2. A section of a colour map image (a) and the desired multi-colored line object (b). Figures 1 and 2 illustrate a key problem in cartographic image interpretation; colour information is vital to the correct interpretation of the image, but is not in itself sufficient to distinguish object class. In these circumstances, colour image segmentation cannot be a one-shot pre-process to an image interpretation system, as it is in existing fully automatic methods. A number of highly effective colour image segmentation methods have been developed in recent years [7-10], and reasonable quality automatic segmentations can now be achieved given comparatively simple images and a welldefined goal. Automatic segmentation of colour map images to the level required to support higher-level interpretation, however, is still beyond the abilities of current algorithms. Effective segmentation of map images typically requires significant user interaction to set parameters and evaluate intermediate results, both of which are difficult to achieve without a clear understanding of the segmentation algorithm being used and some knowledge of how a given segmentation would affect higher-level interpretation. Were standalone segmentation possible, a further problem would remain: the grouping of different regions to form higher-level area and line objects is a challenging task, which cannot yet be achieved automatically.
a).
b) Fig. 1. A multi-coloured area object (a) and the desired connected component (b).
Figure 2a shows a section of a map image containing a multicoloured line object. The desired object is shown in figure 2b.
Colour separation must be fully integrated into the interpretation process. Image interpretation naturally begins with initial colour segmentation, but thereafter the colour separation module should be directed (and the algorithms it employs parameterised) to meet the needs of vectorisation and entity recognition processes. In an interactive system the user should be able to direct and control the colour separation process, focusing attention and varying his/her actions in response to the intermediate results produced by higher-level entity extraction and description modules.
3 System Overview The proposed system (Figure 3) comprises five stages: local colour separation, area and line object detection, local thinning or contour extraction, object tracing and automatic object vectorisation. In addition, tools for seed point snapping and real-time control of the process are available. First, an initial segmentation is performed. The operator specifies a seed point and colour tolerance parameters, which are used to define a boundary condition of the region to be sought. The region-based algorithm described in [11] is applied, growing a connected component of near-uniform colour by adding suitable neighbouring pixels to the seed. This algorithm is both memory- and computationally efficient. Manually placing a seed point inside the required region is tedious and often difficult, particularly when small components (Fig 1.) are to be extracted. If the chosen seed lies outside the object or near an object boundary frustrating errors can occur. To facilitate seed point placement, cursor snap facility and real-time region highlighting are provided. The snapping tool highlights and/or forces the mouse cursor to the nearest minimum gradient magnitude pixel within a user specified neighborhood. Thus, as the mouse cursor is moved by the user, it snaps to nearby, potentially “good” seed pixels. If the cursor stops at any pixel for a time T, then the connected component which would be extracted using that pixel as a seed and the current parameter set is highlighted.
required to form each representation. A bounding contour is describes area objects while skeletons are extracted from line entities. Either local thinning or contour detection is then performed, depending on the class of the object in question. The local nature of this processing distinguishes this step from the globally-applied algorithms used in fully automatic digitisation systems. Focusing on a connected component selected by the user reduces the memory and computational resources required, making real-time display and control of the process practical. The thinning algorithm described in [12] is employed here. The next stage is object tracing, in which distinct components are semi-automatically combined to produce descriptions of multi-coloured objects. Given two line components, the system first attempts to find a path, lying entirely within them, which connects their seed points. If such a path is found, the two regions are combined and vectorised as a single entity. If not, user must decide how to connect the regions. There are two possibilities. He/she may enter further seed points, producing a additional connected components linking the existing ones. Alternatively, the corrupt area may be reconstructed in manual mode. Classical vectorisation and approximation is used to produce the desirable vector representation. In addition, information (colour, thickness,) about the underlying connected component(s) can be recorded. Area objects are combined using object-oriented mathematical morphology.
4 Colour Distance Measures Some definition of colour distance is required to support the measurement of colour gradient and estimation of boundary conditions by the seed point snapping tool and the subsequent extraction of connected components. The RGB model is convenient when storing and transmitting colour information, but it can be difficult to design colour image processing algorithms in this space due to the high correlation between the three RGB planes. Human perception of colour qualities is reflected more accurately by the HSI or HSV models [13]. As hue and saturation are relatively independent of intensity, HSI allows more robust measures of the difference between two colours to be made. However, HSI becomes unusable at low intensities, when a small variation in any one of the three RGB values can produce a large difference in the angle between colours. Low intensity areas are not a problem for RGB, Euclidean distance between two sets of RGB values provides a reasonable measure of colour separation. Fig. 3. The architecture of the system. Once a connected component has been identified, the class (line or area) of that component is determined automatically. This stage is necessary since area and line objects are represented in different ways and separate algorithms are
There are clear advantages to be gained by combining RGB and HSI in colour distance measures. This may be achieved in a number of ways. A saturation and intensity based combination of hue and intensity planes was attempted in [13]. An estimate of the relevance of intensity and saturation difference measures is made in the following manner [13]
α(I ) =
1 − slope ( I −offset )
α (S ) =
1
R=
− slope ( S −offset )
Area[ skeleton] Area[contour ]
1+ e 1+ e where α (I ) and α (S ) are smooth transfer functions, offset where Area[] is an operator that counts the number of pixels defines the transition midpoint and slope describes the slope at that point. Both offset and slope are set experimentally. As intensity increases, the function’s output slowly moves from 0 towards 1. The combined function for two points is defined as
ρ ( I1 , I 2 , S1 , S 2 ) = α ( I1 ) ⋅ α ( I 2 ) ⋅ α ( S1 ) ⋅ α ( S 2 )
in a component. If R is lower then the threshold a contour representation is chosen over a skeleton. However to estimate the threshold value the contour and skeleton representations must both be estimated. This is too time consuming to employ within a semi-automatic system where real-time feedback is required.
This is done as HSV should be used only if both points have high intensities and saturations, otherwise RGB or a combination of both should be used. The Euclidian metric in RGB space can be defined as
E ( P1 , P2 ) =
( R1 − R2 ) 2 + (G1 − G2 ) 2 + ( B1 − B2 ) 2 255 3
w
here 255 3 is a igitize d value and P1 and P2 are colour feature vectors which consist of R, G, B, H, S, I. The anglular distance in HIS space is given by Fig. 4 Thinning area objects
min( H 1 − H 2 , 360 − H 1 − H 2 ) Instead, we based area and line object recognition on the following measure, which does not require thinning to be 180 The combined color distance between two points P1 and performed [12]. w = 0.25 ⋅ ( P − P 2 − 16 ⋅ S ) P2 can be computed in the following manner A( P1 , P2 ) =
ρ ( I , I , S , S ) ⋅ A( P1 , P2 ) + C EA ( P1 , P2 ) = 1 2 1 2 (1 − ρ ( I1 , I 2 , S1 , S 2 )) ⋅ E ( P1 , P2 )
The perimeter P and square S of a connected component can be efficiently obtained on the fly during initial segmentation. Depending on the value of w, a contour or line representation is extracted.
C EA is used for gradient estimation by the seed point 6. Object Tracing
snapping tool.. Colour tolerance during initial segmentation is specified by a single threshold value 0 ≤ CT ≤ 1 . Point P1 is considered to lie in the same connected component as
P2 if
C EA ( P1 , P2 ) ≤ CT 5 Line and Area Object Detection Without the ability to distinguish different object types there is a danger that important information will be lost when an inappropriate representation is computed. Skeletons do not provide an adequate representation of area objects (Fig. 4 ) but are more efficient and better suited to line objects than contour-based techniques, which allow fast editing and coding of area objects. There are several approaches to detection of information loss through inappropriate representation and the classification of area and line objects [12,14] . The technique reported in [12] detects information loss during a thinning procedure by applying a threshold to the value R
In the object tracing stage two or more connected components of a multicoloured object must be combined. Distinct methods are applied to line and area objects.
6.1. Tracing Line Objects Consider the situation in which two multicoloured line objects overlap (Fig. 5(a) or intersect (Fig. 6(a)). Suppose the operator’s goal is a vector representation of a line linking pixels E1 and E2 (Fig. 5(a)). Two seed pixels, S1 and S2, are specified (Fig. 5(a)), and two connected components are extracted (Fig. 5(b). If a connected path can be achieved between S1 and S2 then vectorisation can be done automatically without further user intervention. Overlaps and intersections are, however, common. Additional path reconstruction processes will often be required. First, thinning applied and the skeleton of each component is estimated (marked as a thin white line in figures 5 and 6). Next, feature pixels are found. We define the following types of feature pixel:
• end pixels • node pixels • connected pixels Characteristic numbers are used to determine to which group a given pixel belongs. Pixels are considered to be end points if {A8 , N c 8 , Cn } = 1 , nodes if {N c , C n } ≥ 3 . A pixel is
connected if {N c , C n } =
2.
a) b) Fig. 8 A colour map image with seed pixels and the connected components extracted (b)
The classical operations of closing and opening, however, are not suitable as they corrupt the boundary of the resulting object (Fig. 9(b)). The black pixels in Fig. 9(b) mark the affected pixels. To avoid this problem we use object-oriented morphological operators. a) b) Fig. 5 Interaction of two objects (a) and extracted components (b)
When feature pixels have been marked, the path between seed points can be found.. We examine the neighbourhood of each end point to find places where a line approximation should take place. For each end pixel belonging to one connected component the distance to the nearest pixel on the skeleton of another component is computed. A line then connects the pair of pixels with shortest distance between them (Fig. 5(b)). AN upper limit R is placed on the length of these connecting lines At any time the user can decide if and how to create a line between pixels: to select the most appropriate path, to vary R or to switch to manual approximation mode. The results of applying this process to the examples of figures 5 and 6 are shown in figure 7.
a) b) Fig. 9 Combined object (a) and difference between original components (b)
a) b) Fig. 10 Result of object-oriented dilation (a) and difference between original components (b)
Fig. 6 Tracing a Multicoloured line object
Fig. 7 Digitisation of multi-coloured objects.
6.2. Tracing Area Objects Given area objects the goal is to combine regions. Suppose the operator wishes to igitize the green area of (Fig. 8(a)). Two seed pixels produce two connected components (Fig. 8(b)). Mathematical morphology can be used to combine these components (Fig. 9(a)).
Object-oriented morphological operators are executed only when pixels arising from different connected components are under consideration. In other words, the morphological operators do not affect pixels of the same component. This prevents the corruption of external and internal boundaries of components. The result of the operation is shown in Fig. 10.
7 Results The system described here has been used to digitise area and line objects in a variety of topographic colour maps (e.g. Fig.
11). Figure 12 (a,b,c) shows the boundaries of area and skeletons of line objects extracted in this way.
Acknowledgements This research was supported in part by a Royal Society Joint Project.
References [1] S. Ablameyko, V. Bereishik, M. Homenko, D. Lagunovsky, N. Paramonova and O. Patsko. “A complete system for interpretation of color maps.” International Journal of Image and Graphics, 2 ,3, (2002). [2] N. Ebi, B. Lauterbach, and W. Anheier, “An Image analysis system for automatic data acquisition from colored scanned maps.”, Machine Vision and Applications, 7, (1994), 148-164.
Fig. 12 Color cartographic image
[3] S.Levachkine, “Raster to Vector Conversion of Color Cartographic Maps for Analytical GIS.”, Fifth IAPR International Workshop on Graphics Recognition, 2003. [4] K. H. Lee, S. B. Cho and Y.C. Choy, “Automated vectorization of cartographic maps by a knowledge-based system.”, Engineering Applications of Artificial Intelligence, 13, 2, (2000), 165-178. [5] J.M. Ogier, S. Adam, A. Bessaid, and H. Bechar, “Automatic topographic color map analysis system.”, Proc. 4th Int. Workshop of Graphics Recognition, (2001), 229-244. [6] A. Maeda, K. Wakimoto, H. Nagahisa, I. Nakamura, “An interactive tool for recognizing structured figures with variable shapes”, Machine Vision and Applications, Vol. 11, 1998, 24-36 [7] H.D. Cheng, X.H. Jiang, Y. Sun, Jingli Wang, “Color image segmentation: advances and prospects, Pattern Recognition”, 34, 2001, 2259-2281.
a)
b)
c)
Fig. 13 Extracted area (a, c) and line (b) objects
Comparisons of processing times and boundary accuracy achieved have been made between the proposed semiautomatic system and a standard manual tracing method. Using manual digitization, digitization of the objects in Figure 13 a-c took 120, 50 and 68 seconds respectively. Using the semiautomatic system the same interpretation took at most half those times. The accuracy of the boundary and skeletons extracted by the semi-automatic system was also better in each case. When properly targeted, automatic thinning, contouring and entity extraction are more stable than manual tracing.
8 Conclusion The proposed approach provides an accurate and efficient semi-automated tool for the extraction of multicoloured line and area objects extraction from colour cartographic images. The enhanced interactivity provided by the system, supports high-quality digitisation even in very difficult situations, when automatic systems fail.
[8] J. Shi and J. Malik, Normalised cuts and image segmentation, IEEE Trans. PAMI, 22, 8, 2000, 888-905. [9] L. Vincent and P. Soille, Watersheds in digital spaces, IEEE Trans PAMI, 13, 6, 1991, 583-598. [10] Dempster, A.P., N.M. Laird and D.B. Rubin, Maximum likelihood from incomplete data using the em algorithm, Journal of the Royal Statistical Society, B39, 1977. [11] Smith, Alvy Ray, “Tint Fill”, Computer Graphics Vol. 13, pp. 276-283, 1979. [12] Lei Huang, Genxun Wan, Changping Liu, “An Improved Parallel Thinning Algorithm”, Proceedings of the Seventh International Conference on document Analysis and Recognition (ICDAR 2003) [13] Theo Gevers, Arnold W.M. Smeulders, “Color-based object recognition”, Pattern Recognition, 32 (1999), 453-464. [14] S. Ablameyko, T. Pridmore, “Machine Interpretation of Line Drawings Images ”, Springer, 2000
