Cartographic continuum rendering based on color ... - La recherche IGN

ISPRS Journal of Photogrammetry and Remote Sensing 127 (2017) 27–38

Contents lists available at ScienceDirect

ISPRS Journal of Photogrammetry and Remote Sensing journal homepage: www.elsevier.com/locate/isprsjprs

Cartographic continuum rendering based on color and texture interpolation to enhance photo-realism perception Charlotte Hoarau ⇑, Sidonie Christophe Univ. Paris-Est, LASTIG COGIT, IGN, ENSG, F-94160 Saint-Mande, France

a r t i c l e

i n f o

Article history: Received 18 February 2016 Received in revised form 13 July 2016 Accepted 14 September 2016 Available online 20 October 2016 Keywords: Continuum Map design Interpolation Color Texture Photo-realism perception

a b s t r a c t Graphic interfaces of geoportals allow visualizing and overlaying various (visually) heterogeneous geographical data, often by image blending: vector data, maps, aerial imagery, Digital Terrain Model, etc. Map design and geo-visualization may benefit from methods and tools to hybrid, i.e. visually integrate, heterogeneous geographical data and cartographic representations. In this paper, we aim at designing continuous hybrid visualizations between ortho-imagery and symbolized vector data, in order to control a particular visual property, i.e. the photo-realism perception. The natural appearance (colors, textures) and various texture effects are used to drive the control the photo-realism level of the visualization: color and texture interpolation blocks have been developed. We present a global design method that allows to manipulate the behavior of those interpolation blocks on each type of geographical layer, in various ways, in order to provide various cartographic continua. Ó 2016 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V. All rights reserved.

1. Introduction The increasing diversity of geographical data provides a wide range of representations of the real world. Users are already able to access and visualize heterogeneous data such as topographic vector databases, ortho-images, DTMs, Lidar point clouds, thematic raster data (weather, pollution, population density, etc.). When considering to visualize them together, visual heterogeneity appears and may prevent users from reading and understanding the represented territory. Human computer interaction research proposes a wide range of co-visualization tools such as magnifiers, lenses, swipes, or enslaved views (Pindat et al., 2012; Karnik et al., 2009; Lobo et al., 2015). Nevertheless providing hybrid visualizations would allow merging and visually integrating heterogeneous data in the same visualization: in particular, those visualizations would allow to control abstraction and photo-realism levels. Our long-term research issue is to address the problem of suitable and comprehensive graphic representations of geographical spaces while taking advantage of existing representations to better fit users’ needs and preferences. Ortho-images have been used for a long time as backgrounds to overlay vector data or thematic information (Bildirici et al., 1999; Donnay, 2000; Albertz and Lehmann, 2005; Bianchin, 2007). Their relevancy to possibly better support ⇑ Corresponding author.

some cartographic tasks than abstract representations has been experimented (Wilkening and Fabrikant, 2011; Raposo and Brewer, 2011; Boér et al., 2013; Cöltekin et al., 2015). We assume that topographic map design could be improved by mixing relevant visual properties coming from maps and orthoimages, and by controlling the level of photo-realism. Even if map designers suggest ways to integrate ortho-imagery backgrounds or to blend heterogeneous data together (Raposo and Brewer, 2013; Hoarau et al., 2013; Murphy, 2015), few research focus on providing methods to continuously browse the cartographic space delimited by the data to hybrid, driven by the photo-realism level. In this paper, we aim at designing continuous hybrid visualizations between symbolized vector data and a related ortho-image, while providing controls on the photo-realism level. Our global approach relies on the interpolation of visual properties coming from both representations of both continuum ends. Our proposition is based on: An extraction of colors and the handling of texture effects coming from the imagery, to be faced to colors of symbolized vector data, as relevant visual properties to reduce or enhance the perceived photo-realism (natural appearance). The interpolation of color and texture with the help of elementary blocks, in order to smoothly navigate between visual properties of each geographical layer (color and texture interpolation).

E-mail address: [email protected] (C. Hoarau). http://dx.doi.org/10.1016/j.isprsjprs.2016.09.012 0924-2716/Ó 2016 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V. All rights reserved.

28

C. Hoarau, S. Christophe / ISPRS Journal of Photogrammetry and Remote Sensing 127 (2017) 27–38

A global design method to manage the behavior of those interpolation blocks for each considered geographical layer, and thus make possible cartographic continua between vector data and an ortho-image (global design method). In the following, Section 2 provides the related work. In Section 3, we detail our approach to control the interpolation blocks to enhance the level of photo-realism in a cartographic continuum. Then, we precise the main components of this framework, natural appearance extraction in Section 4 and color and texture interpolation in Section 5. Section 6 illustrates the genericity of our global method by providing the design method to manipulate the interpolation blocks, according to the set of input vector data, in order to make a cartographic continuum to an ortho-image.

2. Related work The availability of heterogeneous geographical data raises new map design issues and invites map designers to revisit theoretical cartography principles. The introduction of ortho-imagery background especially encourages map design, computer graphics and image rendering scientists to make their rendering techniques converge to control abstraction and photo-realism levels in geovisualization tools. Issues of renderings methods and parametrization of the visual properties to control abstract and photo-realist cartographic styles are at stake. The efficiency of various data and representations to fit to users’ needs and tasks, has been evaluated by visual experimentation. In particular, the photo-realism in maps is evaluated regarding its capacity to support some users’ tasks. Users might prefer more realistic looking maps, whereas they do not necessarily perform better with them (Wilkening and Fabrikant, 2011). Correlations between the performance of some cartographic tasks and the abstraction/realism levels of the given representations have been experimented and observed (Hoarau, 2012; Bernabé-Poveda and Çöltekin, 2014; Cöltekin et al., 2015). Therefore, the need for geovisualizations with different levels of abstraction and realism regarding every-day cartographic tasks has been clearly claimed (Boér et al., 2013). Map design research have addressed the design problem of visually merging an ortho-image background with an abstract topographic map for a long time ago. Nevertheless, legibility issues are still at stake, implying issues in graphic semiotics (Bertin, 1983), mostly on colors, color contrasts, and size of objects, but also in rendering techniques to variously blend geographical layers (Porter and Duff, 1984). The adaptation of toponyms regarding orthoimagery background colors and contrasts has been explored (Bildirici et al., 1999; Albertz and Lehmann, 2005). The unique use of the transparency to blend topographic map and orthoimages, in most of geoportals, implies a visual scramble of the information and is thus insufficient to help hybrid those data (Hoarau, 2012). In such a context, researchers suggest specific symbolization methods that take into account orthoimagery background. Raposo and Brewer (2013) provide guidelines to symbolize roads and rivers with a survival symbolization regarding background toggle between topographic map and orthoimagery backgrounds. Hoarau et al. (2013) propose a locally adaptive symbolisation of road casing that take into account orthoimagery colors around considered road features. At the contrary, Murphy (2015) provides imagery processing methods to make features salient in the imagery background regarding an overlaid vector data. Researchers in computer graphics, expressive rendering or geovisualization aim at exploring photorealistic and non-photorealistic rendering techniques to control the style of their visualizations (Brasebin et al., 2015; Masse and Christophe, 2015; Christophe

et al., 2016). In geovisualization, this issue intends to increase the realism of maps, based on photorealism or non-photorealism techniques, in order to make them more realist, expressive and thus efficient. Features natural appearance is used as an inspiration source by several map designers. For instance, natural color maps are investigated (Patterson and Kelso, 2004), relief realism is based on its enhancement by illumination (Patterson, 2002) and by natural texturing (Jenny and Jenny, 2012). Map designers also explore the potential of texturing rendering techniques coming from graphic computers, by synthetic vectorial textures (Loi et al., 2013; Jenny et al., 2014), by watercolorization on oblique views (Jenny et al., 2015); water surfaces are rendered by realist textures (Patterson, 2002), animated textures (Yu et al., 2011), by expressive renderings (Semmo et al., 2013). The concept of continuum has been defined as ‘a series of pictures, iteratively reduced in representation from its referent’, based on photographs, drawings, sketches, etc.: precisely, images of greater realism help to solve the homogeneity problem (distinguishing objects in the same class), whereas images of reduced or distilled detail facilitate object hypotheses (distinguishing between classes of objects) (Medley and Haddad, 2011). Based on this definition, several research works aim at controlling the photo-realism and abstraction levels of a cartographic representation in so-called cartographic continuum. In order to make progressive transitions between various levels of abstraction, the parameterization of rendering methods is explored through various strategies to distribute the level of abstraction in the representation (Semmo et al., 2012, 2013; Semmo and Döllner, 2014): according to the distance from the image center or the saliency of rendered objects (Semmo et al., 2012), river rendering according to more or less cartographic styles (Semmo et al., 2013), more or less complex textures according to scene depth and expected abstraction level (Semmo and Döllner, 2014). Metrics are also used to describe the level of detail in order to make discrete scales of this level of detail (Biljecki et al., 2014). Another lead consists in using techniques to capture the visual attention of the users. For instance, an image is decomposed into a zone of interest (focus zone) and its periphery in the visual field (context zone): irrelevant details are perceptually removed based on a model of the foveal vision (Bektas and Çöltekin, 2012; Bektas et al., 2015). Another example consists in generating masks according to relevant geographical objects and scene depth and highlight specific objects as soon as the users perceive them (Trapp et al., 2011).

3. Approach: controlling color and texture interpolations between vector data and ortho-image Previous research invites us to consider the user control of the level of photo-realism of their geovisualization in order to adapt their representations to the tasks they have to achieve. We need tools and methods to browse the cartographic representation space. In particular, continuous transitions between representations are required in order to make a cartographic continuum, helping users to find the more suitable representation they need. We propose a global design method to make various cartographic continua between two types of representation, by interpolating their graphic parameters. In order to manage these continuous transitions, we consider the level of photo-realism as a main visual and perceptual property to control. Continuous transitions are based on interpolations guided by crossing points selected according to salient visual properties. Current trends in natural map design invite us to focus on the natural visual properties coming from ortho-imagery, i.e. natural colors and textures, in order to convey orthophoto-realism. Color is a powerful visual variable conveying realism perception, when conventionally used

29


to render geographical objects (blue for hydrography and sea, green for vegetation, etc.) and has been widely explored in map design research (Brewer, 1994; Brewer et al., 2003; Christophe, 2011; Brychtova and Cöltekin, 2015). The texture visual variable has been also defined in graphic semiotics (Bertin, 1983): we intend to go further and suggest to take into account orthoimagery backgrounds but also procedural rendering textures, in order to extend the visual properties of the texture visual variable, in the context of hybrid visualizations to control smooth transitions. Our approach is the following: We use the colors and textures coming from ortho-images as relevant visual properties to handle photo-realism perception when being injected in abstract representations (Section 4). We describe color and texture interpolation blocks independently, in order to highlight the various possibilities of continuous transitions they provide: interpolating colors and textures is a relevant lead to manage smooth continuous transitions on one geographical layer, between its representations in vector data and in ortho-imagery (Section 5). We present the global design method to differently manipulate the previous interpolations blocks, for each geographical layer, in order to make a cartographic continuum. According to users strategies of how to control the behavior of the interpolation blocks layer by layer, a wide variety of cartographic continua may be created (Section 6). Fig. 1 presents the framework of our proposition, based on input data, natural appearance and interpolation blocks, being controlled, for each geographical layer, by the global design method.

4. Natural appearance coming from orthoimagery This section describes the two relevant visual properties of the natural appearance coming from orthoimagery we would like to be able to control: the natural color and the texture effect. We intend to use them to smoothly and continuously enhance the photorealism level in the intermediate representations.

4.1. Natural colors extraction Analyzing color distribution on a set of ortho-images shows a very particular range of colors, very different to the one of topographic maps. Indeed, ortho-imagery colors are globally dark and less vivid than topographic map colors. Therefore, we consider that the colors conveyed by ortho-images, that we call natural, are relevant visual properties to be injected in the map design process, to handle the photo-realism level. Our method extracts the natural colors for each type of geographical features: we assume that a type of features share similar colors (greenish colors of a forest, blueish colors of a lake or a river, etc.).

We use related vector data and build a binary mask with their footprints. The ortho-image is then cut out regarding this binary mask: hence, pixels included in feature footprints are selected and extracted as an ortho-image patch. Finally, the main color of the ortho-image patch is computed by color classification for each type of geographical features (Fig. 2). We use the color classification method described in Christophe et al. (2013), i.e. a K-Means classification in the CIELab uniform color space, and keep the mean color of the main color cluster.

4.2. Texture effect handling In order to manage the photo-realism of visualizations, we use the potential of texture effects when hybridizing various representations, i.e. mixing color, initial texture and transparency to provide texture effect. Vector data are mostly represented by plain colors (providing a neutral texture effect) while ortho-imagery convey various textures according to features in the image (providing a natural effect). Procedural textures may also be used with the other type of textures, color and transparency, in order to provide realistic effects well adapted to depict natural elements because of the randomness of noise functions. They are generally defined by a set of controllable parameters. In this paper, we explore such texturing methods with the help of the Perlin Noise texture (Perlin, 1985) which is continuous, multi-scaled and provide a pseudo-random appearance that allows to increase the realism effect. Moreover, as a procedural texture, the Perlin Noise Texture can be parameterized by a large set of parameters: its color range, its scale, its amount, its stretch and its orientation angle. Fig. 3 illustrates the different types of texture effects we can manage (neutral, natural, procedural and mixed) according to the initial input texture, associated color and level of transparency. This figure provides also visual clues to consider the variety of potential designs based on colors and textures types variations. Considering that a vector layer is overlaid to an ortho-image, modifying the opacity of the vector layer will make appear the natural texture of the ortho-image. Another way to convey the natural texture of the ortho-imagery is to convert the ortho-imagery in greyscale: this rendering method allows to manipulate independently the natural texture and colors of the ortho-imagery. Finally, blending modes can be used to blend colors from overlaid vector data and natural colors from the ortho-image. We use the Overlay mode described by Porter and Duff (1984) because it preserves highlights and shadows of the background by reflecting the lightness or darkness of the backdrop color when mixing them with the overlaid color (Cf. Eq. (1)).

Ov erlayðv 1 ; v 2 Þ ¼ where value.

v1

2v 1 v 2 if

v 1 > 0:5 1 2ð1 v 1 Þð1 v 2 Þ otherwise

is the backdrop color value and

Fig. 1. Our framework to make a cartographic continuum.

v2

ð1Þ

is the overlaid color

30


5.1. Color interpolation block

Fig. 2. Extraction of the natural color of the sea from orthoimagery and sea vector data. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 3. Texture effect handling based on various possible colors and textures, in order to define vegetation symbolizations. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

4.3. Injecting natural visual properties in map design Natural colors and different textures may be handled independently, in order to be used jointly to represent each geographical layer and thus provide a wide variety of map designs. Fig. 3 provides samples of crossed color-texture effects that can be managed during the design. Fig. 4 provides four different examples of map designs based on the expressiveness of those color-texture effects, that could be used according to users’ tasks, needs or preferences: vector data symbolized with natural colors (top-left), vector data footprints symbolized with imagery patches conveying natural textures (top-right), vector data symbolized with natural colors overlaid to a grayscaled imagery background conveying natural (but non colored) textures (bottom-left), vector data symbolized with natural colors overlaid to an imagery background conveying natural textures (bottom-right). 5. Color and texture interpolation The challenge of interpolating color and texture is to take into account the specificity of their different types in smooth transitions: it implies to provide a set of interpolation blocks, each of them being adapted to color, opacity, amount of procedural texture, etc. For all those possible interpolation blocks, we aim at creating a regularly evolving graphic path all along our continuum. Therefore, we use a linear interpolation (Cf. Eq. (2)).

f :R ! R x # f ðxÞ ¼ ya þ ðx xa Þ

yb ya xb xa

ð2Þ

where f ðxÞ is the interpolated value at the interpolation step given by x, and (xa ; ya ), (xa ; ya ) the coordinates of the end parameters.

In cartography, color interpolation is often used to create sequential color schemes. Brychtova and Cöltekin (2015) encourage to use varying visual distances between colors within a color scheme to ensure a better readability of resulting colors on the map. Nevertheless, our issue is different: at the contrary of having various distinguishable steps in a color scheme, we aim at designing a regular smooth transition between colors. Therefore, we need to interpolate between colors. The required perceptual regularity between interpolated colors is ensured by the choice of the color space and the choice of the interpolation method. Fig. 5 compares color ranges coming from our interpolation block, but from different colors spaces: RGB is not uniform and doesn’t not help to handle regular and smooth transitions between colors, whereas CIELab as a uniform space allows to handle more smooth and proper perceptual color ranges. We select to systematically interpolate each coordinate of the interpolated colors, with the help of Eq. (2) in the perceptually uniform CIELab color space. The color interpolation block can be used on various data (vector or raster) to generate various types of interpolation. 5.1.1. Color interpolation for plain color vector data symbolization As an example, our color interpolation block can be used to interpolate the colors between two maps designed with similar legend specifications using different colors. Every color layer is interpolated independently between the matched geographical themes of the continuum ends. Fig. 6 introduces an example of such a continuum handling realism level: the left end of the continuum is symbolized by a classical topographic style conveying abstract and conventional colors, whereas the right end of the continuum is symbolized by a set of natural colors extracted from an orthoimage with the method described in Section 4.1. This figure highlights the potential of interpolating between conventional and natural colors, for all geographical layers: smooth color transitions are insured all along the continuum. 5.1.2. Color interpolation for raster data Several raster data types store color information: ortho-images store natural colors seen from the sky, false color images store mixed visible and infra-red colors, relief shaded raster files store simulated sun illumination colors, etc. Our color interpolation block can thus be applied to interpolate the colors between two raster data if they convey the same kind of color information. For example, our color interpolation can allow to interpolate between two different shaded reliefs or two ortho-images of the same area. Moreover, grayscaled orthoimages convey information about the natural texture of the ground surface. Our color interpolation block can be used to manipulate independently the natural color coming from ortho-images without modifying the texture, by interpolating between an ortho-image and its gray level conversion, as illustrated in Fig. 7. 5.2. Texture interpolation blocks Several graphic parameters allow to manipulate the texture and to handle the rendering of a natural, mixed or procedural textures. Therefore, we provide different blocks adapted to each of the related graphic parameters (opacity, amount). Interpolating the opacity of vector data overlaid to an ortho-imagery background allows to control the natural texture integrated in the symbolization. Interpolating the amount of procedural texture allows to control the visibility of such a texture. Handling both interpolation blocks allows to design mixed textures and to control the level of realism of resulting symbolizations. The following paragraphs pre-


31

Fig. 4. In-between cartographic representations. Top-left: vector data symbolized with natural colors. Top-right: vector data footprints symbolized with imagery patches conveying natural textures in color. Bottom-left: vector data symbolized with natural colors overlaid to a grayscaled imagery background conveying natural textures. Bottomright: vector data symbolized with natural colors overlaid to an imagery background conveying natural textures in color. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

ate intermediate transparent styles and to handle the ortho-photorealism coming from the ortho-imagery background, as illustrated for the sea in Fig. 8.

Fig. 5. Comparison of the color space effect on our color linear interpolation method. Top row: RGB color space provides irregular steps in the interpolation. Bottom row: CIELab color space provides a better regularity between colors all along the interpolation. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

sent two types of texture interpolation blocks allowing to create graphic paths based on natural and procedural textures. 5.2.1. Natural texture interpolation block Controlling the opacity of overlaid vector data onto an orthoimage background allows to define how the natural texture coming from the ortho-image is visible in the representation. This graphic parameter is defined between 0 for an opaque symbolization and 100 for a totally transparent symbolization. Therefore, our natural texture interpolation block provides a linear interpolation between two different levels of opacity. Such an interpolation allows to cre-

5.2.2. Procedural texture interpolation block Our procedural texture interpolation block is adapted to the Perlin noise texture. This procedural texture is defined by a set of parameters defining the rendering process of the texture: the scale, the stretch, the amount, the angle, the colors, etc. The amount of the texture describes the amplitude of the Perlin noise generating the texture, impacting the intensity of texture. This parameter is defined between 0 for a non-visible texture to 1 for a maximal amplitude texture. Therefore, it can be interpolated through our procedural texture interpolation block to create graphic transitions between plain color and textured symbolizations (Cf. Fig. 9).

6. Global design method to control color and texture interpolation We face now the issue of how to combine the color and texture interpolation blocks, for each geographical themes of data, in order to provide, at least one cartographic continuum, but potentially several possible cartographic continua.

32


Fig. 6. Color interpolation between an abstract colors topographic map and a natural colors topographic map. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 7. Handling natural colors of an orthoimagery without modifying its natural textures – color interpolation between a colored orthoimage and it grayscaled conversion. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 8. Natural texture interpolation block on the sea, by orthoimagery alpha blending.

6.1. Various steps of the design method The first step, selection of interpolation types for a geographical layer, consists in selecting which type(s) of interpolation will be used (color and/or texture) for one geographical layer. Actually, there is no correlation between the way color and texture may be interpolated between buildings and vegetation, for instance.

The second step, selection of crossing points, is related to the selection of particular crossing points between continuum ends, in order to next drive the behavior of interpolation blocks (step 3). This step is related to the way a cartographer wants the cartographic space to be browsed through the designed continuum. For instance, Fig. 10 shows an example of a graphic crossing point, i.e. an intermediate symbolization, for the representation of the sea


33

Fig. 9. Procedural texture interpolation block on the vegetation, by increasing the amount of texture.

between its representation in an ortho-image (represented by an image patch, and below two squares representing respectively, a presence of texture and the color of the texture) and in a topographic map (represented by an extraction of the sea layer, and below two squares representing respectively, an absence of texture and the plain color used in the map). Illustrated intermediate symbolization combines two visual properties coming from both ends, the natural color from the ortho-image (represented by the related color square) and the absence of texture coming from the map (represented by the related texture square). The third step, specification of the behavior of the interpolation blocks, consists in specifying how interpolation blocks will be used, according to related type of data, users’ choices, etc.: in the same time all along the continuum (color and texture are interpolated from one end to the other), separately or ordered differently (a color interpolation followed by a texture interpolation, or, a texture interpolation followed by a color interpolation) and finally how they are managed during the all continuum (an interpolation block only used on a part of the continuum or all along the continuum). For instance, Fig. 11 shows two possible graphic paths between two representations of the vegetation, on the left in a map and on the right in an ortho-image. For each possible graphic path, two interpolation blocks are used successively, but ordered in a different way between the upper one (step a: same color than the map and texture interpolating; step b: same texture, color interpolating to the one of the ortho-image) and the lower one (step a0 : no texture as the map and color interpolating; step b0 : same color, texture interpolating from non to the one of the ortho-image). Both resulting graphic paths are very different because the difference of the behavior of interpolation blocks go through different representations of the vegetation, while merging differently visual properties of both ends. As a consequence, different strategies can be run by users in order to design one cartographic continuum: users may focus on the selection of the crossing points and on the control of the behav-

ior of interpolation blocks, in order to make various graphic paths, for one geographic layer between both ends of the continuum. We highlight here that this global design method could be used for a continuum between a map or an ortho-image, in both directions, as well as between two maps. The only requirement is to know the structure of represented geographical layer, i.e. the legend specification for a map, and to be able to match data content between the ends of the continuum. 6.2. Results: a cartographic continuum between IGN map and orthoimagery This case study is based on an IGN topographic map, based on the cartographic vector database of the IGN ÓSCAN Express map, and its related ortho-image, coming from the IGN ÓBD ORTHO imagery database, on the territory of Saint Jean de Luz, France. We aim at designing a cartographic continuum between the map and the ortho-image: in order to be able to manage geographical layer independently, we manipulate a set of symbolized vector data. We apply the global design method explained in the previous paragraph, to each geographical layer of the set of vector data, in particular roads, buildings, hydrography, vegetation and background. Each type of geographical layer will be handled differently because of the type of geographical objects (spatial distribution and structure), and also because of the existing cartographic constraints on legibility (for instance, roads have to be always visible whatever the cartographic background is, or, color contrasts have to be preserved) and conventional uses of colors (for instance, hydrography have to be preserved as blue). In the following paragraphs, we detail the design method for each of these geographical layers and refer to Fig. 12 representing the different interpolation blocks and the different crossing points, i.e. intermediate representations, selected by the authors to create the cartographic continuum presented in Fig. 15.

Fig. 10. Selection of a possible intermediate symbolization of the sea between its representations in an ortho-image and in a map. The squares below each representation of the sea represent the presence/absence of texture and the color of the representation. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

34


Fig. 11. Two possible behavior of the color and texture interpolation blocks, to manage two graphic paths for the vegetation. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

roads. Each of these three blocks interpolates the color of one type of road, between the bright color of the map and the natural gray extracted from the imagery. 6.2.2. Buildings’ symbolization interpolation The cartographic proposition for the buildings is to make a smooth transition between the three types of buildings in the map and the main color and texture of the buildings in the image, i.e. roof color and texture, but in two various different steps. We propose to manage first the colors until a crossing point, and second the texture until the ortho-image. The crossing-point is the buildings represented by the natural color of the roofs and no texture. 6.2.3. Hydrography symbolization interpolation The cartographic proposition for the hydrography is to make a smooth transition between the abstract color in the map and the natural color and texture of the hydrography in the ortho-image. We propose to manage first a color interpolation until a crossing point, and then to interpolate texture by opacity: the crossing point is the hydrography represented by the natural color of hydrography in the ortho-image. A color interpolation block, then a texture interpolation block are thus manipulated independently.

Fig. 12. Cartographic continuum parameterization between IGN map and orthoimagery for the continuum of Fig. 15.

6.2.1. Roads’ symbolization interpolation The cartographic requirement for the roads representation is to preserve the administrative categorization coming from the map: the three different categories of roads, ordered by colors, have to be visible and contrasted enough with the rest of data. Three color interpolation blocks are used to interpolate the three categories of

6.2.4. Vegetation symbolization interpolation The cartographic proposition is to manage this particular complex geographical object more subtly, based on its characteristic natural texture. We manage several combinations of natural and procedural textures, with proper colors and alpha blending, in order to smoothly transition between the plain color of vegetation in the map and the vegetation in the ortho-image. A user study has been conducted in order to validate the type of textures and the scheduling of these textures to use, in order to manage the level of orthophoto-realism all along the vegetation continuum: various color and texture interpolation blocks are thus used successively (see Hoarau, 2015). 6.2.5. Background layer interpolation The cartographic background is very clear in the topographic map and should be interpolated to the ortho-image. We propose to use two successive interpolation blocks: first, the white topographic background is interpolated with a grayscaled orthoimagery, second this grayscaled orthoimagery is interpolated with the natural color of the ortho-image. The crossing point is here a


35

Fig. 13. On-demand hybrid 3D map designed with our continuum rendering method.

Fig. 14. Cartographic continuum aiming at supporting interactive focus + context interactions.

grayscaled conversion of the ortho-image, as presented in Section 5.1.

7. Discussion We aimed at proposing a relevant design method to control the level of photo-realism in a cartographic continuum between two various types of representation of a geographical space, i.e. vector data map and ortho-imagery. Our proposition is based on several interpolation blocks, controlling the interpolations between colors and textures for one geographical theme, that are driven by a global design method requiring to specify their behavior for each geographical layers of the input data. Our proposition leads us to make various cartographic continua: one continuum has been explicitly detailed in this paper. We emphasized that various user strategies on the use of color and texture interpolation blocks (together or successively, from which end to which crossing point, etc.) may lead to various continua. This flexibility would allow users to design their own continua, or designers to set-up suitable continua for targeted users. Nevertheless, our proposition raises several issues.

As the behavior of interpolation blocks is specified for each geographical layer independently, the graphic path are cartographically controlled for each geographical layer, but not qualified between all geographical layers in the data set. The choices about the behavior of blocks have been made in order to implicitly preserve the legibility of objects all along the continuum, especially the quality of color contrasts. It would be relevant now to consider and specify proper legibility constraints between all graphic paths to be preserved, all along the continuum. Those legibility constraints, mainly based on graphic semiotics, are actually known when one continuum end is a topographic map: the legend specification integrates all relevant cartographic constraints to be preserved in the topographic map, for instance the categorization and the hierarchy in three categories of roads represented by a relevant color scheme conveying association and order (Bertin, 1983; Brewer, 1994; MacEachren, 1995; Brewer et al., 2003; Christophe, 2011 amongst others). The formalization of relevant cartographic constraints, as well as the specification of constrained interpolations, are current complex research problems in map design. In parallel, even if cartographic continua have been validated by cartographers to properly and efficiently browse the cartographic

36


Fig. 15. Cartographic continuum between a topographic map and an orthoimage based on the method design of Fig. 12.


space between a set of vector data and its related ortho-image, it would be relevant to make various users experiment and qualify the design method. The first purpose would be to evaluate its effective potentiality of exploration of the cartographic space according to cartographic constraints and users’ needs. The second purpose would be to validate the control of the level of photo-realism and the ability to design a suitable intermediate representation according to users’ needs and preferences. Now we have this global method suitable for vector data and ortho-imagery, it would be interesting to experiment the genericity of the method, on other input data, scales, styles and purposes. Another aspect is how to help users to manipulate the design method and the specific interpolation blocks. A map design developer expert would be able to manipulate all components of the framework. But it is relevant to propose to users pre-made crossing points to select or by default crossing points, in order to assist them in the design. Moreover, a global optimization system would allow a better upper control of the design. 8. Conclusion This paper addresses the problem of the interpolation between two representations of a geographical space making the control of the level of photo-realism possible. The approach is based on natural appearance extraction, color and texture interpolation, and photo-realism perception. We propose a method to create a set of hybrid visualizations by mixing continuously heterogeneous representations. We assume that transferring graphic characteristics of ortho-imagery in map design enables cartographers to create better hybrid visualizations. Our work contributes to improve image-based representation design by using imagery as a graphic inspiration source in map design. One of the intermediate representations of the continuum between a topographic map and an ortho-imagery in Fig. 15 has been printed in 3D (Cf. Fig. 13). This hybrid 3D map has been presented to 74 users in order to evaluate its attractiveness and proposed as a potential on-demand cartographic product. Fortythree percents of the users declared that they would enjoy buying such an hybrid 3D map. To go further in the exploration of the expressiveness provided by our interpolation blocks, we would like to apply our methods to other types of heterogeneous data: for instance, an interpolation between an old map an its contemporary one would be interesting to highlight some design choices. The interpolation between two ortho-images, taken at two moments of time would be interesting also in order to make continuous temporal series for multiplexing interfaces. Our method should also be tested at other cartographic scales. Another research perspective issue is the improvement of color and texture interpolation blocks to handle non-linear interpolation and to include constraints to make graphic parameters evolve while taking into account visual or semantical relationships between geographical themes. Finally, style continua created using our interpolation method are conceived as support for the navigation between heterogeneous visualizations in order to allow the users to browse several realism levels and representation styles (Cf. Fig. 14). Therefore, our style continua should improve focus + context interactive tools such as visualization lenses (Karnik et al., 2009; Pindat et al., 2012), by providing locally parameterised transitions. Acknowledgements This work is partially supported by the French National Research Agency, MapStyle project [ANR-12-CORD-0025].

37

References Albertz, J., Lehmann, H., September, 2005. Kartographie und Fernerkundung – Stationen einer Entwicklungüber acht Jahrzehnte. Begleitheft zur Ausstellung der Kommission Kartographie und Fernerkundung – Deutsche Gesellschaft für Kartographie e.V., 21. – bis 23, September, 2005. Bektas, K., Çöltekin, A., 2012. Area of interest based interaction and geovisualization with WebGL. In: Proc. of The Graphical Web Conference 2012, Zrich, Switzerland, September 11–14. Bektas, K., Çöltekin, A., Krüger, J., Duchowski, A., 2015. Combined visual perception models for geographic gaze contingent displays. In: Proc. of the 17th Annual Eurographics Conference on Visualization (EuroVis), Cagliari, Sardinia, Italy, May 25–29. Bernabé-Poveda, M.-A., Çöltekin, A., 2014. Prevalence of the terrain reversal effect in satellite imagery. Int. J. Digital Earth, 1–16. Bertin, J., 1983. Semiology of Graphics: Diagrams, Networks, Maps. University of Wisconsin Press, Madison. Bianchin, A., 2007. Theoretical cartography issues in the face of new representations. In: Proc. of 23th International Cartographic Conference, Moscow, Russia, August 4–7, 2007, Moscow, Russia. Bildirici, O., Kaya, S., Musaoglu, N., Ulugtekin, N., 1999. Satellite images as a cartographic data source for mapping purposes. In: Proc. of International Symposium on Remote Sensing and Integrated Technologies, Istanbul, Turkey, October 20–22. Biljecki, F., Ledoux, H., Stoter, J., Zhao, J., 2014. Formalisation of the level of detail in 3d city modelling. Comput., Environ. Urban Syst. 48, 1–15. Boér, A., Çöltekin, A., Clarke, K.C., 2013. An evaluation of web-based geovisualizations for different levels of abstraction and realism what do users predict? In: 26th International Cartographic Conference, Dresden, Germany, August 25–31. Brasebin, M., Buard, E., Christophe, S., Pelloie, F., 2015. A knowledge base to classify and mix 3d rendering styles. In: Proc. of 27th International Cartographic Conference (ICC’15), Rio de Janeiro, Brazil, August 23–28. Brewer, C.A., 1994. Color use guidelines for mapping and visualization. In: Mac Eachren, A.M., Taylor, D.R.F. (Eds.), Visualization in Modern Cartography. Elsevier Science, Tarrytown, NY, pp. 123–147. Brewer, C.A., Hatchard, G.W., Harrower, M.A., 2003. Colorbrewer in print: a catalog of color schemes for maps. Cartogr. Geogr. Inform. Sci. 30 (1), 5–32. Brychtova, A., Cöltekin, A., 2015. Discriminating classes of sequential and qualitative colour schemes. Int. J. Cartogr. 1 (1), 62–78. Christophe, S., 2011. Creative colours specification based on knowledge (COLorLEGend system). Cartogr. J. 48 (2), 138–145. Christophe, S., Duménieu, B., Turbet, J., Hoarau, C., Mellado, N., Ory, J., Loi, H., Masse, A., Arbelot, B., Vergne, R., Brédif, M., Hurtut, T., Thollot, J., Vanderhaeghe, D., 2016. Map style formalization: rendering techniques extension for cartography. In: Bénard, P., Winnemöller, H. (Eds.), Proc. of Expressive 2016 The Joint Symposium on Computational Aesthetics and Sketch-Based Interfaces and Modeling and Non-Photorealistic Animation and Rendering, May 6–9, 2016. Non-Photorealistic Animation and Rendering, The Eurographics Association, Lisbonne, Portugal. Christophe, S., Perret, J., Hoarau, C., 2013. Extraction de palettes de couleurs pour l’aide à la conception cartographique. Revue des Sciences et Technologies de l’Information (RSTI), Série Technique et Science Informatiques (TSI), Art et Informatique 32 (3–4), 145–152. Cöltekin, A., Lokka, I.-E., Boér, A., 2015. The utilization of publicly available map types by non-experts a choice experiment. In: 27th International Cartographic Conference, August 23–28. Donnay, J.-P., 2000. Nature et Qualités de la Spatiocartographie, La Spatiocartographie, Bulletin de la Société Géographique de Liège, vol. 38, pp. 11–23. Hoarau, C., 2012. Orthoimage or map visualization in use in geoportals, case study on the French Geoportal. In: Proc. of AutoCarto2012 Conference, Columbus, Ohio, USA, September 16–18. Hoarau, C., 2015. Représentations cartographiques intermédiaires – comment covisualiser une carte et une orthophotographie pour naviguer entre abstraction et réalisme? PhD Thesis Université Paris Est. Hoarau, C., Christophe, S., Mustière, S., 2013. Mixing, blending, merging or scrambling topographic maps and orthoimagery in geovisualizations? In: Proc. of the 26th International Cartographic Conference (ICC’13), 25–30 August, Dresden, Germany. Jenny, B., Buddeberg, J., Hoarau, C., Liem, J., 2015. Plan oblique relief for web maps. Cartogr. Geogr. Inform. Sci. 42 (5), 1–9. Jenny, B., Gilgen, J., Geisthövel, R., Marston, B.E., Hurni, L., 2014. Design principles for swiss-style rock drawing. Cartogr. J. 51 (4), 360–371. Jenny, H., Jenny, B., 2012. Recycling cartographic masterworks – challenges in adapting example-based texture synthesis for panoramic map creation. In: Proc. of AutoCarto2012 Conference, Columbus, Ohio, USA, September 16–18. Karnik, P., Cline, D., Jeschke, S., Razdan, A., Wonka, P., 2009. Route visualization using detail lenses. IEEE Trans. Visual. Comput. Graph. 16 (2), 235–247. Lobo, M.-J., Pietriga, E., Appert, C., 2015. An evaluation of interactive map comparison techniques. In: Proc. of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI ’15. ACM, New York, NY, USA, pp. 3573–3582.

38


Loi, H., Hurtut, T., Vergne, R., Thollot, J., 2013. Discrete texture design using a programmable approach. In: Proc. of ACM SIGGRAPH 2013 Talks, SIGGRAPH ’13. ACM, New York, NY, USA (pp. 43:1–43:1). MacEachren, A., 1995. How Maps Work: Representation, Visualization, and Design. Guilford Publications, New York. Masse, A., Christophe, S., 2015. Geovisualization of coastal areas from heterogeneous spatio-temporal data. ISPRS Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. XL-3/W3. Medley, S., Haddad, H., 2011. The realism continuum, representation and perception. Int. J. Image 1 (2), 145–156. Murphy, C., 2015. Intellectual highlighting of remote sensing imagery for better image map design. In: Proc. of 27th International Cartographic Conference, Rio de Janeiro, Brazil, August 23–28. Patterson, T., 2002. Getting real: reflecting on the new look of national park service maps. Cartogr. Perspect. (43), 43–56 Patterson, T., Kelso, N.V., 2004. Hal Shelton revisited: designing and producing natural-color maps with satellite land cover data. Cartogr. Perspect. (47), 28–55 Perlin, K., 1985. An image synthesizer. SIGGRAPH Comput. Graph. 19 (3), 287–296. Pindat, C., Pietriga, E., Chapuis, O., Puech, C., 2012. JellyLens: content-aware adaptive lenses. In: Proc. of the 25th Annual ACM Symposium on User Interface Software and Technology, Cambridge, Massachusetts, USA, UIST ’12. ACM, New York, NY, USA, pp. 261–270. Porter, T., Duff, T., 1984. Compositing digital images. In: Proc. of ACM Siggraph Computer Graphics. ACM, pp. 253–259 (18 (3)). Raposo, P., Brewer, C.A., 2011. Comparison of topographic map designs for overlay on orthoimage backgrounds. In: Proc. of the 25th International Cartographic Conference, Paris, France, May 4–8.

Raposo, P., Brewer, C.A., 2013. Guidelines for consistently readable topographic vectors and labels with toggling backgrounds. In: Proc. of the 26th International Cartographic Conference, Dresden, Germany, August 25–30. Semmo, A., Döllner, J., 2014. Image filtering for interactive level-of-abstraction visualization of 3d scenes. In: Proc. of International Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging (CAe), Anaheim, California, USA, pp. 5–14. Semmo, A., Kyprianidis, J.E., Trapp, M., Döllner, J., 2013. Real-time rendering of water surfaces with cartography-oriented design. In: Proc. of International Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging (CAe), Anaheim, California, USA, pp. 5–14. Semmo, A., Trapp, M., Kyprianidis, J.E., Döllner, J., 2012. Interactive visualization of generalized virtual 3d city models using level-of-abstraction transitions. In: Computer Graphics Forum, Proc. EuroVis 2012, Vienna, Austria, June 5–8 2012, vol. 31, pp. 3. Trapp, M., Beesk, C., Pasewaldt, S., Döllner, J., 2011. Interactive rendering techniques for highlighting in 3d geovirtual environments. In: Kolbe, T.H., Knig, G., Nagel, C. (Eds.), Advances in 3D Geo-Information Sciences, Lecture Notes in Geoinformation and Cartography. Springer, Berlin Heidelberg, pp. 197–210. Wilkening, J., Fabrikant, S.I., 2011. How do decision time and realism affect mapbased decision making? In: Proc. of the 10th international conference on Spatial Information Theory. COSIT’11. Springer-Verlag, Berlin, Heidelberg. Yu, Q., Neyret, F., Bruneton, E., Holzschuch, N., 2011. Lagrangian texture advection: preserving both spectrum and velocity field. IEEE Trans. Visual. Comput. Graph. 17 (11), 1612–1623.