Evaluating Cartogram Effectiveness

Evaluating Cartogram Effectiveness Sabrina Nusrat

Md. Jawaherul Alam

Stephen G. Kobourov

arXiv:1504.02218v1 [cs.HC] 9 Apr 2015

Department of Computer Science, University of Arizona

(a)

(b)

Fig. 1. Red-blue (Republican-Democrat) map of the USA showing 2004 election results from the New York Times [3]: (a) geographically accurate map, (b) a population cartogram. Abstract— Cartograms are maps in which areas of geographic regions (countries, states) appear in proportion to some variable of interest (population, income). Cartograms are popular visualizations for geo-referenced data that have been used for over a century and that make it possible to gain insight into patterns and trends in the world around us. Despite the popularity of cartograms and the large number of cartogram types, there are few studies evaluating the effectiveness of cartograms in conveying information. Based on a recent task taxonomy for cartograms, we evaluate four major different types of cartograms: contiguous, non-contiguous, rectangular, and Dorling cartograms. Specifically, we evaluate the effectiveness of these cartograms by quantitative performance analysis, as well as by subjective preferences. We analyze the results of our study in the context of some prevailing assumptions in the literature of cartography and cognitive science. Finally, we make recommendations for the use of different types of cartograms for different tasks and settings.

1

I NTRODUCTION

Cartograms are maps in which areas of geographic regions, such as countries and states, appear in proportion to some variable of interest, such as population or income. They are popular visualizations for geo-referenced data (there are more than 412,000 Google hits for “cartogram”) that have been used for over a century [56]. As such visualizations make it possible to gain insight into patterns and trends in the world around us, they have gained a great deal of attention from researchers in cartography, geography, and GIS. Many different types of cartograms have been proposed and implemented and they optimize different aspects, such as cartographic errors, preserving the general outline of the original geographic shapes, and maintaining the correct adjacencies between countries. Motivation: Cartograms provide a compact and visually appealing way to present the world’s political, social and economic realities. Red-and-blue population cartograms of the United States have become an accepted standard for representing presidential election results. For example, in the 2004 election, geographically accurate maps seemed to show an overwhelming victory for George W. Bush; while the population cartograms effectively communicated the near even split; see Fig. 1. Likely due to aesthetic appeal and the possibility to visualize data and put political and socioeconomic reality into perspective, cartograms are widely used in newspapers, magazines, textbooks, blogs,

Manuscript received 31 Mar. 2014; accepted 1 Aug. 2014; date of publication xx xxx 2014; date of current version xx xxx 2014. For information on obtaining reprints of this article, please send e-mail to: [email protected].

and presentations. For example, the New York Times [4] shows the election results of 2006 using some nice interactive maps and cartograms. The Los Angeles Times [10] follows the trend by showing 2012 election results using cartograms. Cartograms are also used to show the European Union election results of 2009 in the Dutch daily newspaper NRC [7]. In addition to visualizing election outcomes, cartograms are frequently used to represent other kinds of geo-referenced data. Dorling cartograms are used in the UK Guardian newspaper [9] to visualize social structure and in the New York Times to show the distribution of medals in the 2008 summer Olympic games [5]. Popular TED talks use cartograms to show how the news media makes us perceive the world [41], to expose myths about the developing world [48], and to visualize the complex risk factors of deadly diseases [49]. Cartograms continue to be used in textbooks, for example, to teach middle-school and high-school students about global demographics and human development [45]. Despite the popularity of cartograms and the large number of cartogram variants, there are very few studies evaluating cartograms. In order to design effective visualizations we need to compare cartograms generated by different methods on a variety of suitable tasks. Our Contribution: In this paper we take a task-based approach for a quantitative evaluation of cartograms. We assess four main types of cartograms (in terms of completion time and errors) with a controlled experiment that covers seven different tasks taken from a task taxonomy of cartograms [43]. We also collect and analyze the subjective preferences of the participants in the study. We analyze the results of our study in the context of some prevailing assumptions in the literature of cartography and cognitive science. Finally, we make recommendations for the use of different types of cartograms for different tasks and settings.

2

R ELATED W ORK

Cartograms have a fairly long history. There has been a good amount of work in the quantitative evaluation of cartograms. However, little work is done in the qualitative and systematic evaluation of the relative effectiveness of cartograms. Dent [22] was one of the first to test the effectiveness of cartograms and wrote that “attitudes point out that these (value-by-area) cartograms are thought to be confusing and difficult to read; at the same time they appear interesting, generalized, innovative, unusual, and having – as opposed to lacking – style”. Dent also suggested some techniques for effective communication of cartograms if the audience is not familiar with geographical shapes of statistical units, like providing an inset map and labeling the statistical units on the cartogram. Following Dent, Griffin [28] studied the task of identifying locations in cartograms and found that cartograms are effective. Olson [44] designed methods for the construction of noncontiguous cartograms and studied their characteristics. Krauss [38] studied non-contiguous cartograms as a means of communicating geographic information. She chose three distinct evaluation tasks from the range of very general to specific in order to find out how well the geographic information is communicated by cartograms, and found out that non-contiguous cartograms worked well for showing general distribution of information, but did not work well for showing specific information (ratios between two states). Koletsis et al. [37] reported on early findings to identify possible approaches for evaluating the usability of different types of maps (e.g., Nautical maps, Pedestrian maps, Statistical maps etc.). The aspects of map usability evaluation include: think aloud protocols, questionnaires, focus groups, participant feedback/formal and informal interviews, completion of map reading tasks, use of real and simulated environments, and statistical analyses for interpretation of results. While studying effective way to display geo-referenced statistical data, Pickle [46] devised a set of recommendations for statistical maps: the map should be designed for a particular audience and purpose, a standard legend should be used, colors should be chosen for the visually impaired and consistent with color conventions, unreliable rates should be identified, and researchers should be aware that multiple maps are often needed to address different questions. Although Pickle considered choropleth maps and other thematic maps, most of these recommendations (with some extensions and modification) apply to other statistical maps, including cartograms. While there has been some work on quantitative evaluation of cartogram generation algorithms [12], there is very little work on qualitative comparison between different types of cartograms. Sun and Li [52] analyzed the effectiveness of different types of cartograms by collecting subjective preferences. Two types of experimental tests were conducted: (1) comparison of cartograms with thematic maps (choropleth maps, proportional symbol maps and dot maps), and (2) comparison between cartograms (non-contiguous cartogram, diffusion cartogram, rubber sheet cartogram, Dorling cartogram, and pseudocartogram). The participants in this study were asked to select one map that is more effective for the representation of the given dataset and to provide reasons for this choice. The results indicate that cartograms are more effective in the representation of qualitative result (or nominal result, aggregation of data: who won–republican or democrats?), but thematic maps are more effective in the representation quantitative results (ordinal data - the population growth rate of a state). Note that for both experiments the subjects gave their preferences, but were not asked to perform any specific tasks. In a more recent study, Kaspar et al. [34] investigated how people make sense of population data depicted in contiguous (value-byarea) cartograms, compared to choropleth maps, combined with graduated circle maps. The subjects were asked to perform tasks, based on Bertin’s map reading levels (elementary, intermediate and overall) [14]. The overall results showed that choropleth/graduated circle are more effective (as measured by accurate responses) and more efficient (as measured by faster responses) than the cartograms. The results seemed to depend on the complexity of the task (simple tasks are easier to perform in both maps compared to complex tasks), and the shape of the polygon. Note that only one type of cartogram (Gastner-

Newman diffusion or contiguous cartogram) was used in this study. In order to improve cartogram design, Manting Tao [53] conducted an online survey to collect views and suggestions from map users. The majority of the participants found cartograms to be difficult to understand but at least agreed that cartograms are commonly regarded as members of the map “family”. Jennifer Ware [57] evaluated the effectiveness of animation in cartograms with a user-study in which “locate” and “compare” tasks were considered. The results indicate that although the participants preferred animated cartograms, the response time for the tasks was best in static cartograms. All of these studies indicate an interest in cartograms and their effectiveness. While some specific types of cartograms have been evaluated on some specific tasks, a more comprehensive evaluation of different types of cartograms with a varied set of questions is lacking. In this paper we consider both qualitative and quantitative measurements, covering the spectrum of cartogram tasks, using four of the main types of cartograms. There is a great deal of research in visualization and cartography about the impact of length, area, color, hue, and texture on map visualization and understanding. Bertin [14] was one of the first to provide systematic guidelines to test visual encodings. Cleveland and McGill [18] extended Bertin’s work. Their human-subjects experiments established a significant accuracy advantage for position judgments over both length and angle judgments, which in turn proved to be better than area judgments. These test results were used to refine variables of visual encoding. Stevens [51] modeled the mapping between the physical intensity of a stimulus and its perceived intensity as a power law. His experiments showed that subjects perceive length with minimal bias, but underestimate differences in area. This finding is further supported by Cleveland et al. [17]. In a more recent study, Heer and Bostock [29] investigated the accuracy of area judgment between rectangles and circles. They found that they have similar judgment accuracy, and both are worse than length judgments. They also found that when rectangles are drawn with aspect ratios in {2/3, 1, 3/2}, squares have the least judgment accuracy. This supports what earlier results about graphical comparison by bars, squares, circles, and cubes [20]. Teghtsoonian [54] found that there is a stronger correlation between actual area and apparent area for irregular polygons than for circles. Dent [22] provided a nice survey of related work in magnitude estimation and suggested that the shapes of the enumeration units in cartograms should be irregular polygons or squares. From this discussion we can see that there has been some remarkable studies on graphical perception of area. This is very relevant to cartograms as different algorithms generate different types of shapes (circles, rectangles, irregular polygons). However, it is difficult to use these experiments directly to determine that would work best in the cartogram setting, because the datasets used, the tasks that were given, and experimental conditions varied widely from experiment to experiment. 3 C ARTOGRAM T YPES There is a wide variety of algorithms that generate cartograms. There are three major design dimensions along which cartograms may vary: • Minimizing cartographic error: cartographic error represents how well do the modified areas represent the corresponding statistic shown (e.g., population or GDP). • Shape and relative position preservation: how much the modified shapes and locations of the regions (countries and states) resemble the originals. • Topology preservation: how well the topology of the original map is preserved as measured by adjacent regions and the relative locations of the regions. There is no “perfect” cartogram that preserves shapes, preserves the topology, and also has zero cartographic error [12]. Some cartograms preserve shape at the expense of cartographic error, others preserve

(a)

(b)

(c)

(d)

Fig. 2. (a) A contiguous cartogram showing population in the US; (b) a non-contiguous cartogram showing GDP in Germany; (c) a Dorling cartogram showing criminal activity (arson) in Italy; (d) a rectangular cartogram showing GDP in Italy.

topology, still others preserve shapes and relative positions. In the literature, cartograms are broadly categorized in four types [40] (see Fig. 2): contiguous, non-contiguous, Dorling, rectangular. 3.1 Contiguous Cartograms These cartograms deform the regions of a map, so that the desired size/area is obtained, while adjacencies are maintained; see 2(a). They are also called deformation cartograms [12], since the original geographic map is modified (by pulling, pushing, and stretching the boundaries) to change the areas of the countries on the map. Among deformation cartograms the most popular variant is the ones generated by the diffusion-based algorithms of Gastner and Newman [27], which we use in our evaluation. Others of this type include the rubber-map cartograms by Tobler [55], contiguous area cartograms by Dougenik et al. [25], homeomorphic deformation by Welzl et al. [59], CartoDraw by Keim et al. [35], constraint-based continuous area cartograms by House and Kocmoud [31], and medial-axisbased cartograms by Keim et al. [36]. More recent are circular arc cartograms [33]. In deformation cartograms, since the input map is deformed to realize some given weights, the original map is often recognizable, but the shapes of some countries might be distorted. Recent variants for contiguous cartograms allow for some cartographic error in order to better preserve shape and topology [6]. Worldmapper [2] has a rich collection of diffusion-based cartograms. 3.2 Non-Contiguous Cartograms These cartograms begin with the regions of a map, and scale down each region independently, so that the desired size/area is obtained; see 2(b). The result is piece-wise continuous but overall noncontiguous [44]. They satisfy area and shape constraints, but do not preserve the topology of the original map [38]. These cartograms are easy to generate and there is some evidence that loss of the topology of the original map might not cause perceptual problems [36]. Since the size of the final regions depends on their original size and statistic to be shown, some regions may become too small [1]. 3.3 Dorling Cartograms Dorling cartograms represent areas by circles [23, 24]. Data values are realized by size of the circle: the bigger the circle, the larger the data value; see 2(c). However, in order to avoid overlaps, circles might need to be moved (typically as little as possible) away from their original geographic locations. Unlike contiguous and non-contiguous cartograms, Dorling cartograms preserve neither shape nor topology. Recent Dorling cartograms have become very popular on the web, given JavaScript libraries such as D3 [11] and Protovis [8]. These JavaScript libraries make it easy to render and even animate these cartograms. 3.4 Rectangular Cartograms Rectangular cartograms schematize the regions in the map with rectangles; see 2(d). These are “topological cartograms” where the topology of the map (which country is a neighbor of which other country) is represented by the dual graph of the map, and that graph is used to

obtain a schematized representation with rectangles. In rectangular cartograms there is often a trade-off between achieving zero (or small) cartographic error and preserving the map properties (relative position of the regions, adjacencies between them). Rectangular cartograms have been used for more than 80 years [47]. Several more recent methods for computing rectangular cartogram have also been proposed [16, 30, 39, 40]. In our study, we use a stateof-the-art rectangular cartograms algorithm [16]. There are several options for this type of algorithm and we choose the variant where the generated cartogram preserves topology (adjacencies), at the possible expense of some cartographic error. Note that in addition to possible cartographic errors in this particular variant, rectangular cartograms in general have one major problem. To make a map realizable with a rectangular cartogram, it might be necessary to merge two countries into one (which is highly undesirable in practice), or to split one country into two parts [40]. When recombining them this leads to regions that are no longer rectangular. In our study, we used the variant where the regions remain rectangular, at the expense of some countries getting merged with other countries. In particular 5 states in the map of USA, 3 states in Germany and 2 regions in Italy get merged in this algorithm. 4 V ISUALIZATION TASKS IN C ARTOGRAMS Here we describe the visualization tasks used in our study; these are also included in Table 1, where the exact input setting, along with the exact questions given to the participants, are summarized. A recent task taxonomy for cartograms categorized tasks in different dimensions, grouping similar tasks together [43]. We use seven tasks that cover all the categories, and hence are expected to include all the visualization goals for cartogram visualization. Compare: The compare task has been frequently used in taxonomies and evaluations [57, 50, 42, 58]. The task typically asks for similarities or differences between attributes. Detect change: In cartograms the size of a state/country is changed in order to realize the input weights. Since change in size (i.e., whether a region has grown or shrunk) is a central feature, it is crucial that the viewer be able to detect such change. Locate: The task in this context corresponds to searching and finding the position of a state in a cartogram. In some taxonomies this task is denoted as locate and in others as lookup, but these are not necessarily the same [15]. Since cartograms often drastically deform an existing map, even if the viewer is familiar with the underlying maps, finding something in the cartogram might not be a simple lookup. Recognize: One of the goals in generating cartograms is to keep the original map recognizable, while distorting it to realize the given statistic. Therefore, this is an important task in our taxonomy. The aim of this task is to find out if the viewer can recognize the shape of a region from the original map when looking at the cartogram. Find top-k: This is another commonly used task in visualization. Here the goal is to find k entries with the maximum (or minimum) values of a given attribute. This task generalizes tasks, such as “Find extremum” and “Sort”. In our evaluation, we ask the subjects to find out the state with the highest or second highest value of an attribute.

Find adjacency: Some cartograms preserve topology, others do not. In order to understand the map characteristics properly, it is important to identify the neighboring states of a given state. Summarize (Analyze / Compare Distributions and Patterns): Cartograms are most often used to convey the “big picture”. Summarize tasks ask the viewer to find a patterns or trends in the cartogram. 5

E XPERIMENT

We conduct a series of controlled experiments aimed at producing a set of design guidelines for creating effective cartograms. We assess the effectiveness of our visualizations by performance (in terms of accuracy and completion time for visualization tasks) and subject reactions (attitude). 5.1

Hypotheses Formulation

Our hypotheses are informed by prior cartogram evaluations, perception studies, and popular critiques of cartograms. One of the most common criticism on cartograms is associated with shape distortion which makes it hard to realize familiar geographic regions. Woodruff [60], a cartographer, criticizes the popular red-blue diffusion cartograms for a number of reasons, including: “Topology preservation at the expense of shape: even if I know what a county looks like on a normal map, I’m going to have a hard time identifying it here.” Dorling [24] says “A frequent criticism of cartograms is that even cartograms based upon the same variable for the same areas of a country can look very different”. Fotheringham et al. [26] in considering Dorling cartograms, says that a “cartogram can be hard to interpret without additional information to help the user locate towns and cities”. Dent [22] also suggests that an inset map should be provided with cartograms depicting the realities of geographical space. We consider these limitations of cartograms, and take the suggestions of early researchers by adding an undistorted map for the relevant tasks so that participants are more familiar with the geographical locations, and by adding labels when necessary. We also review the related work [54, 21, 20, 19, 29] on area comparison of circles, squares, rectangles, and irregular polygons; see section 2. Based on these observations we formulate the following hypotheses: • H1: For location tasks, contiguous and non-contiguous cartograms are likely to be better than the other cartograms because these two types preserve the relative position of states. Dorling cartograms will likely be better than rectangular cartograms. • H2: For recognition tasks, non-contiguous cartograms are likely better than the rest since they preserve the original shapes. (For recognizing the shape of a state we only test contiguous and noncontiguous cartograms, because rectangular and Dorling cartograms replace the original shapes with rectangles and circles; testing shape recognizability would lead to predictably high errors and time). • H3: For detecting change (whether a state has grown or shrunk in cartogram), and comparison of areas (size comparison, find topk), contiguous cartograms are likely better than the rest. Dorling cartograms should be worse than contiguous cartograms as the judgment of size of circles is difficult [54], and potentially large aspect ratios for rectangular cartograms can make the changes/comparisons difficult to perceive. (We do not use non-contiguous cartograms detecting change because they use a different normalization of the areas than the other cartograms: while the ratio between two states is the same in all four cartograms, determining whether one state has grown or shrunk is very difficult in non-contiguous cartograms as it involves more than just comparing the areas of the original and final states.) • H4: For finding adjacencies, contiguous cartograms and rectangular cartograms are likely better than the rest, because they preserve topology, whereas the non-contiguous and Dorling cartograms seems to be ill-suited for such tasks.

• H5: For summarizing the results and understanding data patterns, Dorling, non-contiguous and contiguous cartograms will work better than rectangular cartograms, as the first three types better preserve the map characteristics (location, shape and topology) [12]. With respect to user preference, we expect that the participants in our study are likely to prefer contiguous and Dorling cartograms, as they are popular in the media. 5.2

Test Environment

We have designed and implemented a simple application software that guided the participants through the experiment, provided task instructions and collected data about time and accuracy. To ensure that participants were fully at ease, they were briefed about the purpose of the study, data, and the techniques used, before the main experiment. We then asked them to complete the training tasks using the software as quickly and accurately as possible. The study was conducted using one computer (with i7 CPU 860 @ 2.80 GHz processor and 24 inch screen with 1600x900 pixel resolution), where the participants interacted with a standard mouse and keyboard to answer the questions. The participants were encouraged to ask as many questions as needed during the training session. In the experiment the participants generally answered multiple choice questions about different visualizations, all of which involved the four types of cartograms under consideration. The number of errors made and the time taken to answer the questions were recorded and analyzed. The participants were also asked to give preferences and subjectively evaluated the cartograms. 5.3

Participants

We recruited 33 participants by sending emails to students in selected classes at the University of Arizona. A brief description of the study was included in the email and potential participants were asked to reply if interested. The students had different academic backgrounds, with the majority from the department of Computer Science. Among the 33 participants, 24 were male, 9 female; 23 were between 18–25 years of age, 10 were between 25–40; 9 listed high school, 12 listed undergrad, 8 listed Masters and 4 listed PhD as their highest education level. Since some of our tasks require the subjects to identify states highlighted with red and blue colors, we checked color blindness of the participants using an ishihara test [32], and every participant passed the test. To be on the safe side, we also used color-blindness safe colors to highlight the states. 5.4

Datasets and Questions

We evaluated four different types of cartograms, using seven types of tasks. In order to guard against potential bias introduced by just one dataset, we used three different maps for our evaluation: USA, Germany, Italy. Similarly, we used eleven different geographical statistics. Specifically, for all six tasks (with the exception of summarize), we used population and GDP in the USA, Germany and Italy from 2010. For summarize tasks, we used population of the USA in 1960 and 2010; GDP of Germany in 2010, crime rates (arson) in Italy, and three election results (2000, 2004, and 2008) in the USA. We used a within-subject experimental design [13]. For each subject, questions were selected from all the cartogram types and all the tasks. To guard against any adversary effect from the order of the questions, we took a random permutation of the questions for each subject. For each of the tasks, the participants worked with all three country maps (US, Germany, Italy). For all the tasks except recognize and detect change, the participants worked with all four cartogram types. For recognize tasks, we used only contiguous and noncontiguous cartograms, since all the state shapes in Dorling and rectangular cartograms are circles and rectangles. For detect change, we omit non-contiguous cartogram since they use a different normalization of the areas than the other cartograms and finding whether one state has grown or shrunk involves more than just comparing the areas of the original and final states. For each task, the questions were

Input

Question

Time (s)

Error %

Locate this state in the cartogram.

25 20 15 10 5 0

Error (%)

An original undistorted map is given and a state is highlighted in red. A cartogram of the map is shown.

Time (s)

Locate

H1

F = 5.4, P = 0.002

Cont

60 50 40 30 20 10 0

Rect NonCont Dor

Cont

Find out which cartogram state corresponds to the state from the original map.

Cont

18 16 14 12 10 8 6 4 2 0

NonCont

Cont

Which state is bigger: blue or red?

Cont

14 12 10 8 6 4 2 0 Cont

Rect NonCont Dor

F = 2.11, P = 0.1 15

Error (%)

Find the state/region with the highest/second highest value of a statistic (e.g., population, GDP)

Time (s)

Find top-k

H3

20

A cartogram of a country is shown.

10 5 0 Cont

NonCont

F = 10.54, P < 0.001 Error (%)

A cartogram is shown with a red state and a blue state highlighted.

Time (s)

Compare

F = 5.25, P = 0.002 10 8 6 4 2 0

Rect NonCont Dor

F = 13.53, P < 0.001 Error (%)

A state from the map of a country, and shapes of three states from the cartogram of that country are shown.

Time (s)

Recognize

H2

F = 1.08, P = 0.3 14 12 10 8 6 4 2 0

F = 47, P < 0.001

45 40 35 30 25 20 15 10 5 0

F = 32.82, P < 0.001

Cont

Rect NonCont Dor

Rect NonCont Dor

Rect NonCont Dor

Compared to the red state in the map, has the blue state in the cartogram grown or shrunk?

Error (%)

A map and a cartogram are shown. A state is highlighted in red on the map and in blue on the cartogram.

Time (s)

Detect Change

F = 2.71, P = 0.07 10 8 6 4 2 0 Cont

Rect

50 40 30 20 10 0

Dor

Cont

Which state is a neighbor of the highlighted state?

Two separate US population cartograms of 1960 and 2010 are shown.

Which part of the country contributes more to GDP? Which one of these was the closest election between the republicans (red) and the democrats (blue)? What can you say about the trend in population growth?

35 30 25 20 15 10 5 0

Cont

F = 1.35, P = 0.26

Cont

Rect NonCont Dor

Dor

50 40 30 20 10 0

Rect NonCont Dor

Rect NonCont Dor

F = 9.6, P < 0.001 Error (%)

The red-blue cartograms show the U.S. Presidential Election results in three different years.

Where is this criminal activity high compared to other areas?

Time (s)

Summarize

H5

Cont A cartogram of Italy shows the number of criminal incidents involving arson. A cartogram shows the GDP (Gross Domestic Product) of Germany.

Rect

F = 23.37, P < 0.001 Error (%)

A cartogram is shown and a state is highlighted in red. A geographically undistorted map is given for reference.

Time (s)

Find Adjacency

H4

F = 2.68, P = 0.1 30 25 20 15 10 5 0

F = 48.54, P < 0.001

70 60 50 40 30 20 10 0 Cont

Rect NonCont Dor

Table 1. Description of the tasks. For each task, the last two columns show average completion time in seconds and error percentage for different cartogram types, along with the F and p values from ANOVA F-tests. The red line segments indicate statistically significant relationships, obtained using the Least Significant Difference Test.

drawn from a pool of questions involving all cartograms used for the task. Therefore, each participant answered 4 cartograms × 3 maps = 12 questions for four of the tasks. Since we evaluated only contiguous and non-contiguous cartograms for shape recognition, this task involved 2 cartograms × 3 maps = 6 questions. Similarly for detect change, there were 3 cartograms × 3 maps = 9 questions. For the summarize task, where the participants compared/analyzed the overall data trend in the map, we used 4 different data sets: crime rate (arson) in Italy, GDP of Germany, population changes (from1960 to 2010) in US, and Presidential election results of the US. These four datasets were used on four different cartograms for each subject. In total, there were 4 tasks × 12 questions + 6 questions + 9 questions + 4 questions = 67 cartogram task-based questions. The order of the tasks, and the cartograms varied for each subsequent user. In addition to the 67 cartogram task, we assessed the preferences and attitudes of participants. To understand user preferences about different aspects of cartograms, we adapted the semantic differential technique. Specifically, we used a rating scale between pairs of words or phrases that are polar opposites. There were five marks between these phrases and the participants selected the mark on a linear scale that best represented their attitudes for a given map and a given aspect. We used three aspects: general attitude about helpfulness and usability of the visualization, appearance, and readability. This approach for cartograms was first used by Dent [22] to evaluate contiguous cartograms. We asked the participants whether they were familiar with the different cartogram types before the experiment, using a Likert scale. After all the tasks were completed, we asked the participants to choose one of the four cartogram types for last five questions. Finally, we asked the participant how likely they are to use each type of cartogram. The goal of this set of questions was to help us detect potential familiarity bias and we were also curious whether initial preferences might change after 67 tasks were performed. AND

DATA A NALYSIS Contiguous Rectangular

Noncontiguous Dorling 18

5

Number of People Selecting

R ESULTS

The last two columns of Table 1 show the quantitative results (in terms of error percentage and completion time) for all the tasks. The critical values for F are F0.05 (3, 128) = 2.68 for all tasks except for recognize and detect change. For these two tasks the critical values are F0.05 (1, 64) = 3.99 and F0.05 (2, 96) = 3.09, respectively. There is strong evidence for rejecting the null hypotheses in several cases. There is strong evidence in support of Hypothesis 1, based on the results of the locate task. Specifically, there is a statistically significant difference between the performance on contiguous and noncontiguous cartograms, compared against Dorling and rectangular cartograms, both in terms of error rate and completion time. There is a statistically significant difference in errors for Dorling compared to rectangular cartograms, although the same is not true for completion time. There is partial evidence in support of Hypothesis 2, based on the results of the recognize tasks. Specifically, there is a statistically significant difference in errors between contiguous and non-contiguous cartograms, although the same is not true for completion time. Note that the difference in errors is very large – about a factor of 4. There is weak evidence in support of Hypothesis 3. We test this hypothesis with three different tasks (compare, find top-k, detect change). For all three tasks the errors were the lowest in the contiguous cartogram setting, but there was statistical significance between contiguous and all the other cartograms only in two of the three tasks (in the third task there was statistically significant difference between contiguous and rectangular cartograms only). This is consistent with the findings of Dent [22]. Even though the time spent was the lowest in the contiguous cartogram setting for two of the three tasks, there was statistical significance only between contiguous and rectangular cartograms. Although previous cognitive studies show that judgment of circle sizes is not very effective, in our study Dorling cartograms performed well. This could be due to the fact that our tasks do not require participants to estimate the size (area) of circles exactly, but rather to compare two or more circles and to find the circle with the largest area.

16

4.5

Average Rating

6

There is partial evidence in support of Hypothesis 4, based on the results of the find adjacency task. Specifically, there is a statistically significant difference between the performance on contiguous and rectangular cartograms compared against Dorling and noncontiguous cartograms, in terms of error rates, although the same is not true for completion time. There is no statistically significant difference in terms of time for this task and any pair of cartograms. Note that even though we provide an undistorted geographical map along with the cartogram, as suggested by Dent [22] and Griffin [28], the error rates for non-contiguous and Dorling cartograms are much larger. There is partial evidence in support of Hypothesis 5, based on the results of the summarize task. Specifically, there is a statistically significant difference between the performance of rectangular cartograms compared against all others, in terms of error rates, although the same is not true for completion time. In general, the results of this part of the study show significant differences in performance (in terms of time and accuracy) between the four types of cartograms. As indicated by our hypotheses, different tasks seem better suited to different types of cartograms. These results are consistent with a previous work on quantitative evaluation of cartograms [12]. Achieving perfection (with respect to minimum cartographic error, shape recognizability and topology preservation) in cartograms is difficult and no cartogram is equally effective in all three dimensions. Rectangular cartograms preserve adjacency relations, and that is reflected in the results. Non-contiguous cartograms maintain perfect shape, making the “recognize” task easy, but the “sparseness” of the map makes it difficult to understand adjacencies. Dorling cartograms disrupt the adjacency relations but somewhat preserve the relative positions of states, and are good at getting the “big picture”. Contiguous cartograms more or less preserve localities, state shapes, and adjacencies, and give the best performance for almost all the tasks. The familiarity with contiguous cartograms might play a role in this regard.

14

4

12

3.5

10

3 2.5 2 1.5 1

8 6 4 2 0

Cartogram Types

(a)

Cartogram Types

(b)

Fig. 3. (a) Subjective rating of different cartograms; (b) the number of participants selecting different cartograms to perform remaining tasks.

6.1

Subjective preferences

We asked the participants several preference and attitude questions in addition to the visualization tasks. At the beginning, the subjects rated all four cartograms using a Likert scale (excellent = 5, good = 4, average = 3, poor = 2, very poor =1); see Fig. 3(a). The results confirm our expectation that contiguous (average rating 3.66) and Dorling (3.84) cartograms are rated higher that rectangular (2.54) and non-contiguous (2.75). After performing the visualization tasks, subjects were asked to choose one of the four cartograms on which they would be asked five more questions. Contiguous and Dorling cartograms continue to be the most preferred cartograms at the end of the visualization tasks as well (Out of 33 participants, 17 chose Contiguous, 15 chose Dorling, 1 chose non-contiguous, and 0 chose rectangular as their preferred cartogram to work with); see Fig. 3(b). In addition to the ease and efficiency in performing tasks with these two cartograms, the preference towards contiguous and Dorling cartograms might partially be due to familiarity with these two cartograms in the news and on social media

Contiguous Helpful Entertaining Interested to use later Well−organized Elegant Innovative Showing magnitude clearly Easy to Understand

Rectangular Hindering Boring Not interested to use later Poorly−organized Drab Conventional Showing magnitude poorly Difficult to understand

Noncontiguous

Dorling Helpful

Entertaining Interested to use later Well−organized Elegant Innovative Showing magnitude clearly Easy to Understand

Hindering Boring Not interested to use later Poorly−organized Drab Conventional Showing magnitude poorly Difficult to understand

Fig. 4. Evaluation of the different cartograms by mode (a) and mean (b): contiguous and Dorling cartograms clearly outperform the others.

(10 participants reported that they are familiar with contiguous cartograms, 15 were familiar with Dorling cartograms, 7 with rectangular cartograms and 2 with non-contiguous cartograms). At the end of the survey the participants were asked to rate the different cartogram types according to different categories such as the helpfulness of the visualization, readability, and appearance. The rank in each scale was constructed by calculating the mode (most frequent response) and the mean; see Fig. 4. This also indicates a clear preferences for contiguous and Dorling cartograms over the rest. Specifically, the participants found contiguous cartograms to be helpful, wellorganized and showing relative magnitude clearly, and Dorling cartograms to be entertaining, elegant, innovative, showing magnitude clearly, and easy to understand. The answers to the question “Will you use this visualization later?” also favor contiguous and Dorling cartograms. 6.2

Implication for Design

Cartograms are good at summarizing data and showing broader trends and patterns, as shown in early research [38, 52] and in this study. Contiguous, non-contiguous, and Dorling cartograms perform well in the tasks involving analyzing and comparing trends, with Dorling cartograms giving the best results. The reason might be that the simple circular shapes convey the data pattern easily, whereas the distortion in shape and size for other cartograms distract the viewers. For tasks, such as locate, find top-k and detect change, contiguous cartograms give the best results. Rectangular cartograms do not yield very good results, although they are excellent at accurately converting the underlying statistic (as they usually have very low cartographic error). But since few of the common cartographic tasks require high precision, rectangular cartograms do not outperform the other types in any of our tests. Overall, there is an overwhelming preference for contiguous and Dorling cartograms over the other two. Thus, for the purpose of cartogram visualizations that involve only looking at the overall patterns, Dorling cartograms seem to be a good choice. On the other hand, in instances where the geographic locations and country adjacencies are important aspects, and the map reading required is more detailed, contiguous cartograms might be more suitable. Non-contiguous cartograms perfectly preserve state shapes and geographic locations, and their performance seems to be good for almost all the tasks, with the clear exception of finding adjacencies. Another drawback is that these cartograms make it difficult to detect change (since nearly all regions shrink) and they were excluded from the detect-change task. These cartograms seem to be less popular, perhaps because they are not as commonly used as the other types. Overall they might be suitable for visualizations where state adjacencies are not important and detecting change is not required. Rectangular cartograms are more or less a clear outlier in both the analysis of quantitative efficiencies and in the qualitative subjective

preference. This suggests that cartograms that are severely distorted (in relative positions) from the original map might be a poor choice for cartogram visualization in practice. A much more promising compromise might be offered by rectilinear cartograms as that in Fig. 1(b). 7 L IMITATIONS Our list of tasks may be limited with respect to cartographic/geographic aspects, as we were mostly focusing on the information visualization aspects of cartogram visualization. We also did not consider tasks for dynamic and interactive cartogram visualization systems. With the addition of good interaction techniques it is possible that both the effectiveness and preferences might change. Note also that we excluded different types of cartograms f when they seemed particularly unsuited for a given task. Similarly, we did not consider careful size estimation questions, which would favor zero-cartographic error type of cartograms. We evaluated these cartograms as representatives of the four main types of cartograms in literature [40]. However, there are many other cartograms (e.g., rectilinear cartograms, square cartograms, etc.) and variants that we did not consider. 8 D ISCUSSION AND C ONCLUSION Cartograms represent geo-referenced data, promote visual thinking, and transform data into stories. Encoding size by area is not as effective as encoding size by length [51]. However, unlike bar graphs (which represent size better), cartograms contain geographic information and adjacency relations. This makes it possible to see broader patterns and trends. These are non-trivial advantages that make it possible to provide better overview and “big-picture” summary of the underlying data. Given the popularity of cartograms in representing geo-spatial data and trends, we believe that cartograms should be studied more carefully by information visualization researchers. While it is unlikely that a single evaluation study will be complete and will cover all possible dimensions, we believe that our work can be a useful guideline for the design and use of cartograms. Strengthening the results of our study can be done by evaluating more data (with more countries), more georeferenced statistics, more extreme distributions, and by using other types of cartograms, including other geo-statistical visualization (e.g., choropleth maps). It is likely worthwhile to also extend the task taxonomy to interactive and dynamic cartograms and compare those to their static counterparts. All the data, questionnaires, and analysis of this study can be found at http://cartogram.cs.arizona.edu. 9 ACKNOWLEDGMENTS We thank Carlos Scheidegger for useful comments about this project. We also thank Sara Fabrikant and her research group for providing a

relevant thesis and valuable comments. R EFERENCES R [1] SHOW / a new way to look at the world. http://show. mappingworlds.com/. [Online; accessed 11/14/2014]. [2] Worldmapper: The world as you’ve never seen it before. http://www. worldmapper.org/. [Online; accessed 3/30/2015]. [3] NY Times Election 2004. http://goo.gl/kL7Asj, 2004. [Online; accessed 12/5/2014]. [4] Election News. http://goo.gl/hcnNSH, 2006. [Online; accessed 11/04/2014]. [5] A Map of Olympic Medals. http://goo.gl/CCT6tl, 2008. [Online; accessed 11/11/2014]. [6] indiemaps.com/blog >> cartogram design. http://goo.gl/ Loq7q7, 2008. [Online; accessed 1/26/2015]. [7] Europe goes to the polls. http://goo.gl/oa8uop, 2009. [Online; accessed 11/04/2014]. [8] Protovis – dorling cartograms. http://goo.gl/pbfQkkF, 2010. [Online; accessed 11/15/2014]. [9] How to visualise social structure — news — The Guardian. http:// goo.gl/2Vrg3K, 2012. [Online; accessed 11/05/2014]. [10] U.S. election results. http://goo.gl/AVA2nd, 2012. [Online; accessed 11/052014]. [11] D3.js - data-driven documents. http://d3js.org/, 2013. [Online; accessed 3/22/2015]. [12] M. J. Alam, S. G. Kobourov, and S. Veeramoni. Quantitative measures for cartogram generation techniques. Computer Graphics Forum, special issue on EuroVis’15, 2015 (to appear). [13] C. Barnum, E. Henderson, A. Hood, and R. Jordan. Index versus full-text search: a usability study of user preference and performance. Technical Communication, 51(2):185–206, 2004. [14] J. Bertin. Semiology of Graphics: Diagrams, Networks, Maps. The University of Wisconsin Press, Madison, 1983. [15] M. Brehmer and T. Munzner. A multi-level typology of abstract visualization tasks. IEEE Transaction on Visualization and Computer Graphics, 19(12):2376–2385, 2013. [16] K. Buchin, B. Speckmann, and S. Verdonschot. Evolution strategies for optimizing rectangular cartograms. In Geographic Information Science (GIS’12), volume 7478, pages 29–42, 2012. [17] W. S. Cleveland, C. S. Harris, and R. McGill. Judgments of circle sizes on statistical maps. Journal of the American Statistical Association, 77(379):541–547, 1982. [18] W. S. Cleveland and R. McGill. Graphical perception: Theory, experimentation, and application to the development of graphical methods. Journal of the American statistical association, 79(387):531–554, 1984. [19] W. S. Cleveland and R. McGill. An experiment in graphical perception. International Journal of Man-Machine Studies, 25(5):491–500, 1986. [20] F. E. Croxton and H. Stein. Graphic comparisons by bars, squares, circles, and cubes. Journal of the American Statistical Association, 27(177):54– 60, 1932. [21] B. Dent. Cartography: Thematic map design. New York: McGraw Hill, 1999. [22] B. D. Dent. Communication aspects of value-by-area cartograms. The American Cartographer, 2(2):154–168, 1975. [23] D. Dorling. The visualization of spatial structure. PhD thesis, Department of Geography, University of Newcastle-upon-Tyne, U.K., 1991. [24] D. Dorling. Area Cartograms: Their Use and Creation, volume 59 of Concepts and Techniques in Modern Geography. University of East Anglia, 1996. [25] J. A. Dougenik, N. R. Chrisman, and D. R. Niemeyer. An algorithm to construct continuous area cartograms. The Professional Geographer, 37(1):75–81, 1985. [26] A. Fotheringham, C. Brunsdon, and M. Charlton. Quantitative Geography. 2000. [27] M. T. Gastner and M. E. J. Newman. Diffusion-based method for producing density-equalizing maps. In The National Academy of Sciences of the United States of America, volume 101, pages 7499–7504, 2004. [28] T. Griffin. Recognition of area units on topological cartograms. The American Cartographer, 10(1):17–29, 1983. [29] J. Heer and M. Bostock. Crowdsourcing graphical perception: using mechanical turk to assess visualization design. In SIGCHI Conference on Human Factors in Computing Systems, pages 203–212. ACM, 2010.

[30] R. Heilmann, D. Keim, C. Panse, and M. Sips. Recmap: Rectangular map approximations. In IEEE Symp. Information Visualization (INFOVIS), pages 33–40, 2004. [31] D. H. House and C. J. Kocmoud. Continuous cartogram construction. In IEEE Visualization, pages 197–204, 1998. [32] S. Ishihara. Tests for color-blindness. 1917. [33] J.-H. K¨amper, S. G. Kobourov, and M. N¨ollenburg. Circular-arc cartograms. In IEEE Pacific Visualization Symposium (PacificVis’13), pages 1–8, 2013. [34] S. Kaspar, S. Fabrikant, and P. Freckmann. Empirical study of cartograms, 2013. [35] D. Keim, S. North, C. Panse, and J. Schneidewind. Visualizing geographic information: Visualpoints vs. cartodraw. Information Visualization, 2(1):58–67, 2003. [36] D. A. Keim, C. Panse, and S. C. North. Medial-axis-based cartograms. IEEE Computer Graphics and Applications, 25(3):60–68, 2005. [37] M. E. Koletsis, W. Cartwright, and N. Chrisman. Identifying approaches to usability evaluation. 2014. [38] M. R. D. Krauss. The relative effectiveness of the noncontiguous cartogram. PhD thesis, Virginia Polytechnic Institute, 1989. [39] M. Kreveld and B. Speckmann. Rectangular cartogram computation with sea regions. In International Cartographic Conference, 2005. [40] M. Kreveld and B. Speckmann. On rectangular cartograms. Computational Geo., 37:175–187, 2007. [41] A. Miller. TED talk: The news about the news. http://goo.gl/ pn4XLB, 2008. [Online; accessed 10/20/2014]. [42] A. M.Maceachren, M. Wachowicz, R. Edsall, and D. Haug. Constructing knowledge from multivariate spatiotemporal data: integrating geographical visualization with knowledge discovery in database methods. Intl. J. of Geo. Info. Science, pages 311–334, 1999. [43] S. Nusrat and S. Kobourov. Task taxonomy for cartograms. EuroVis (short paper), 2015 (to appear). [44] J. Olson. Noncontiguous area cartogram. The Professional Geographer, 28(4):371 – 380, 1976. [45] T. Pelkofer. Using cartograms to learn about Latin America. http: //goo.gl/yB8nQs. [Online; accessed 10/20/2014]. [46] L. W. Pickle. Usability testing of map designs. In Proceedings of Symposium on the Interface of Computing Science and Statistics, pages 42–56, 2003. [47] E. Raisz. The rectangular statistical cartogram. Geographical Review, 24(3):292–296, 1934. [48] H. Rosling. TED Talk: The best stats you’ve ever seen. http://goo. gl/VYGvh9, 2006. [Online; accessed 11/13/2014]. [49] H. Rosling. TED Talk: Insights on HIV, in stunning data visuals. http: //goo.gl/DJ4LYP, 2009. [Online; accessed 10/20/2014]. [50] R. E. Roth. An empirically-derived taxonomy of interaction primitives for interactive cartography and geovisualization. IEEE Transaction on Visualization and Computer Graphics, 19(12):2356–2365, 2013. [51] S. S. Steven. On the psychophysical law. Psychological Review, 64(3), 1957. [52] H. Sun and Z. Li. Effectiveness of cartogram for the representation of spatial data. The Cartographic Journal, 47(1):12 – 21, 2010. [53] M. Tao. Using Cartograms in Disease Mapping. PhD thesis, The University of Sheffield, 2010. [54] M. Teghtsoonian. The judgment of size. The American journal of psychology, pages 392–402, 1965. [55] W. Tobler. A continuous transformation useful for districting. Annals, New York Academy of Sciences, 219:215 – 220, 1973. [56] W. Tobler. Thirty five years of computer cartograms. Annals of Association of American Geographers, 94:58–73, 2004. [57] J. A. Ware. Using animation to improve the communicative aspect of cartograms. PhD thesis, Michigan State University, 1998. [58] S. Wehrend. Appendix B: Taxonomy of visualization goals. In Visual cues: Practical data visualization., pages 203–212. IEEE Computer Society Press, 1993. [59] E. Welzl, H. Edelsbrunner, and R. Waupotitsch. A combinatorial approach to cartograms. Computational Geometry, 7:343–360, 1997. [60] A. Woodruff. I hate your favorite election map. http://goo.gl/ DXbpG6, 2008. [Online; accessed 12/04/2014].