Static vs. Dynamic Time Mapping in Radial ...

Static vs. Dynamic Time Mapping in Radial Composite Indicator Visualization Yael Albo1, Joel Lanir1, Peter Bak2, Sheizaf Rafaeli1 1

University of Haifa, Mt. Carmel, Haifa, 31905, Israel IBM Research Haifa Lab, Mount Carmel, 31905 Haifa, Israel

2

[email protected], [email protected], [email protected], [email protected] ABSTRACT Composite Indicators (CIs), are a common measurement and benchmarking tool that are used to reflect and measure multidimensional concepts such as digital divides, individual’s wellbeing and more. Measurement iterations produce a series of timeoriented data, which stakeholders as well as the general public might be interested to interpret. Visualization of a CI is highly recommended in order to ease interpretation, and many CI websites use radial solutions to visualize CIs. Yet it is unclear how to visualize the temporal dynamics in radial diagrams. Static solutions, mapping time to small multiples might be challenging due to screen space issues. Dynamic solutions are appealing, yet, there is no clear empirical evidence on benefits of dynamic time coding in radial diagrams. In this paper, we compare static vs. dynamic time mapping using two radial CI visualization methods. The popular Radar chart technique is compared to the innovative Flower chart as used in the well-known OECD Better Life index. We compare users' performance and preferences empirically under formal task taxonomy, adjusted to CI tasks. Results indicate that in general, static time encoding was more effective than dynamic encoding. Still, an in depth analysis showed that the dynamic approach is a feasible and sometimes even better solution for important CIs tasks, leveraged by the fact that users seem to like and enjoy it.

Keywords Animation; composite indicators; visualization; small multiples.

evaluation;

information

1. INTRODUCTION A Composite Indicator (CI) is a common measuring and benchmark (M&B) method that measures multi-dimensional concepts that cannot be captured by a single indicator. Subindicators are compiled into a single index in a hierarchical way, as a weighted variable tree. For example, an Information and Communication Technologies (ICT) CI might be composed of measurements of ICT infrastructure, ICT use, and ICT impact. When evaluated at regular intervals, an indicator can point out trends of change across different entities over time. A composite indicator might be useful in setting policy priorities and in benchmarking or monitoring growth and improvement year after year. CIs are increasingly recognized as important tools in policy analysis and public communication, enabling transparency of data and citizens involvement [9]. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. AVI '16, June 07-10, 2016, Bari, Italy © 2016 ACM. ISBN 978-1-4503-4131-8/16/06…$15.00 DOI: http://dx.doi.org/10.1145/2909132.2909250

The number of CIs around the world is growing continuously, reflecting various domains such as education, health and environmental issues [5]. Questions related to CIs might be linked with growth potential as well as gap risks: Does a divide exist? What is the gap's size and trend – is it getting wider or narrower? What is the rate of change? Which indices should be improved, and what is the best practice one should learn from? [24]. Visualizing a composite indicator and its components has been acknowledged as a useful way to examine these questions [27]. An effective visualization might facilitate understanding of current problems, selecting improvement measures and evaluating policy-makers' actions taken. Although interpretation of CIs might be influenced by the visualization method used, there is a lack of guidelines how to visualize CIs [24]. Out of the various ways to visualize CIs, radial visualizations are a common way of representing the multidimensionality of the indicators, as can be seen in many CI websites and reports [12, 13, 28, 31]. Radial diagrams have recently gained increasing popularity in many other application domains as well [7]. In previous work, Albo et al. [2] examined several static radial options for CI visualizations, among them the Radar charts and the Flower charts [28]. Static time mapping in both techniques uses small multiples for showing time. Illustrating the time dimension is an important challenge of CI visualization [27]. The time dimension can be mapped either in a static, or in a dynamic way. Dynamic time mapping, in the form of animation, has become common use in CI visualizations [21, 34, 35], especially as new easy to implement technologies emerge, such as HTML5 and dynamic web programming [18]. Although high cognitive load is associated with interpreting animation [33], people seem to like it [30]. Static approaches to encode the temporal dynamics in CI visualization often use small-multiple views. Small multiples show moments in time as parallel spread or thumbnail type views. Static time mapping requires using small icons and occupying most of the screen-space, but enables direct visual comparison of different time slices [32]. Dynamic time encoding visualizations (animation or slide shows) might solve screen space limitations, enable bigger picture use, enlarge time scalability and improve the "fun" experience [30]. But can dynamic time mapping be a feasible solution for CI visualization? Can it facilitate performance in common CI tasks? Little effort has been devoted for studying radial visualization [11], even more so in the aspect of static vs. dynamic time mapping methods. The aim of the paper is to make a step towards filling the gap of static vs. dynamic Radar and Flower charts by a controlled empirical comparison using a real-world dataset. We use a formal task taxonomy adjusted with the help of domain experts to CI questions. The contribution of this work is first, to widen the growing knowledge of CI visualization design by examining how to

encode the time dimension within the context of radial visualizations. Second and most importantly, we extend existing believes on the differences between static and dynamic approaches to radial solutions within a comprehensive tasks taxonomy. This enables us to devise specific design guidelines for visualization practitioners.

2. RELATED WORK Three elements are in the focus of this paper: composite indicator visualizations, radial visualizations and static vs. dynamic time mapping of time-oriented data.

2.1 Visualizing Composite Indicators Composite Indicators are often visualized with the aim of making the information accessible to various stakeholders as well as to the general public (e.g., [21, 28, 31, 35, 36]). The latter involves the need for an aesthetic and engaging visualization [9]. The OECD handbook on constructing CIs states that CI visualization "should receive proper attention, given that the visualization can influence (or help to enhance) interpretability". Visualization is needed "To identify a coherent set of presentational tools for the target audience; to select the visualization technique which communicates the most information"; and "to present the composite indicator results in a clear and accurate manner" [27].

2.2 Radial visualizations The term radial visualization was coined by Hoffman et al. [20] in the 1990s, but underlying concepts were rooted in statistical graphics of the 19th century when William Playfair and Florence Nightingale became pioneers of radial visualizations by using pie charts for statistical graphics. Draper et al. [11] present a survey of Radial methods for information visualization, calling for more evaluations of radial visualizations methods. Burch and Weiskopf [7] discuss benefits and drawbacks of radial diagrams. Perceptual challenges and cognitive effort are associated with radial visualization. Angle, slope, and area judgments (used in radial diagrams) are less accurate than position and length judgments (used in Cartesian diagrams), as demonstrated by Cleveland and McGill [8]. Diehl et al. [10] characterized Radial layouts through empirical studies, finding Cartesian visualizations tend to outperform their radial counterparts. Goldberg and Hoffman [17] found in an eye tracking evaluation that tasks were completed more quickly on linear graphs than on those with radial layout, all supporting Tufte's guideline: "Graphic should tend toward the horizontal" [32]. On the other hand, radial charts are compact and aesthetic [10]. A radial presentation places visual objects in the field of view and hence, it is in easy reach to the viewer [7]. Fuchs et al. [16] showed that radial encodings of time were more efficient than linear encoding in particular for temporal location tasks. Diehl et al. [10] suggested implementing radial methods when there is a need to focus on a particular data dimension which would then be presented by circle sectors.

2.3 Static and Dynamic Visualizations of Time-Oriented Data Aigner et al. [1] present a systematic classification of techniques for visualizing time and time-oriented data. There are two major possibilities of time property encoding: static and dynamic presentations. Static representations use spatial features such as one of the display dimensions (X or Y-axis, e.g. line plots) or Small Multiples. Since line plots are limited to showing a relatively low amount of dimensions, in our study for static time mapping condition we focus on small multiples. In dynamic

representations, time is mapped to time, usually in the form of a slideshow or animation. Animation use in visualization (as videostyle playback of a multi-frame sequence) is in dispute. On the one hand, the motion channel is extremely salient and separable, and animation strongly draws attention. User experience improves as people find animation enjoyable and exciting [30]. There are studies that support animation as a powerful visualization instrument [19, 26, 29]. On the other hand, there are critical voices against using animation. Consulting our memory to compare a current view with a view that was seen before requires higher cognitive load than using our eyes to switch between different views that are visible simultaneously (“eyes beat memory”) [25]. Another argument against animation is the perceptual effect of change blindness, which can lead to an unnoticed miss of important changes. Tversky et al. [33] conducted a survey of evaluations that compared static and animated representations, arguing that many evaluations, that saw animation beneficial, suffered from several faults. They pointed out that interactivity may be the key to overcoming the drawbacks of animation. Robertson et al. [30] compared animation, trace line and small multiples visualization techniques on multi-dimensional data. Animation was found to be the less effective for analysis, but more effective for presentation. In addition, participants preferred animation stating that it was fun to use. Archambault et al. [4] found small multiples were faster than animation for graph evolution over time, but error rate was higher on some tasks. Kriglstein et al. [22] provide an overview of empirical evaluations and recommendations for the design of visualizations for timedependent data, relating separately to radial layouts and to animation vs. static representation issues. As far as we know, there were no works that compared dynamic and static methods in the context of radial visualization techniques.

3. VISUALIZATION METHODS AND INTERACTIONS We focus on Static vs. Dynamic comparison of two radial visualizations for CIs: the well-known Radar charts and the innovative Flower charts. Table 1 shows a summary of the visual encoding of data properties of both static and dynamic Radar and Flower charts. Visual choices were derived from our aim to code indices to segments. The motivation was the need to have a satisfactory solution for indices scalability, as expected in CIs. Other properties' coding was chosen in correspondence to this initial choice. It is important to emphasize that comparison is not a Radar icon vs. a Flower icon but rather about comparing techniques as a whole setting. We follow [2], and believe that a fair basis for valid comparison was created. Table 1. Summary of visual encoding of the data properties to different properties of the selected charts. Design decisions were made under the constraints of experiment validity. Data property

Radar Chart Static

Time

Dynamic Axes

Indices Small Multiples

Slides / Animation

Flower Chart Static

Dynamic

Segments (color) Small Multiples

Slides / Animation

Items

Data Lines (color)

Small Multiples (rows)

Values

Distance from Center

Distance from Center

3.1 Radar charts Radar charts are commonly used for CI visualization [12, 13, 31]. When using the Radar chart, indices are mapped to the radial

property (axes), while items are usually mapped to lines connecting the same item's data points. Dynamic Radar charts were in use on the former WebIndex visualization, presenting a unique use of a dynamic radial method. While quite popular, the use of Radar charts is in dispute. Critics on Radar visualizations in the context of CIs have emerged lately [2, 14]. Because each item is represented by a polygon, the area of the polygon depends on the placement of axis dimensions, and thus may be inappropriate for comparing items. Furthermore, a Radar chart may turn out to be too cluttered if showing multiple items. Another problem with Radar charts is that we tend to prefer polygons with symmetrical shapes. Yet, symmetry does not relate to the magnitude of ratings. Figure 1-a1 shows our use of Static Radars (SR) by presenting small multiples for each year. Figure 1-a2 shows our use of Radar chart in the dynamic time mapping condition (DR).

3.2 Flower charts The popular OECD Better Life index [28] introduces an innovative radial visualization for a CI. Though this visualization was developed for a concrete purpose, it was recommended as an aesthetically engaging and informative method for multidimensional data [6, 15]. In this radial-based solution, each flower glyph represents an item (i.e., a city). Each petal of the flower corresponds to a different indicator, with every indicator uniquely colored. The length of the petal is mapped to the value of that indicator. Time dimension (which does not exist in the original Betterlife website) is mapped to columns in the Static Flowers (SF) condition, while items are arranged to rows as can be seen in figure 1-b1. In the dynamic condition, Dynamic Flowers (DF) can be reached by direct access per year or by automatic sequential access (meaning animation) as can be seen in figure 1-b2.

3.3 Interactions It is important to describe the interaction techniques developed for each visualization method, in order to improve settings comparison fairness. We developed a "Drag" interaction for the Flowers visualization. The user could move a whole row in the static condition, or a single flower in the dynamic condition, enabling to bring one city presentation closer to another city. Hence, items comparison becomes more comfortable (with the Radar, one icon includes many items that facilitate items comparison). We also developed a "Filter" interaction for the Radar visualization, which enables items visual isolation. The user could filter out Radar items (cities), by pressing on the city label in the legend, leaving only the cities of interest (with Flowers, one icon represents one item only). Tooltip interaction was available in all visualizations, displaying concrete value titled by the related index name when hovering over an item. For the dynamic visualizations, interactions included "play interaction", which enabled using the play button to trigger the animation starting from the first year and ending with the last year, in a rate of two frames per second. Users could pause the animation, or gain direct access to a selected year using the "select time-slice", which brings a specific year to the front, using smooth transitions.

4. METHODOLOGY We performed a user study to examine static and dynamic CI visualizations. We chose to fix the number of items to five, since we saw in existing CIs visualizations that comparisons are interesting in the context of "my" performances vs. mean, best, worst and another selected item practice (Altogether 5 items) [36]. Choosing five indices (or dimensions) relies on a reasonable estimate for the average size of a single hierarchy's level. Number of indices was not found to be an important factor in similar

studies [2]. However, this should be considered as a limitation as discussed in section 6.2 and as an issue for a future study. We used a 2 (Time encoding: Static vs. Dynamic) x 2 (Vis: Radars vs. Flowers) x 10 (Tasks) within-subject design. Dependent measured variables were task completion time and task accuracy.

4.1 Dataset Construction of a new national ICT index provided us a unique opportunity to use a real world dataset. Our data source is taken from real marketing and media surveys conducted during 20032012 asking about ICT use. The data contains information from a representative sample of 30 regions and about 10,000 adults annually. Data was collected and analyzed, and the indices were calculated and displayed for the purposes of measuring, benchmarking and making decisions about digital divide policy at the national and regional level. Data was normalized to 1-100 time-dependent scores for each indicator. We prepared a construct of five indices, representing typical sub-indicators of an ICT index. Four different data blocks were prepared to accommodate the 2 (Time) x 2 (Vis) experiment design. Each data block consisted data of 5 regions (items).

4.2 Tasks We use the formal task model by Andrienko & Andrienko [3] for task analysis, as detailed in Table 2. On the upper level, tasks are divided into elementary and synoptic tasks. Elementary tasks address individual and separated data elements (values or groups of data) and include: lookup, comparison and relation seeking tasks. Synoptic tasks involve a general view (sets of values or groups of data in their entirely), and are divided into descriptive (including: lookup, comparison and relation seeking tasks) and connectional tasks (homogeneous and heterogeneous behavior). We developed 10 relevant CIs tasks, consulting with digital divide stakeholders in adjusting synoptic tasks to relevant domain tasks.

4.3 Participants Twenty-four participants took part in the study (13 female). The average participant age was 27.08 (SD: 4.6). All participants had normal or corrected-to-normal eyesight, and no participants were color blind (self-reported).

4.4 Procedure Experiments were conducted in a quiet room, one participant at a time. Participants were seated in front of a 24" screen using a full HD resolution of 1920x1080. A short description of the study's purpose was given, followed by a personal information questionnaire. Afterwards, participants were given four consecutive session blocks each consisting of a different Vis (SR, DR, SF, DF). A block consisted of a training session of four tutorial tasks followed by the ten actual tasks. Participants were asked to work as quickly and accurately as possible. The visualization was presented in full screen, with the task description written on the bottom of the screen. After the participant chose an answer, a next button was pressed to prompt the next task. Completion time and result for each task were recorded, as well as all user interactions. After each block, a questionnaire was given to assess participant's subjective opinion of that visualization. Visualization order (block), as well as Visdata block match, was counterbalanced using a Latin Square design to avoid any order or learning effects. Within each block, tasks order was fixed with the more difficult tasks at the end. This allowed participants to build their skills as they proceeded. At the end of the session, a general comparative questionnaire was filled out.

)a1) Static Radar (SR) chart

)a2) Dynamic Radar (DR) chart chart

)b1) Static Flower (SF) chart

)b2) Dynamic Flower (DF) chart Fig. 1: Two chart types (Radars and Flowers) encode CI data using Static and Dynamic Time mapping. We compared users' performances and preference empirically for elementary and synoptic tasks.

Table 2. User tasks given in the experiment. Task type refers to task taxonomy in [3]. Italic font is used as placeholder for data attributes

significant effect (F(9,207)=2.93, p=.003). Significant effect of Vis was found on two elementary tasks: 2 and 5 and on one synoptic task: task 10.

Task No.

Question

Task type

5.1.3 Error rate

1

What was the score of item a in index x on year y?

Elementary Lookup

2

When was the lowest/highest score of item a in index x?

Elementary Reversed Lookup

3

Was score of index x in item a higher than item b in year y?

Elementary Comparison

4

Was score of index x in item a higher than v before / after year y?

Elementary Comparison

5

Mark 2 years on which index x1 was higher than index x2 in item a. What was index x trend in item a between years y1-y2? Which indicators shows maximum divide in year y? Does divide between item a and item b in index x getting narrower/wider? Which item increased the most at any index from first to last year? Growth rate of index x in item a was higher/lower than item b

Elementary RelationSeeking Synoptic Pattern Identification Synoptic Behavior Comparison Synoptic Behavior Comparison Synoptic RelationSeeking Synoptic Behavior Comparison

7 8 9 10

5. RESULTS We present the study results in three parts: effectiveness (in task completion time and accuracy), behavior (in user interactions) and subjective preferences.

5.1 Effectiveness In order to analyze task completion time, we conducted a 2 (Time encoding: Static vs. Dynamic) x 2 (Vis: Radars vs. Flowers) x 10 (Tasks) within-subjects Analysis of Variance (ANOVA). Results were analyzed in log time to control for the skewness in reaction time data, but are reported in seconds for clarity. In order to analyze accuracy, we conducted a model for binary responses on correctness.

5.1.1 Main differences in completion times We compared overall completion times of the two time encoding conditions: Static and Dynamic and of the two visualizations: Radars and Flowers. In terms of the Time encoding, we observed a significant main effect, F(1,23)=4.51, p=.045, =.164. The average time when using the Static condition (M = 35.0 sec., SD = 21.24) was faster than when using Dynamic condition (M = 36.7 sec., SD = 20.76). A highly significant main effect was also found for Vis, F(1,23)=15.89, p=0.001, =.409. The average time when using Flowers (M = 34.08 sec., SD = 19.97) was faster than when using Radars (M = 37.61 sec., SD = 21.87). We did not observe a significant interaction effect of Time coding and Vis (F(1,23)=0.01, p=.93). This indicates that Static methods were faster than Dynamic methods, independently on the visualization technique. Similarly, the Flowers technique was faster than the Radars technique, independently on the time coding method.

5.1.2 Completion times within tasks Looking at completion times within tasks, interaction effect of Time encoding and Tasks was found to be significant (F(9,207)=5.66, p