On slide-based contextual cues for presentation ... - Semantic Scholar

Session: Personalization, Search & Usability

IUI'12, February 14-17, 2012, Lisbon, Portugal

On Slide-Based Contextual Cues for Presentation Reuse 1

Moushumi Sharmin1, Lawrence Bergman2, Jie Lu2, Ravi Konuru2 University of Illinois at Urbana-Champaign, Urbana, IL, USA {[email protected]} 2 IBM T.J. Watson Research Center, 19 Skyline Drive, Hawthorne, NY, USA {bergmanl, jielu, rkonuru}@us.ibm.com

ABSTRACT

primary challenge in presentation reuse is selecting the most appropriate material from many similar versions that exist on a user’s machine.

Reuse of existing presentation materials is prevalent among knowledge workers. However, finding the most appropriate material for reuse is challenging. Existing information management and search tools provide inadequate support for reuse due to their dependence on users’ ability to effectively categorize, recall, and recognize existing materials. Based on our findings from an online survey and contextual interviews, we designed and implemented a slide-based contextual recommender, ConReP, for supporting reuse of presentation materials. ConReP utilizes a user-selected slide as a search-key, recommends materials based on similarity to the selected slide, and provides a local-context-based visual representation of the recommendations. Users input provides new insight into presentation reuse and reveals that slide-based search is more effective than keyword-based search, local-contextbased visual representation helps in better recall and recognition, and shows the promise of this general approach of exploiting individual slides and local-context for better presentation reuse.

Information management systems and search systems are commonly used for locating desired materials [12]. Information management systems facilitate retrieval by utilizing descriptive naming and hierarchical file organization. To access a presentation given in IUI 2011, for example, a user can utilize date (2011) and event (IUI) to identify the desired presentation. However, users often create multiple versions of similar presentations, use similar names to describe them, and often struggle to devise appropriate naming [13]. Desktop search systems primarily rely on user-provided keywords to retrieve desired materials. There has also been prior work on using provenance, activity history, and creation/access times for supporting personal information search [10]. However, effectiveness of these systems depends heavily on what users remember about their materials, which is often difficult to translate into a search query. For example, a user goal such as “find all slides that contain a world-map showing our office locations in green” would be extremely difficult if not impossible to describe using a keyword-based query.

Author Keywords

Contextual recommendation; slide-based search; local context; visual representation. ACM Classification Keywords

A recent study conducted in a large organization confirmed the prevalence of presentation reuse among knowledge workers and pointed out the need for better reuse support systems, but offered insufficient guidelines for designing such systems [15]. In our research, we aim to gain deeper understanding of user practices and needs surrounding presentation reuse for designing effective support tools. We conducted an online survey (N=108) and a set of contextual interviews with avid presentation users (N=8). We found that users want to reuse, have specific information goals when trying to reuse, follow a specific process when reusing materials, prefer personalized organization of materials over desktop search systems, and express a need for better reuse support tools.

H.5.2 [Information Interfaces and Presentation]: User Interfaces  User interaction styles, User-centered design; H.3.3 [Information Storage and Retrieval]: Information Search and Retrieval  Retrieval Models, Search process. General Terms

Design; Human Factors. INTRODUCTION

Presentation composition is ubiquitous among knowledge workers. One common practice in creating new presentations is to start with an existing presentation [15]. Common reuse tasks include tailoring existing presentations for different audiences, creating summary or overview presentations by collecting materials from several existing presentations, and modifying existing presentations for specific events such as conferences and client meetings. A

Based on these findings, we postulated that an intelligent assistive interface that supports presentation reuse by exploiting slide-similarity metrics and contextual information will facilitate reuse. To validate this approach and elicit both concrete reuse patterns and user feedback, we designed, implemented and evaluated a contextual recommender, ConReP, for presentation creation tasks. When a ConReP user opens a presentation and selects a

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. IUI’12, February 14–17, 2012, Lisbon, Portugal. Copyright 2012 ACM 978-1-4503-1048-2/12/02...$10.00.

129



slide, the system automatically constructs a “contextual search query” by extracting text and images from the slide and uses these in conjunction with presentation attributes to find and recommend relevant slides stored on the user’s machine. User feedback on ConReP is positive and indicates the potential of such systems for supporting reuse.

keyword-based search for retrieving materials from their machines [12, 19]; instead, they navigate to their search targets by utilizing contextual knowledge about the desired material [24]. Since users remember the context of use or their experience with items better than keywords, research suggests that systems should utilize these cues in the search process [5, 6]. We believe a natural way to address the problem with keyword-based search is to take a semiautomated approach, in which users’ tasks lead to automatic generation of search keywords, an approach adopted in ConReP. Another problem associated with current search systems is users’ failure to recall sufficient details about an item by looking at textual search results [6]. To address this problem, we utilize a set of thumbnails from each presentation to create a visual representation of the search results, which not only enables direct access to the desired slides, but also provides context that helps in identifying desired materials.

Our work makes the following contributions: - Identifies specific patterns and challenges in the domain of presentation reuse. - Presents a slide-based recommendation technique that exploits slide similarity and local context. - Presents a novel recommender interface integrated into a presentation composition environment, which provides a local-context-based visual representation of the results to support recall and recognition. In the next section we present related work. We then describe a survey and a set of contextual interviews we conducted to deepen our understanding of presentation reuse. This is followed by a set of system design criteria derived from the study, and a description of the resulting ConReP slide recommender. We present results of an evaluation of ConReP and conclude with possibilities for future work.

Systems utilizing semantic document models try to make atomic content units (CUs) such as paragraphs and tables reusable [17], but their success is largely dependent on user annotations of the CUs. These systems also suffer from a lack of connection between the annotation and the content creation tools [12]. While semantic systems such as CPoint mitigate the connection problem by integrating annotation capabilities in the content creation environment [12], they still suffer from users’ unwillingness to spend time on annotation that may benefit future reuse [21]. To alleviate these problems, we not only integrate our system in the content creation environment but also auto-compute contextual similarity between contents, requiring little effort from the users to make contents reusable.

RELATED WORK

Existing research poses personal information reuse as a search and retrieval problem, namely “How can we help users to identify the best materials by building systems that assist in their retrieval?” [8]. Research in this area suggests that systems designed to support reuse need to: 1) support users’ current practices [15], 2) consider what users remember about their materials to facilitate the search process [21], and 3) create novel representations of the search results to support identification of desired materials [20, 21]. A deeper understanding of users’ reuse practices is the first step in designing better reuse support systems. In this research, we investigate how users reuse presentation materials and then design and evaluate the effectiveness of a slide-based contextual recommender for supporting presentation reuse tasks.

Several prior presentation composition systems have enabled comparison between different versions of the same presentation [4, 18, 25]. Other approaches to presentation composition have included outline matching [2], topic clustering [23], and hierarchical organization [1]. Very few researchers have investigated the design of recommender systems to support presentation composition and reuse. We extend this thread of research by investigating the effectiveness of slide-based contextual recommendation for facilitating presentation reuse.

Research in the area of personal information management suggests two psychological problems associated with retrieval: difficulties in generating effective categorization and difficulties in remembering the labels that support retrieval [13]. Studies suggest that location, file type, time, keywords, and associated events are the attributes best remembered [3, 7]. However, only a subset of these attributes (e.g., file type and keywords) is utilized by current reuse support systems. In addition to these attributes, visual elements are often remembered [13] and need to be considered, since reuse of visual elements is a key factor in presentation reuse [15].

UNDERSTANDING PRESENTATION REUSE Method

We conducted an online survey to learn about the prevalence of presentation reuse, types of materials reused, frequency of reuse, and sources of reused materials. We also collected information about the individual, social and organizational culture of reuse. We posted the survey link in an internal company mailing list and collected 108 completed responses over the course of two weeks. Survey results regarding individual, social, and organizational aspects of reuse are reported in [15]; here we report additional findings.

Search systems provide inadequate support for reuse due to their dependence on user-provided keywords [6, 22]. Researchers have reported that users seldom perform

130



for reuse. For example, people modify/update existing content to present to a different audience (retail industry vs. insurance industry), to give the presentation a different look and feel (informal vs. formal, technical vs. general), or to create a summary presentation (by combining materials from different presentations).

Survey Why do you reuse presentation materials? What type of materials do you reuse (text, charts, graphs, images, template, etc.)? How often do you start composing a presentation from scratch and from an existing presentation? Interviews

Type and Unit of Reuse

Presentation materials that are often reused include graphical elements (e.g. images, graphs), slides, and text. Presentations are also used as sources of data, and as templates for new presentations. Figure 1, based on the survey, shows reuse of different types of presentation elements. While slides and graphical elements are the most frequently reused units, existing systems offer little support for searching for specific slides or graphical elements. Participants reported trying to collect desired materials from existing presentations by searching for a remembered slide or specific visual elements such as graphs and then locate other needed materials within that presentation.

What is your process of reusing existing presentations? How do you find existing presentations (or slides) that you want to reuse? What types of information and tool do you utilize when you are looking for materials for reuse? What are the challenges (if any) you encounter during the presentation reuse process? Table 1. Sample questions from the survey and the interviews.

Following the survey, we conducted semi-structured interviews (N=8) to learn about details of the presentation reuse process, strategies utilized for supporting reuse, and overall reuse experiences. Participants were recruited from different departments of our organization, with different job roles (managers vs. researchers) and varying degrees of presentation reuse (people who reuse frequently vs. people who reuse occasionally). Interviews consisted of openended questions focusing on presentation reuse practices, lasted between 30 and 60 minutes, and were audiorecorded. During each interview, we used our questions as a guide, but welcomed tangents. To ground the discussion, we asked participants to walk us through presentations they had recently created utilizing existing materials. See Table 1 for a list of sample questions.

Process of Reuse

Study Results

Users mentioned formulating search criteria as the most difficult among all phases of the reuse process. All of our interview participants commented on the difficulty of formulating the right keywords, which is nicely summarized by the following quote:

From our contextual interviews, we learned that reuse is a complex and iterative process which often involves overlapping phases of formulating search criteria, searching and browsing existing materials, collecting the needed materials by selecting the most appropriate materials from the search and browsing phase, and finally constructing a new presentation by utilizing the collected materials. Figure 2 presents a diagrammatic representation of the main tasks involved in the reuse process. The success of reuse depends on the quality of the formulated search criteria, the success of finding relevant materials, and finally, the identification of the best available material for reuse. Challenges in Reuse Finding the right keywords for search is difficult

Prevalence of Reuse and Reasons for Reuse

Reusing existing presentation materials is common; only 3 out of 108 survey participants reported never reusing presentation content. Participants considered reusing existing materials to be a “natural practice.” Repurposing existing content for different audiences, events, formats, etc. are the primary reasons mentioned by our participants

“I have a hard time doing it. I remember ‘Oh, I had a slide I remember that looked a little like this’; I can think about this in my head but I can’t type-in the right keyword for that, so that’s never found. That would require me to remember a lot of keywords.” – Participant 4

Figure 1. Reuse of different types of presentation materials.

Figure 2. Process of reusing presentation materials.

131



This difficulty discourages users from utilizing keywordbased search tools for finding relevant presentation materials. Only one out of eight interview participants mentioned occasionally using a desktop search tool when she remembered exactly what presentations she was looking for, but described it as a very difficult process.

names and presentation names that users utilize for supporting retrieval. All the interview participants reported using personalized directory and naming schemes to facilitate retrieval. The degree of difficulty associated with such location-based or name-based search and retrieval is evident from the following quote:

Gaining an overview of all relevant materials is timeconsuming

“(For retrieval) I use location of file. I just try to remember (where I store it). I try a combination of lot of factors I think. Titles of meetings or projects aren’t that much (helpful), I have to remember what I was writing the presentation for, what the meeting was about, that kind of thing.” – Participant 7 Another problem with personalized directory structure and naming schemes is that with time the names lose their effectiveness as memory cues. While a name like “speech technology 9-7-2002” indicates that this presentation contains materials regarding to speech technology project(s) and it was created in September 2002, this name does not indicate what type of materials were contained in this presentation, why this presentation was created, and whether there exist other similar presentations about similar topics. Additionally, projects expand, change directions, and new projects emerge that have overlap with past projects. However, the directory names that were used for storing the early presentations are typically not updated to reflect these changes. Presentations also get modified based on the changing direction of the projects, often making presentations stored in the same directory very dissimilar. One interview participant commented on this aspect of organization and the problems associated with this strategy:

Many similar versions of presentations, slides, and slide contents exist; an extremely challenging aspect of the reuse process is the accumulation and exploration of all the different versions of relevant materials. Some interview participants mentioned trying to collect the different versions before the presentation construction phase to gain an overview and to compare the differences between the versions in order to select the best version. Participants felt challenged, since it not only was time consuming but also required them to remember the names and locations of the sought-after materials. Seven out of eight interview participants mentioned that gaining an overview of the available relevant materials is the main challenge in the material collection phase. One participant stated: “If I get a summarized variation of what I have available, then (a system) already helps me tremendously (in the reuse process).” – Participant 1 Currently, participants open presentations in sequence and browse the slides within each presentation to get a better understanding of the available materials. Two interview participants also mentioned creating “master decks” which contain all desired slides on a topic, but mentioned the difficulty of updating the deck with new materials.

“I name my directories meaningfully. But the meaning is never updated, that’s the problem. When I start a new project, I create new directory but then the very semantic of the thing (project) changes and it evolves. I don’t do any updates of my directory and I create it once and I stick with it. It grows and grows and the taxonomy that I defined in the beginning becomes meaningless as I get deeper into the project and time but I still live with it.” – Participant 5

Identifying the most appropriate version is non-trivial

Selecting the most appropriate version of presentation content requires recalling information about the presentation content and recognizing the difference between similar materials. Users reported struggling to correctly identify the most appropriate presentation by only looking at presentation names or thumbnails. Presentations pertaining to the same project often have similar names with content that differs slightly based on the intended audience (insurance industry vs. retail industry), the specific event type (group presentation vs. conference talk), and progression of the project (weekly project status vs. biyearly project status). To mitigate the challenges associated with recall and recognition, we observed that all of our interview participants are “satisficing” [22] instead of investing time and effort to identify the best version.

SYSTEM AND USER INTERFACE DESIGN Design Goals

Based on our findings, we identified the following design goals for a system that facilitates presentation reuse: 1) Support reuse starting from an existing presentation. This is a common practice as shown by prior work [15] as well as our survey and interviews; users more frequently begin with an existing presentation than “from scratch.”

Personalized organization is not an effective retrieval strategy

2) Support easy identification of the “best” version of any given slide within existing presentations. As noted in our study, finding the best version is of value to users, but challenging with existing tools.

Users predominantly use descriptive file naming and personalized directory structure when storing their presentation materials. While this behavior is common for other types of electronic documents, the unique characteristic of presentation materials (e.g. the target audience, the event) are often reflected in the directory

3) Support the user’s ability to recognize materials. Users’ ability to recognize and recall is enhanced by information

132



that stimulates their ability to remember the presentation’s context such as purpose, audience, and event [3, 5]. 4) Support users in easily locating and recognizing additional materials related to a slide in the existing presentation. Our interview findings indicate that users often remember information about co-occurrence of slides or specific materials within a presentation and use a “remembered slide” to gain access to the desired materials. In support of these goals, we identified several key design features: 1) Slide-based recommendation. As discussed, keywordbased search is difficult for presentation materials. Rather than have the users construct queries, we chose to use the content of slides within an existing presentation as input to a recommendation algorithm. A local database containing information about similar slides and presentations stored on the user’s machine is used to produce slide-specific recommendations.

Figure 3. System Architecture.

presentation and slide contents, creates thumbnails of the slides, and passes the slide information back to the content analyzer. The content and attribute analyzers compute pairwise slide similarities and presentation similarities respectively and store the similarity scores in a database. Desktop crawling, presentation uploading, and similarity computations are executed as offline pre-processes.

2) Support for visual comparison of recommended slides. For selecting a best version, the user needs to be able to readily compare multiple similar slides. 3) Support for visual presentation of local-context for recommended slides. Single slides, particularly when multiple versions are being presented, are not adequate cues to support users in recognizing and recalling the needed materials. Furthermore, local-context-based representation provides support for finding related materials located nearby within a presentation.

A ConReP user begins by loading an initial presentation into PowerPoint, then initiates the recommendation process by selecting a presentation slide and clicking a toolbar button (Figure 4(a)). ConReP presents recommendations based on the selected slide in a separate, repositionable, and resizable window, which by default is placed near the slide thumbnails shown in PowerPoint (Figure 4(l)). The recommended slides are displayed vertically as thumbnail images (Figure 4(e)), with the most highly recommended at the top. In addition, each recommendation is flanked by local-context slide thumbnails (Figure 4(f)) – slides preceding and following the recommendation in the containing presentation. By double-clicking on any thumbnail, the user can view a popup window containing a larger slide image (comparable in size to the main PowerPoint slide view). The interface also contains controls to page through recommendation results (Figure 4(d)), as well as local-context slides (Figure 4(h) and 4(i)). Finally, the user can insert any particular slide into her presentation using the “insert” or “replace” control provided in the larger slide window (not shown in Figure 4).

System Overview

We developed the ConReP system to provide slide-level content recommendations. To closely integrate ConReP’s recommendations into users’ presentation creation tasks, we implemented ConReP as an add-in for Microsoft PowerPoint 2003, a widely used presentation editor [16]. Figure 3 shows the architecture of ConReP. It has two main subsystems: a front end containing the ConReP UI and a recommender, and a back end containing two similarity analyzers (content and attribute analyzer), a slide information aggregator, a desktop crawler, and a presentation uploader. The recommender formulates the search query by extracting text and image from the user selected slide and then by accumulating information about the containing presentation (e.g., file attributes). The similarity analyzers match this query against a local database to extract identifiers for other similar slides and associated presentations. These are passed to the slide information aggregator, which collects slide thumbnails from the presentation repository and passes them to the recommender, which creates a visual representation of the recommendations. The desktop crawler is responsible for collecting presentations and their attributes, and for passing them to the presentation uploader and to the attribute analyzer, respectively. The presentation repository extracts

User Interface Design and Preliminary Evaluation

Our study indicated that reuse of slides is common in presentations; any given slide is likely to be present in multiple files. Showing only similar slides (along with presentation name) may be inadequate to allow users to recognize particular versions, and to select the best of them. For this reason, we considered it important to include local context for each recommendation. We use the term “localcontext” to refer to a small group of co-located slides from a presentation surrounding the recommendation slide.

133



Figure 4. The ConReP system is implemented as an add-in (a) for Microsoft Power Point. User selects a slide from the active presentation (b) and clicks ConReP to view recommendations. ConReP uses a page-based recommendation interface (l) consisting of ranked thumbnail representations of multiple presentations showing - (c) total number of recommendations, (d) control for viewing next group of results, (e) recommended similar slide, (f) local-context slide, (g) slide title and location in presentation, (h and i) controls for navigating to the other slides of the presentation, (j) control for returning to the original recommendation, and (k) presentation title.

An open question remained: how much context is required for recognition, and how to best present that context. To explore this, we developed three initial prototype visualizations, each containing a different level of context. The three prototypes provided minimal-, maximal-, and local-context slides in the recommendation window. The minimal-context interface (Figure 5) shows all relevant slides in one window (one slide per presentation), but allows users to examine the other slides from a presentation on demand (double-clicking the recommendation slide opens another window showing thumbnails for the other slides from the same presentation). The maximal-context interface presents all slides from the relevant presentations in separate scrolling inner-windows, displaying the recommendation slide in the middle (Figure 6).

Figure 5. Minimal-context interface.

The local-context interface on the other hand provides a group of neighboring slides from each presentation in separate inner-windows with the recommendation slide highlighted in yellow (Figure 7). In all three interfaces, presentation names and slide titles are displayed to provide more information about the slides and their containing presentations. Providing slide titles also help to differentiate similar-looking text-heavy slides that are relatively difficult to distinguish from the thumbnail representations. We conducted a preliminary design evaluation of these three prototypes (N=5) to investigate the advantage and disadvantage of each prototype. Each participant used all three interfaces with the same set of presentation materials. We counter-balanced interface use to remove order bias.

Figure 6. Maximal-context interface.

134

Figure 7. Local-context interface.



All of our participants preferred the local-context interface, commenting that neighboring slides made it easy to differentiate between similar presentations, especially where the recommendation slides were the same or very similar. They also liked this local-context-based interface because it rendered the recommendations faster than the maximalcontext interface (due to the smaller number of slides returned) yet at the same time provided access to the slide’s context to help them identify different versions. Based on the feedback received, we selected the local-context interface for our final implementation (Figure 4 shows the final design).

greater than 0.5, and the resulting pairs (including slide identifier and associated presentation identifier for each pair of slides, and the overall similarity score) are stored in a local database. Recommendations are produced by retrieving all database records containing the slide identifier of the slide selected by the user from the PowerPoint interface. This gives us the slide identifiers and the identifiers of the containing presentations for each slide similar to the query slide. The retrieved records are sorted in descending order on similarity score. The list is then filtered to ensure that only a single slide (the one with the highest similarity score) is retained from each presentation. Since our UI permits navigation to any slide within a presentation, we felt that ensuring each presentation was included only once would minimize confusion. Each slide retained in the recommendation list is enhanced with the presentation name, and a set of localcontext slides. The list is presented to the user, 3 presentations at a time, in order of relevance.

Recommendation Algorithm

ConReP recommends slides based on assessment of pairwise relevance between each slide on the user’s desktop and the slide selected in the main PowerPoint UI. We measure relevance between two slides by computing the similarity between the slides’ contents and the similarity between their contexts (attributes of the presentation files). Below we describe how to compute each similarity, and how they are combined to determine a recommendation set.

EVALUATION

Our evaluation of ConReP (with the local-context interface) explored how well we met our design goals, and how well the system supports typical reuse behavior.

Content similarity is computed based on two components: 1) all text (which includes slide title, bullet text, image captions), and 2) all images included on each slide. Text similarity and image similarity are computed individually for each pair of slides. We utilize a modified version of Levenshtein distance [14] for computing text similarity and Jaccard similarity [9] for images. Details of these similarity measures are reported in [15].

Method

We conducted a lab study with six users, all employed in a large organization. We recruited participants with a variety of job responsibilities – marketing directors (2), senior managers (2), and research scientists (2). We selected participants who described themselves as “avid presentation users” and affirmed their reuse practice as “extensive reuse, reusing own presentation materials for creating new presentations more than 50% of the time.” We focused on participants meeting this criterion as we believe that they are likely to benefit most from a presentation reuse support system. Each of the study sessions lasted about an hour and was structured as a discussion about current reuse practice followed by a short system briefing, a presentation creation task, and finally, a post-interview. Prior to the lab session, we collected a set of presentation materials from each participant (20 presentations on average) for use during the session. These materials allowed us to simulate the effect of a personal presentation recommender system.

Contextual similarity is computed based on file attribute similarity of presentations. Since user-assigned file names (including file path) often contain important identifying information including event/project name, presentation date, or purpose, attribute similarity is computed using file names. For each presentation, a bag of words was extracted from the fully-qualified (full path) file name. The file name is split on delimiters, including the path delimiter (a slash or backslash), and other delimiters that users typically include within file names such as periods, dashes, and underscores. The “words” that are obtained by splitting the file name were filtered by removing stop-words. The attribute similarity between two presentations is then computed as the Jaccard similarity between the resulting word bags for the two associated file names. We define the attribute similarity between two individual slides to be the attribute similarity between their containing presentations. As such, attribute similarity is calculated only once for each pair of presentations.

The goal of the system briefing was to inform participants about the functionality offered by ConReP. Next, we asked them to create a presentation similar to any of their existing presentations, but to repurpose it for a different audience (as mentioned earlier, for each participant a set of presentations was stored in the study machine). We chose this task because our survey and interview results indicated repurposing existing presentations for a different audience as the most common reuse scenario. Participants were encouraged to think-aloud during the presentation creation process. We concluded the session with a post-interview in which we asked participants to share their opinions about the search process, the quality of the recommendations, the

An overall similarity score is computed for each pair of slides on the user’s desktop (a desktop crawler is utilized to find presentations and extract slides). The overall similarity score is the normalized sum of the text similarity, image similarity, and attribute similarity scores. Since each of the component similarities is in the range of [0, 1], overall normalization is accomplished simply by dividing by three. The pair-wise similarities are filtered to retain only those with a value

135



effectiveness of the interface design, and their overall opinion of the system. We also asked what features they liked and disliked and any missing features they would like included in the system. We audio and video recorded each session. The video recordings were intended to capture interactions with ConReP, use of different features, and any system errors. The audio recordings (both the think-aloud comments and the interviews) were transcribed and later analyzed using a thematic analysis approach.

created to convey the same information. Participant 1 who had concerns about the quality of automated recommender, stated: “I am very surprised to see how well this system works for (recommending) graph-heavy slides.” – Participant 1 All participants liked the visual representation, commenting that “the representation seems natural and intuitive.” All six participants also considered the representations easily interpretable, all giving it 5/5 in the post-study questionnaire. To quote a participant:

Results Overall Opinions

“It’s nice, it’s very nice. I think what I like is the visuals, you know for slides it has to be a visually oriented operation, so I can view all the matches and context. I think this is the right idea.” – Participant 6 Participants were also able to identify the different versions of similar presentations easily (average score = 4.67/5.0). While four out of six participants reported that identification of different versions of similar slides were easy, two pointed out that identifying the differences between text-heavy slides could become difficult (average score = 3.83/5.0).

Table 2 presents a summary of our participants’ opinions about ConReP as a contextual recommender using a 5-point Likert scale. Four out of six participants mentioned that they are very interested in continued use and they see themselves using this system a lot (they also gave it a 5/5 in terms of overall rating and interest in continued use). The remaining two participants gave it 4/5 mentioning that they would use it on a regular basis (average score = 4.67/5.0). Prior to the study, four out of six participants were interested but hesitant about the usefulness of an automated presentation recommender, one commenting that such systems “may not be intelligent enough to recommend anything useful.” However, after using ConReP for the presentation creation task, their opinions were changed. While using the system our participants used words such as “Nice!”, “Excellent!”, “Great Idea!” and “Impressive!”

Effectiveness of Slide-Based Search

ConReP uses slide-based search with the goal of freeing users from the burden of selecting appropriate keywords. All of our study participants considered slide-based search very effective, commenting on how it freed them from remembering keywords. Four out of six participants commented on the usefulness of this search technique for searching graph-heavy slides, as graphical elements are reused more often than text and forming appropriate search queries to retrieve graphical elements is extremely difficult. Figure 4 provides an example in which Participant 2 was trying to find presentations that contain materials similar to the graph marked as “b.” After exploring ConReP’s recommendations he commented:

Users found the interface engaging and were impressed with the recommendations offered (average effectiveness of recommendation = 4.16/5.0). Five out of six participants spent most of their time exploring recommendations for a number of different types of slides (text-heavy, graphicheavy, slides with graphic and text, slides with a single image, slides with multiple images, etc.). They also compared the slides they selected and the recommended slides while speculating the algorithm behind the recommendation. One user was “pleasantly surprised” to see two recommendations that were visually different but were Metric Overall rating

“I have a slide that looked like this [pointing to b] but I can’t type in the right keywords for this slide because there can be a lot of keywords. What I like about this (ConReP) is that it is using more than keywords.” – Participant 2

Average Score 4.33

Interest in future use

4.67

Interpretability of presented information

5.00

Local context supporting recall

4.83

Recognize different versions of presentations

4.67

Relevance of recommendation

4.17

Ease of locating material within presentation

4.17

Identify different versions of similar materials

3.83

Ability to View and Select among Multiple Versions

Five out of six participants mentioned that the visual representation of the recommendations allowed them to gain an overview of the relevant materials and helped them to examine a variety of options. To quote two participants: “This graphic is so complex and it’s a pain to find the right version. Showing all the versions are useful because otherwise you have to manually go and find the other ones.” – Participant 5 “I think it is a really necessary tool, especially in our environment where you do have quite often to combine slides on demand very quickly, put them together. Just having this visual overview of multiple presentations is very useful because you can navigate and just build a picture of what you want to put together.” – Participant 3

Table 2. Opinion about ConReP as a contextual recommender (Using a 5 point Likert-Scale).

136



Usefulness of Local Context

version. I think it is useful to see it in the context.” – Participant 3 Local-context slides not only offered a better understanding of the presentation content, but also helped users understand the flow of the presentation and the context of use of specific slides. Our participants mentioned that presenting localcontext slides was critical, since they helped in understanding the lead-up slides and supported understanding of slidegroups that are part of a story. To quote a participant:

For the vast majority of recommendations, the context slides flanking the recommendation slide (2 on each side) provided a unique “signature”; only 2% of the total recommended presentations had identical local-context slides. Figure 8 illustrates an example where the top three recommendations were copies of the same slide utilized in three different presentations; however, the local-context slides were different, helping the user recognize the different versions of the presentation.

“Sometimes you know you have a slide that is part of a story and just showing the thumbnail (only one thumbnail for the similar slide) won’t tell you how did you get there and what lead up to it.” – Participant 4 Showing multiple slides also helped in collecting nearby relevant slides. We observed that participants often used the local-context slides to select the best version of the presentation from which other desired material could be collected. Users expressed that ConReP made it easy to locate the needed materials within a presentation (average score = 4.16/5.0).

User behavior seems to support the notion that local-context slides were sufficient to support recall and recognition. Users could navigate to the other slides from the recommendation slides by utilizing the left and right navigation controls, but they rarely needed to do so for recognizing the presentations; only 6% of the total slides explored during the study were not among the initial recommendations. All of our participants considered local-context slides very effective since they provided a better way of identifying different versions of similar presentations, and helped them recall the presentations’ construction story, audience, and purpose. Five out of six participants considered local-context “very useful” for supporting recall with one selecting “useful” (average score = 4.83/5.0). Participants commented on the effectiveness of local context:

DISCUSSION Limitation

Although our lab study involved collection of a considerable amount of materials from users, better indicators of effectiveness would be gained by a long-term deployment on users’ machines. We hope to gain additional insight into the potential of contextual recommenders in assisting users in presentation reuse through such a future field deployment.

“Local context is useful because it tells you, gives a sense of what presentation it was.” – Participant 5 “Local context does (help in recall), it gives me more information. I can recall that I had started with this older

Areas of Improvement

The current ConReP system utilizes a user-selected slide as a search-key and offers recommendations based on the similarity of extracted slide content, file attributes such as file name and directory path. While we consider this to be a reasonable first step, utilization of other information is probably desirable. For example, integrating provenance information would enable the system to recommend presentations that are different in terms of content but were used by the user as a reference during the original presentation creation task. Using provenance and temporal proximity of use may shed light on relatedness among dissimilar presentations. Currently, ConReP does not require users to provide any search keywords and instead requires users to start with an existing presentation similar to what they are looking for. However, sometimes, especially for recent presentations, users may know exactly what they are looking for and may not need to see all the variations. Allowing users to enter search keywords in addition to selecting a slide as a searchkey may offer better support for some reuse scenarios. In most cases, users were successful at comparing different versions of slides by viewing the thumbnails and commented that they did not need to utilize the larger sized view for examining slide content. During the study, however, we

Figure 8. Recommendations generated by ConReP. The top thumbnail (row 1) represents the selected-slide and rows 2 through 4 represent recommendations. Slides are ordered based on the level of similarity (Portion of the presentation names are blurred to ensure confidentiality).

137



8. Hansen, D.L. and Golbeck, J. Mixing it up: recommending collections of items. Proc. CHI, ACM Press (2009), 1217-1226.

observed that our participants examined the details of 15% of all recommendation slides (by double-clicking on the thumbnail to view the larger image). Participants later stated that their primary motivation for doing so stemmed from a curiosity to compare the recommendation slide with the originally selected slide (used as a search-key). Additionally, we observed that all of the slides for which details were explored were text-heavy; indicating that for such slides additional measures could be taken to highlight the differences.

9. Jaccard, P. Distribution de la flore alpine dans le bassin desDranses et dans quelques régions voisines. Bulletin de la Société Vaudoise des Sciences Naturelles, 37, (1901), 241-272. 10. Jensen, C. et al. The Life and times of files and information: a study of desktop provenance. Proc. CHI 2010, ACM Press (2010), 767-776. 11. Khoo, C. S. G., Luyt, B., Ee, C., Osman, J., Lim, H.–H, and Yong, S. How users organize electronic files on their workstations in the office environment: a preliminary study of personal information organization behaviour. Information Research 11, 2 (2007).

CONCLUSION

We have presented ConReP, a contextual recommender that enables users to gain an overview of existing relevant materials and identify the most appropriate material for reuse from several similar versions. ConReP’s design is motivated by user needs surrounding presentation reuse and is guided by literature in the area of behavioral psychology, personal information management, desktop search and recommendation. We have investigated the efficacy of this system in a lab study with users exploring their own presentation materials. The study participants found ConReP to be highly usable and useful, and validated the general approach. Our semi-automated approach (search queries automatically created from user-selected slides) helps users overcome the challenges associated with traditional keywordbased search. The system aided them in locating and identifying multiple similar versions of content, as well as useful accompanying material. ConReP is valuable as both a tool for supporting reuse of existing presentation materials and as a mechanism to gain new insight in the domain of presentation reuse that can guide the design of new reuse support tools.

12. Kohlhase, A. and Kohlhase, M. CPoint: Dissolving the Author’s Dilemma. MKM 2594, (2003), 175-189. 13. Lansdale, M. The psychology of personal information management. Applied Ergonomics 19, 1 (1988), 55-66. 14. Levenshtein, V.I. Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady 10, 8 (1966), 707-710. 15. Mejova, Y., Schepper, K.D., Bergman, L., and Lu, J. Reuse in the wild: an empirical and ethnographic study of organizational content reuse. Proc. CHI 2011, ACM Press (2011), 2877-2886. 16. Microsoft. “Microsoft PowerPoint.” Accessed Oct 19, 2011, from http://office.microsoft.com/en-us/powerpoint/. 17. Nešić, S. Semantic Document Model to Enhance Data and Knowledge Interoperability, ICWE 6, 3(2009), 135-160. 18. Perlin, K. and Fox, D. Pad: An alternative approach to the computer interface. Proc. SIGGRAPH, ACM Press (2003), 5764. 19. Ravasio, P., Schär, S.G., and Krueger, H. In pursuit of desktop evolution: User problems and practices with modern desktop systems. Computer-Human Interaction 11, 2 (2004), 156-180.

REFERENCES 1. Bederson, B.B. and Hollan, J.D. Pad++: a zooming graphical interface for exploring alternate interface physics. Proc. UIST 1994, ACM Press (1994), 17-26.

20. Secord, A., Winnemöller, H., Li, W., and Dontcheva, M. Creating collections with automatic suggestions and examplebased refinement. Proc. UIST 2010, ACM Press (2010), 249258.

2. Bergman, L., Lu, J., Konuru, R., MacNaught, J., and Yeh, D. Outline wizard: presentation composition and search. Proc. IUI 2010, ACM Press (2010), 209-218.

21. Sharmin, M. et al. Understanding knowledge management practices for early design activity and its implications for reuse. Proc. CHI 2009, ACM Press (2009), 2367-2376.

3. Blanc-Brude, T. and Scapin, D. What do people recall about their Documents? Implications for Desktop Search Tools. Proc. IUI 2007, ACM Press (2007), 102-111.

22. Simon, H.A. A Behavioral Model of Rational Choice. The Quarterly Journal of Economics 69, 1 (1955), 99-118.

4. Drucker, S.M., Petschnigg, G., and Agrawala, M. Comparing and managing multiple versions of slide presentations. Proc. UIST 2006, ACM Press (2006), 47-56.

23. Spicer, R.P. and Kelliher, A. NextSlidePlease: navigation and time management for hyperpresentations. Ext. Abstracts CHI, 2009.

5. Dumais, S., Cutrell, E., Cadiz, J., Jancke, G., Sarin, R., and Robbins, D.C. Stuff I’ve seen: a system for personal information retrieval and re-use. Proc. IR 2003, ACM Press (2003), 72-79.

24. Teevan, J. et al. The perfect search engine is not enough: a study of orienteering behavior in directed search. Proc. CHI, ACM Press (2004), 415-422.

6. Fuller, M., Kelly, L., and Jones, G. Applying Contextual Memory Cues for Retrieval from Personal Information Archives. PIM Workshop at Proc. CHI, 2008.

25. Zongker, D. and Salesin, D. On Creating Animated Presentations. Proc. SIGGRAPH, ACM Press (2003), 298-308.

7. Gonçalves, D. and Jorge, J. A. Describing documents: what can users tell us?. Proc. IUI 2004, ACM Press (2004), 247-249.

138