Content driven web scene creation for a tab manager and window ...

Content Driven Web Scene Creation for a Tab Manager and Window Manager Siva Kumar Gunturi, Deepak Mittal, Joy Bose Web Group Samsung R&D Institute Bangalore, India {siva.gunturi, deepak.m1, joy.bose}@samsung.com Abstract— Existing tab/window based applications display a static thumbnail of the page or web app representing the onscreen contents of the display area to the user. There is no way for the content developer to specify what needs to be displayed in the tab/window manager, when a certain page or web app goes to the background. The paper presents a new method, where the content provider can specify the contents that can be displayed, when switched to the tab/window manager. The specified content can be an image, group of images or other html content which the content provider wants to show, based on user’s interests and past activity. In our implementation a web scene is created with the specified content and shown on the tab/window manager thumbnail area. We discuss the different use cases that benefit both the content developer and the end user’s interests. A field survey with a small group of users is conducted to gauge the usefulness and effectiveness of the feature. The paper also discusses a method for representing the content provided image or web scene by resizing tabs in the tab manager UX through prioritization of web-scenes. Keywords— Web apps; tab preview; static bitmap tabs; live rendered tabs; tab manager; window manager; web scene; static thumbnail; Advertisements

I. INTRODUCTION Computer systems display various windows into which users may input data, manipulate data or activate procedures. If many applications are running simultaneously, all the corresponding windows are displayed. Also, there could be multiple windows for any one application. So, when the user is viewing a video on a media player, there will be a separate window corresponding to the media player application. As the number of active applications increases, so does the number of windows that are displayed on the computer display. While running multiple applications simultaneously, the user is often faced with multiple windows on the display causing a cluttered desktop, leading to confusion and frustration. The user has to waste time in finding a desired window each time the user wishes to work on a different application. The user may also have to minimize windows or quit applications altogether in order to decrease clutter. However, if windows are minimized, then the user no longer has immediate access to the corresponding application. To access the minimized application, the user must first locate the desired window and open it, which is time consuming.

Likewise, if the user quits the application to clear desktop clutter, then the application is no longer active and consumes even more time to re-launch and access the application. Currently, when the user needs to change an application window or a browser tab, the tab managers or the window managers in the web platform and browsers display only the static content or viewport captured snapshots. There seems to be no motion or animated preview of the webpage being shown to the user based on the predefined criteria in the tab manager. Thus, there is a need for a method where the content developer can specify what to show in the tab manager and the window manager. The above mentioned shortcomings, disadvantages and problems are addressed in this paper. Today we are seeing a major change in the contents of active web pages, especially in the case of e-commerce web sites that prompt the users to check many discount options on various products in order to keep the user glued to the current website for long. Their intention is to persuade the users to spend more time on the site. The content providers are interested in informing the user when the price of a product is reduced or a new offer is available on a product the user is interested in. But once a user has put the current webpage in the background and opened another tab in the browser, there is no way for the content provider to inform the user of price changes or special offers on those products and services. In this paper, we address this problem by using the tab manager in a mobile and window manager in a desktop to show interested advertisements or offers on the corresponding tab when it is minimized and shown in the tab manager. This opens a door of new opportunity for both the content developer and the end user. We propose a novel method of using the tab manager to show the user an interested area on the thumbnail of the tab of corresponding page. We create a web scene of the webpage/ web apps in place of view port display/static bitmap thumbnail, based on new contents specified by the content provider. The content provider can derive this new content by gathering the browsing information of the user on his website and displaying the contents according to the user’s interests. This will help the content provider to update the user of discounts and offers that the user may be interested in, rather than showing mundane static thumbnails of the webpage that the user was browsing.

978-1-5090-3646-2/16/$31.00 ©2016 IEEE

This paper is organized as follows: Section II surveys prior work in this field. Section III provides details of current way of showing the tabs in the tab manager and the proposed new approach of content driven web scene creation for showing tabs in the tab manager. Section IV provides an evaluation of the model through a user study, and lists advantages of the proposed approach. Finally in section V we conclude and provide future work in this direction. II. RELATED WORK There has been previous work done in the area of tracking the recent changes to the websites, which we require for the purpose of pushing advertisements to the areas of the web page where the user is interested in. There are several free tools [1] and browser extensions one can use to track and monitor changes on websites, as well as studies on tabbed browsing behavior [2-3]. Here we survey some of these web-based tools and browser extensions. Versionista [4] is one such tool that can monitor webpage usage. It requires an account to begin monitoring web pages. Once we add a webpage that we need to monitor, it will scan the pages within the website. In the settings, one can filter out numeric changes (for visit numbers), days of the week and HTML codes. This ensures that the changes that one sees are part of the content only. VisualPing Page Monitor [5] is a Chrome extension that can monitor desired pages and notify the users of any changes to a given site. Once it is installed in the user’s Chrome browser, the page monitor button at the end of the Chrome address bar will have a number on it to signify that there is a new notification. In the options, one can set the intervals from once every 5 seconds to once every 2 days. There is also the option to enable desktop notifications with a sound alert that will remain on one’s desktop until removed. A cool option in Page Monitor is that one can select specific elements on webpages to monitor. So instead of monitoring the whole site, it can monitor just a specific HTML div wrapper of one’s choice. For Firefox users, there’s a similar add-on called Alert Box [6]. After installing Alert Box, one can go to the site one wants to monitor and click on the Alert Box button at the end of the Firefox address bar. With Alert Box, one can choose specific areas to monitor, by selecting it with the mouse. It highlights boxes of content by selecting the HTML div. Any changes subsequently made on the webpage will be reflected with a notification on Firefox, together with a sound. Check intervals can be set between every 2 minutes to every few hours. Similarly, there are several online site tracking tools for any changes in the webpage by rectangular differentiation in the original and changed pages. There are also some patents [9-10] existing in the field of live thumbnails and pushing ads. But none of the above related works can create a content provider desired image to be displayed in one of the tabs in the tab manager or window manager, based on the content embedded in the web page to be shown to the user. In this aspect, our work fills the gap.

III. TABS IN THE TAB OR WINDOW MANAGER A. Existing display of tabs in the browser’s tab manager Existing tab managers or window managers in web platforms and browsers display the static captured thumbnails of the onscreen contents. Instead of showing static thumbnails, Chrome tab managers can show live onscreen contents. In the live onscreen mode, the live contents are rendered to the tab thumbnail region for the active tab. Due to memory constraints this is shown only for the active tab, and the inactive tabs still use the static bitmaps of the background pages. So, the tab manager thumbnails are either static bitmaps of the onscreen display, or live contents rendered on to the thumbnail region. In both cases, there is no way the content provider can specify their own desired content to be displayed in the tab manager. Fig. 1(a) shows the Amazon website opened in the Chrome browser on Android, while the corresponding static thumbnail captured in the window manager is shown in Fig. 1(b). As we can see, the window manager shows a static bitmap of the window and does not consider any update of the user’s area of interest in the screen captured.

(a)

(b)

Fig. 1. (a) Amazon.in website shown in the Chrome browser, and (b) a static thumbnail of the same shown in the window manager along with other chrome browser tabs.

This prompts the need for finding a provision in the web content to enable the content provider to decide what to display to the user when the webpage goes to the background mode in the web browser. B. Proposed content driven scene creation of a tab in the tab manager In our system, we aim to show content relevant to the user’s area of interest on the thumbnails in the tab manager. Let us consider a use case where the user opens the amazon.in web page in the Chrome mobile browser and searches for men’s watches. This is shown in Fig. 2(a). After browsing for some time, the user switches to some other website and leaves the browser tab open as shown in Fig. 2(b). When the same user revisits the window manager any time later, Amazon can display to the user the latest offers on watches as an image in the amazon tab area, as shown in Fig. 2(c). If the user likes the content presented by Amazon in the window manager, they

will click on the image and then will be redirected to the particular product.

(a)

(b)

(c)

Fig. 2. (a) Amazon.in website opened in the Chrome browser. (b) Window manager along with other Chrome browser tabs. (c) An advertisement image from the amazon site in the Chrome browser tab.

In our system, the web scene creation of the webpage/web apps (in place of viewport display/static bitmap thumbnail) is based on new proposed CSS (Cascading style sheet) property of show-in-preview, for elements which the content provider wants to get displayed in the webpage preview. In order to do this, the content developers can give options to DIV or selected elements which the content developer wants to get displayed as part of the preview scene when the browser or web app requests the web engine to display. The criteria according to which the content developer can specify content to show as part of preview is one of the following: 

User preferences for content tracking (for images, video, animations, ads, offers etc.)



User interested areas using web-page content analysis.



Most rasterized portions of content, other than animations.



View port real content change detection.



Inside viewport /outside viewport. Available rasterized content / on demand rasterized content.

As mentioned, our system creates a web scene to be displayed in a tab manager or window manager based on the content embedded in the webpage. The method we use for this comprises of the following steps 

Analyzing content based on preview CSS (cascading style sheet) properties.



Prioritizing the identified content based on content provided properties.



Determining one or more display regions of the identified contents.



Based on the determined regions, render the content to create the web scene.



Play the rendered content in the order specified by content provider.

The overall process is illustrated in the block diagram in Fig. 3, which shows the interaction between different web engine components. Fig. 4(a) shows the code snippet of the content and the render tree of the content when shown in the preview mode. The content comprises of a HTML div element, having four image elements and in the style for the div it is specified to be shown in the preview mode (show-in-preview-mode) with a repeat option. When the user switches to the tab manager, the browser requests the engine to send only the contents that are specified to be shown in the preview mode, so the images specified in the below tree are rendered by the engine in background and web scene is created in the tab manager. When the browser engine switches to the preview mode, only the div elements are marked to render and all other elements are disabled. This is shown in Fig. 4(b). The content specified can be outside the visible screen as well. However, when switched to preview mode the engine gets all the contents specified under the preview mode and renders it to the tab manager display.

Fig. 3. Block diagram showing web scene creation process. (a)

Mobile Viewport

Tab Manager

(b) Fig. 4. (a) Code snippet of content for content scene creation. (b) Render tree for the web scene creation process.

Fig. 5 shows the snapshot of the tab web scene which is specified in the content. Sample images of amazon advertisements are taken as an example in the code snippet shown in Fig. 4(a). When the user switches to the tab manager, the images specified in the html are shown as a slide show. In Fig. 5(a) the image added in the code snippet for the preview in the tab manager of the amazon webpage is displayed. The green rectangular marking represents the present frame in the scene which is getting displayed. The upcoming web scene frames are also displayed as preview. Similarly Fig. 5(b) displays another such image. The content provider can display relevant web scenes as shown in Fig. 5.

Fig. 5. Tab manager showing web scene of Amazon site with advertising images.

Fig 6 shows the frames being generated in a web scene for web page content. The view port is the portion of the webpage that is visible on the user’s mobile or electronic display. The content of the web page that is outside the view port is selected through contextual analysis and is represented on the tab in the tab manager. Fig 7 shows a visual push of the animated slides and tracking a tab. The web scene preview could alternatively be pushed to a push notification [7], so that the preview scene will be displayed in the push notification.

Fig. 6. Facebook Page Content with Mobile Viewport and corresponding Tab manager. The Tab manager frames represent different areas of the web page that could be shown as animated frames.

Fig. 7. Illustration of the push notifications for the new offers and deals, and tracking these offers on the notifications space.

Fig 8 shows the flow chart of the purposed algorithm. Also, when a user switches to another application, he will be able to track if there are any updates in the previous window. An overlay preview scene is played on top of the active application if any visible preview is generated in the previous tab. Once the preview scene is done, the overlay preview tab will get dismissed. Fig 9 shows the overlay in the current page.

Fig. 9. Tracking a background tab in the web browser, the system displays the background updates in a floating overlay window.

IV. USER STUDY AND ADVANTAGES To evaluate the usefulness of the model, we conducted a small user study. There were 9 participants, 8 male and 1 female, aged 25-35. We showed the users the working prototype of the model, with special offers from the content providers displayed in the previews of the tabs in the tab manager. During the study, we used the following four approaches for getting user interaction with the tab manager 

Showing the Images of discount offer.



Showing the Slideshow of offers.



Showing discounts by push notifications.



Showing tab manager overlay updates for background tab on current page.

We have implemented the content for these approaches and tested them on quick access pages in the browser. We have hosted a test server with modified quick access test content pages so that users can access those pages using the test device. We have implemented this as part of default Samsung mobile browser [8].

Fig. 8. Flow chart for method of generating web scene preview in the browser rendering engine using contextual analysis.

Fig. 10. Results of the user study, averaged over 9 users, to gauge the usefulness of the system for showing the dynamic content in the tab manager.

We asked the users to rate the system for the following five criteria on a scale of 1-10 (1 being least and 10 being most): Effectiveness, annoying, likelihood of recommendation to friends, likelihood of buying the advertised product and usefulness of showing an animated ad in the tab preview as opposed to a static ad.

or window manager. We have presented some real time use cases of the system, and validated the effectiveness through a small user study.

The results are plotted in Fig. 10. As we can see, the users rated the feature high on usefulness and relatively low on annoyingness. They also rated it high on preference for animation and likelihood of recommending to friends. However, in other comments, some of the users mentioned the lack of familiarity for first time users of the feature. Although this is a rather small user study, we can state that preliminary results show a positive user response.

The work on this paper can be extended in the future for understanding user’s interaction on the features provided by the content provider that comes on the tab or window manager.

The advantages of our approach are many, some of them being listed below. 

Show the preview of user interested updates as a live scene to the user in the preview mode on the tab manager in the web browser.



The content developer can select the option of customizing this preview by using the CSS properties in the webpage content.



The preview generation can be for a web app, or webpage.



The system can track background updates of renderers/apps.



The system gives flexibility to content developers to choose the preview scene to be displayed to the user.



The preview scene can be created automatically based on user preferences, in case nothing is specified by the content developer.



The preview scenes can be represented as the state of app or background renderer in window manager/Tab manager/visual notifications or as an overlay scene. V. CONCLUSION

In this paper, we have proposed a new method of displaying dynamic thumbnail content or web scenes in the tab

A related patent [11] has also been recently filed for this work at the Indian patent office.

REFERENCES [1]

Brian Voo. Hongkiat. Free tools to notify you of website content changes. [Online]. Available: hongkiat.com/blog/detect-website-changenotification (2013) [2] M.Viermetz, Heinrich-Heine-Universitat Dusseldorf, C. Stolz, V. Gedov, M. Skubacz. Relevance and Impact of Tabbed Browsing Behavior on Web Usage Mining. In Proc. IEEE/WIC/ACM International Conference on Web Intelligence (WI'06), Hong Kong, 2006. [3] K. Kumar , J. Bose. Segregating user data by tabs in web browsers. In Proc. IEEE Asia Pacific Conference on Wireless and Mobile (APWiMob), Bali, 2014. [4] Versionista-Monitor Changes to Any Website. [Online]. Available: versionista.com. [5] Chrome Web Store. Visualping Page Monitor. [Online]. Available: chrome.google.com/webstore/detail/visualping/pemhgklkefakciniebenbf clihhmmfcd [6] Mozilla Firefox Add Ons. Distill Web Monitor (formerly Alert Box). [Online]. Available: addons.mozilla.org/en-us/firefox/addon/alertbox/ [7] Samsung Developers Technical Documentation. Samsung Internet for Android Overview. 2016. [Online]. Available: developer.samsung.com/technical-doc/view.do?v=T000000202. [8] Google Developers Web Updates. Push Notifications on the open web. 2015. [Online]. Available: developers.google.com/web/updates/2015/03/push-notifications-on-theopen-web. [9] Ramarao et al. Method and system for displaying proiritized live thumbnail of process graphic views. US Patent US 20130021355 A1. Filed Dec 28, 2010. [10] Yue Yading et al. Based on the scene information push advertising methods, systems and equipment. China patent CN101079063A. Filed June 2007. [11] Siva Kumar Gunturi, Munukutla Subrahmanya Praveen, Srirama Chandra Sekhar Mogali, Ancil K. George. System and Method of creating a web scene to be displayed in one of a tab manager and window manager. Indian Patent 201641010651. Filed March 2016.