User Intention Mining: A Non-Intrusive Approach to ...

User Intention Mining: A Non-Intrusive Approach to Track User Activities for Web Application Piyush Yadav, Kejul Kalyani, Ravi Mahamuni Tata Research Development and Design Centre (TRDDC), Tata Consultancy Services Limited, Pune, India {piyush.yadav1, kejul.kalyani, ravi.mahamuni}@tcs.com

Abstract. Monitoring user interaction with web applications is of vital importance as it helps in finding the user’s cognitive behavior towards applications and also helps in analyzing various web metrics. Understanding activity data and finding user insights is next challenge to the web world. User intention can be understood if the user activity logs are associated appropriately with application context. In current scenario, web applications are using crawlers, scrappers, bugs, bots etc. to track user activities and extract out relevant information. Thus, by modifying the source code, it is easy to track and maintain the logs of user activities. But, many organization use third party web applications, where changing the source code is out of reach, making it trickier to maintain the logs of user activities. In this paper, we present an innovative way of logging, processing and extracting out meaningful information by tracking user’s activities on web application. Keywords: Web Content Mining, User activity tracking, User monitoring, DOM Parsing, Visualization, Browser, Semantic Analysis, User Intention

1

Introduction

In recent years, there has been an enormous advent in the web technologies leading to its wide spread adoption. Monitoring user activities is becoming more challenging with increase in web usage. In an enterprise, where multiple people access key applications, it becomes a big challenge to investigate and find out their malicious activities. Monitoring can also help in generating user specific profiles, considering their interest and search history. Effective monitoring of user activities gives insights about many interesting information such as what exactly user did to perform various actions such as Signing Up, Searching a product, responding to surveys etc., i.e. deducing the user intention. User Intention can further be explained as the purpose behind the user’s basic actions. Many models have been proposed to monitor user activity. Earlier only navigational data was captured, and as the web complexity grew, data was captured page wise and semantically.

In this paper, we propose a way to capture the semantics of user activities. Our solution is based on the notion that, semantics of the user actions are on the webpage itself. The semantics of the user activities are extracted out by parsing the DOM page which is then categorized logically. It then explains how these data can be used to find out the user intent. 1.1

Related Work

In recent years, there has been lot of study on user activity tracking. User interaction with the system has been studied for wide variety of investigative purposes such as usability metrics, website performance, debugging etc. Various methods have been proposed to track user actions on website and these tracked data is used for calculating various metrics. One such method tracks user actions by using USA proxy which modifies the code by inserting javascript. It collects data about the mouse movements, keyboard inputs etc. [1][10][11][12]and then uses it to deduce navigational behavior and time based metrics. Later a different concept, accessibility-in-use was coined to differentiate real world data from laboratory studies [2]. Easily deployable tool was developed by modifying USA Proxy which captures longitudinal data unobtrusively. WebQuilt [6], a web logging and visualization tool, was introduced to show various user activity results such as the web pages people viewed, or the most common paths taken through the website for a given task or the optimal path for that task as designated by the designer. Letizia[7], another user agent tool that tracks user behavior and attempts to anticipate items of interest by doing concurrent, autonomous exploration of links from the user's current position. Later another method was proposed to study user behavior from a longitudinal perspective [3]. To understand the user behavior, user task models were defined and compared with actual user activity on the website. As explained in [5], user interactions were captured and analyzed, and the results were compared with the task model of the website supporting remote usability evaluation of websites. But, in all these studies, less amount of work was performed to understand the semantics of the captured user interaction. Later, as logging user activities achieved maturity, advancement in associating the user activity with semantics was needed. A study on understanding of semantics of user interaction was performed, which associated the semantics using microformats [4]. Microformats are the simple codes embedded inside the HTML tags which tracks the user interaction and extracts out semantic information using the same. Major drawbacks of using it are that not every element on a website can be given microformats. Also, very less meta-data information can be extracted out from same. User intention mining was also done to predict user actions based on the past recorded data [8]. In contrast to all these studies, our approach not only tracks the user activities but also associates semantics which is used to find user intention.

2

2

Method: Determining User Intention

Our solution is based on the fact that, user interacts with webpage and user intentions can be deduced from the webpage itself. HTML is the basic building block of webpage and its tags can be broadly categorized into three components viz. interactive, decorative and structural as shown in Fig. 1. Interactive refers to those tags which are capable of taking inputs from the user or to invoke some action or change the state of the webpage. Examples of interactive tags are input, submit etc. Decorative refer to those tags which try to add meaning to the input tags such as label or span. These tags are used to justify the existence of the input tags. Structural tags are the ones which are used for laying out the HTML pages. They define the structure of the HTML pages. To track user activities, we have extended browser so that whenever any page is loaded various listeners are attached to the elements present in that page. Event listeners such as tab out , mouse click and double click are attached only to interactive elements. Appropriate semantics of the user interacted elements are extracted out in two steps i.e. semantics extraction and aggregation.

Fig. 1. HTML Components: Interactive, Decorative and Structural 2.1

Semantic Extraction

Semantics or meta-data of user activities can be determined from the webpage decorator elements itself. We aim to find the correct decorator tag by parsing the structural tags for the interactive component. Layout design on various webpages was studied. We found that the placing of the interactive component and the decorator occurs in few defined formats. Many visual forms were studied to find different ways a decorator element can be placed around an interactive element. Fig. 2 lists various visual forms where a decorator element is placed around interactive elements. If there is more than one way of decorator element being placed around an interactive element, then relevant decorator element is selected depending on its most occurring position. To find the correct decorator element for the interactive element, DOM tree is parsed. Parsing the DOM tree to obtain the decorator element had lot of complexities, as structural elements used to place the decorator and interactive element is imple-

3

mentation specific. A tree parsing algorithm was followed, wherein the starting point is interactive element.

Fig. 2. Few samples of decorative and interactive elements and their relative placement

Let E: Any DOM element Let I: Interactive element Let D: Decorative element isDecorator(E):returns true , if element E is decorative otherwise return false getPrevSibling(E):returns previous sibling of element E getNextSibling(E):returns next sibling of element E getParent(E): returns the parent element of E in the DOM tree getLastChild(E): returns the last child element of E in the DOM tree isRegular(E): returns false if element E is checkbox, radio etc. i.e. those interactive components whose probability of finding the decorative element to the right is high else return true getDecorativeAttributeValue(E): returns the decorative attribute value of the element E. Decorative attribute value are the elements such as alt, HTML inner text etc. getType(E): returns the type of the element E. HTML element types can be radio, checkbox, submit etc. getNextChild(E):returns the child element one by one starting from first child element, every time when this function is called If element I is regular, then its decorative element is present before its location in the DOM tree and vice versa. Fig. 3 shows algorithm to get the previous decorator element of I. To find decorative element if element I is not regular replace getPreviousSibling by getNextSibling in algorithm getPreviousDecoratorElement and getNextChild by getLastChild at line 4 in getDecoratorInSequence . After predicting the decorator element, next step Aggregation is performed to segregate various actions and group similar elements and find the final intent. 4

Fig. 3. Algorithm to find previous decorator element for element I

2.2

Aggregation

To gain insight into user activities, aggregation of the activity logs is performed. Thus, it is important to establish the relation among the data. Relation among the decorative elements can bind or categorize these elements in a group. To achieve this, we have extracted the parent decorator element of the decorator element. After extracting the parent elements, a tree is created to get the common parent decorator. Thus, the parent at the top links the data. Logged data can create more than one tree if user worked in more than one task on the same webpage. Fig. 4 shows the normal “Sign Up” page and the result tree which is created to establish relation among various decorator elements. As shown in Fig. 4, the decorator element for interactive elements textbox1, text-box2, textbox3, textbox4, combobox1, male, female are email id, password, re-enter password , secret question , city , gender respectively. Data is extracted out in the semantic extraction step. Then the association between this data is identified again by parsing the DOM tree and then association tree is created.

Fig. 4. Aggregated Data

5

3

Experimental Results

We conducted few experiments to find out the accuracy of the above mentioned algorithm and results are presented below. 3.1

Semantic Extraction Algorithm Results

Semantic Extraction algorithm was executed on various interactive elements to find accuracy in each category. Interactive elements were categorized as input, link, image, span, select and button. Algorithm was executed on more than 300 elements in each category spanning around more than 50 websites from various domains like banking, finance, travel, technology etc. having rich content. Fig. 5 showcases the accuracy in finding out the decorator element for various HTML tags. Consider input, link, span (also consists of header, font, strong, td, tr etc. tags), select and button tags accuracy was above 85 percent. Decorator element for an image achieved 76 percent as most of the times, the decorator attribute for an image is on the image itself and to find the text from the image is not the part of the algorithm. Finding the correct decorator element has many challenges, as placing the structural element depends completely on the user’s creativity. After analyzing the results we found that if the algorithm be further modified with the incremental learning mechanism, and UI guidelines are followed by websites, more accurate results can be achieved.

Fig. 5. Experimental Data showing percentage of accuracy achieved for various HTML interactive elements

3.2

Inferring User Actions

Fig. 6 shows an instance of user activity on a corporate “Facebook” account which is

handled by multiple users. User activities were tracked for a day. Activity Score is no. of times user visits certain links or items. Similarly horizontal axis represents the labels of the interactive elements. It can be easily inferred that Password /Change 6

Password link is used many times by User2. Now, for an enterprise account this can be a malicious activity where that user is not allowed to change the password. It can also be inferred that User1 is interested in GIS related groups.

Fig. 6. Tracked user activities

4

Limitations

Since the repository of HTML elements is too big, so it is difficult to cover all the elements .We have focused on the most occurring HTML elements. The algorithm can become more robust by adding different elements. The experimental result in Fig. 5 shows that on an average, more than 87% percent of the time, we have found the correct decorator label for the element. Our algorithm is based on standard web practices which are being followed. If web pages are coded and designed poorly then it may happen that the results come up with wrong labels as every data is retrieved from DOM tree itself. We have tried to preserve the privacy by not tracking labels specified by user.

5

Conclusion and Future Work

This paper discusses various methods developed to track user interaction and its various applications. It explains how user interaction logging can be enhanced further to find the user intention. It then explains the method of semantic extraction of the user interacted components and aggregation to find the user intention. The method is demonstrated through various experiments and accuracy and useful charts are shown. Our future work is dedicated in finding insights which involves the process flow spanning multiple web pages. 7

6

References

1. Atterer, Richard, Monika Wnuk, Albrecht Schmidt.: Knowing the user's every move: user activity tracking for website usability evaluation and implicit interaction. In: Proceedings of the 15th international conference on World Wide Web, pp. 203-212. ACM (2006 ) 2. Apaolaza, Aitor, Simon Harper, Caroline Jay.: Understanding users in the wild. In: Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility, p. 13. ACM (2013 ) 3. Apaolaza, Aitor.: Identifying emergent behaviours from longitudinal web use. In: Proceedings of the adjunct publication of the 26th annual ACM symposium on User interface software and technology, pp. 53-56. ACM (2013) 4. Plumbaum, Till, Tino Stelter, Alexander Korth.: Semantic web usage mining: Using semantics to understand user intentions. In: User Modeling, Adaptation, and Personalization, pp. 391-396. Springer Berlin Heidelberg (2009) 5. Paganelli, Laila, Fabio Paternò.: Intelligent analysis of user interactions with web applications. In: Proceedings of the 7th international conference on Intelligent user interfaces, pp. 111-118. ACM (2002) 6. Hong, Jason I., James A. Landay.: WebQuilt: a framework for capturing and visualizing the web experience. In: Proceedings of the 10th international conference on World Wide Web, pp. 717-724. ACM (2001) 7. Lieberman, Henry.: Letizia: An agent that assists web browsing. IJCAI (1), 924-929 (1995) 8. Chen, Zheng, Fan Lin, Huan Liu, Yin Liu, Wei-Ying Ma, Liu Wenyin.: User intention modeling in web applications using data mining. World Wide Web 5, no. 3, 181-19 (2002) 9. Bigham, Jeffrey P., Craig M. Prince, Richard E. Ladner.: Web Anywhere: a screen reader on-the-go. In: Proceedings of the 2008 international cross-disciplinary conference on Web accessibility (W4A), pp. 73-82. ACM (2008) 10. Atterer, Richard, Albrecht Schmidt.: Tracking the interaction of users with AJAX applications for usability testing. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 1347-1350. ACM (2007) 11. Guo, Qi, Eugene Agichtein.: Exploring mouse movements for inferring query intent. In: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, pp. 707-708. ACM (2008) 12. Kammenhuber, Nils, Julia Luxenburger, Anja Feldmann, and Gerhard Weikum.: Web search clickstreams. In: Proceedings of the 6th ACM SIGCOMM conference on Internet measurement, pp. 245-250. ACM (2006)

8