RATING VISUAL CONTENTS OF WEBSITE USING BRAIN COMPUTER INTERFACE Muhammad Saif ul Islam
Humera Farooq
Department of Computer Science Bahria University 13- National Stadium Road, Karachi
[email protected]
Department of Computer Science Bahria University 13- National Stadium Road, Karachi
Abstract - Websites are part of our everyday life and are used to exchange and convey information between user communities. Despite website proliferation, assessment of site quality remains a challenging area of research. Scientific literature identifies several aspects, or criteria, of quality, which are often aggregated to content, structure, design, navigation and multimedia but they cannot determine what the user is thinking about the content while browsing. Previous researches supported the notion that the internet users do not read the websites. Only thing that attract them is the visual contents. This project aims to develop an EEG (Electroencephalogram) based application to rate interest in presented visual content. The idea is to analyze the feeling difference of a human mind as compare to the subjects’ verbal interest. Some popular websites are selected for this purpose having similarities in the main contents. Brain signals collected through EEG device are send to an application for processing. It compares results of all the websites observed by the user and produces a rating which is displayed on the user interface of the application as a feed back to the user. Knowledge from the results can be basically used when designing websites and marketing campaigns online.
Index Terms – electroencephalograph, rating, website.
I.
INTRODUCTION
The technology that powers modern computer systems is advancing fast, converting ideas that were thought to be science fiction into reality. Imagine if one could control a computer or machinery just by using thoughts. A. BCI (Brain Computer Interface) The term ‘BCI’ was first used by Jacques Vidal in his paper which was the first ever research output on this topic [1]. Brain computer interface is a technology that allows humans to control and communicate with electronic devices using their very own brain. The control can be done for a robot, a prosthetic limb or even the cursor of your computer system. A BCI uses an interface, typically a wearable device with sensors as shown in figure 1, allows us to capture brain signals, analyze those signals and use them for different purposes.
[email protected] BCIs can be invasive, partially-invasive or non-invasive based on the placement of electrical sensors on the brain. B. EEG (Electroencephalogram) Electroencephalogram is a non-invasive procedure that detects electrical activity of brain using small, flat metal disc called electrodes attached to the scalp. Bio-electrical signals from the scalp are recorded and the wave forms they produce reflects the cortical electrical activity. Recently, BCIs used in lab are too complex and expensive to be widely used. Several companies have developed their easy-wearing and cheap BCIs [2]. In this proposed project we will choose one called NeuroSky MindWave as shown in Figure 1. C. Brain Waves Human brain consists of billions of neurons connected with each other communication via message passing in the form of electrical signals. These electrical signals can be collected from the top of scalp and known as brain waves. Different type of waves are produced for different state of mind. Alpha waves are produced in posterior region of brain and they represents relaxed state of mind. Beta waves are most prominent in frontal regions and it represent attention and concentration. In this paper, user’s interest level while browsing a website is determined through the level of beta activity of brain.
Figure 1: NeuroSky Mindwave EEG headset
II. BACKGROUND AND RELATED WORK History of BCI begins with the discovery by Hans Berger of the electrical activity of human brain and development of electroencephalography (EEG) in 1924 [3]. Different scientists put their services in this field by doing experiments by establishing modest BCI sensors inside rats, mice, monkeys, and humans. The research field on Brain-Computer-Interface (BCI) offers a new way to build an interactive system which can transform human brainwaves into control signals for electronic devices. There are several studies by means of the signals of brain activity to control machines. For example, BCI systems were constructed to control electric wheelchair to benefit paralyzed patients [4-5]. Many efforts have been carried out to assist disabled people in pick and place tasks using low cost BCI based prosthetic arm controls [6-7]. EEG based BCI were developed and tested for users to regulate a cursor on a computer display [8]. Recently, with the development of BCI, several research institutions and organizations have developed some Android based 3D racing game that are interacted with BCI to enrich user's experience [9]. An android based speller is also a mentionable progress in the field of BCI [10]. Presently one developing a website has no idea which kind of content of a website is interesting to user or which website among a list of websites is of user’s interest. Some manual methods using eye-gaze to determine time spent on different sections of a webpage to gauge user interest have been used previously but they are not reliable indicator of actual user’s interest [11-12]. Perception created in viewers mind by seeing the visual contents is important to be known. Recently in 2016, a research has been carried out that was based on investigating brain activity during browsing websites [13]. Both eye-gauze and EEG were collectively used to detect electrical activity of brain. Based on their results it is arguable the users do not perform active thinking when browsing through pages, but during activities that require attention – decision making situations, increased beta activity. III. DESIGN AND METHODOLOGY In the presented work we aim to assess the subjects rating for different visual contents. For this purpose, four websites are selected. Healthy subjects are invited for rating the selected website. The websites has been rate on 5 features i.e. colour, font, multimedia, picture and video. Later on a rating has been displayed on the GUI of the application as a feed back to the user. We have used NeuroSky headset with 8 channels for this purpose.
In order to retrieve the brain signals and convert them into useful data the proposed approach is broadly divided into four main parts i.e. Signal acquisition, signal processing, content rating and feedback modules as shown in the Figure 2.
Figure 2: Architecture of the Proposed Approach
A. SIGNAL ACQUISITION In signal acquisition module, wireless EEG headset was placed on subject’s head. Some selected websites, whose detail is discussed in experiment section, were presented to the subject and he/she had to browse through them one after the other for a limited interval of time. In this process, low level filtering are performed and passes the data on to be interpreted at the signal processing module. During this process brain signals from the headset were collected and fed to an application. B. SIGNAL PROCESSING In this module signals extracted from subject’s brain are fed into a custom made java application for processing. Brains waves are analysed on the basic of the activity of beta waves that represent active condition of human mind. A rating (0-10) is produced for each website visited. C. FILTERING AND FEATURE EXTRACTION The acquired signals received by signal processing module will be filter out with abrupt changes. After the filtration process, the process of feature extraction begins. It detects whether a particular feature or a potential is present in the data. Thy required features for the presented study are to visual contents based on Colour, Font, Text and Look and Feel
of the different websites. D. CONTENT RATING AND FEEDBACK
category by each subject. Tourism category was not selected by any of the four subjects.
All generated ratings are compared and website with the highest rating i.e. website of maximum user interest is selected and presented to the user as a feedback or output. IV. EXPERIMENTAL SETUP AND RESULTS A. SUBJECTS In order to perform the experiments four healthy subjects (3 males and a female) are invited. All of these subjects are university student of age between 22 and 25. The background and goals of experiment were explained to each selected subject and procedure was started with their consent. B. WEBSITES SELECTION The presented research deals with the human aesthetic sense; how people see and interpret beauty of the visual contents. Websites for this research are selected keeping human aesthetics in mind. Websites containing excess multimedia content were shortlisted. They were then grouped into different categories like Food, E-Commerce, Social, Gaming. Each category contained four websites.
Figure 3: User Feedback
TABLE I.
SELECTED CATEGORY BY DIFFERENT SUBJECTS
SUBJECTS
Selected Category
Subject 1
Food
Subject 2
Social
Subject 3
Online Shopping
Subject 4
Food
C. RATING CRITERIA Brain activity is represented by the type of brain waves generated. Alpha waves represent the relaxed state of mind. Their frequency ranges from 9-14 cycles per second. Beta waves range from 15-40 cycles per second and represent active/working mind state. Rating was based on the attention level derived from beta waves and was scaled from 1 up to 10. D. EXPERIMENTAL RESULTS Initially the subject had to select one of the available categories and then view all four websites one by one in that category for a limited assigned period of time while wearing the EEG based headset. Subject had to rate all of them manually and during this procedure the designed application calculates the rating based on their brain activity. After competing this procedure for all websites, a comparison of manual and predicted rating is done by the application and the website which subject liked the most according the brain data is presented to the user as a feedback as shown in the Figure 3. Subjects were shown 4 categories (Food, Social, Online shopping, Tourism) out of which they had to select one. Each category had 4 websites which were shown to the subjects for 30 seconds each. They had to rate each website manually in scale of 1-10. The designed system will give a predicted value rating based on their brain data. Table 1 shows the selected
Predicted and manual rating of all four subjects are shown in Table II and also represented graphically in Figures [4-7]. Each subject performed 05 sessions for evaluating the visual contents. The graphical results show a prominent difference between like and dislike between different subjects. For example, Figure 4 shows that subject-1 liked website#1 the most as compared to website#3 & 4 with the lowest ratings. However, Figure 5 shows highest that subject-2 liked website#4 the most as compared to website#2 with lowest ratings. Similarly Subject 3 like website 04 as compare to other websites as shown in Figure 6 and subject 4 like website 02 in all others as shown in Figure 7. For some cases it was also found that the manual rating and predicted rating all same or almost same i.e. subject 1 and 3 for website 1. The overall rating of all subjects was not more than 7 on the scale of 1-10.
TABLE II.
MANUAL AND PREDICTED RATING FOR EACH SUBJECT
Websites Subjects
Subject 1
Subject 2
Subject 3
Subject 4
Attributes
Predicted Rating Manual Rating Predicted Rating Manual Rating Predicted Rating Manual Rating Predicted Rating Manual Rating
1
2
3
4
5
4
4
4
5
3
4
6
4
2
4
4
5
4
7
4
4
3
4
6
4
3
5
5
4
5
5
4
6
7
6
5
Figure 6: Comparison of Manual and Predicted rating for Subject 3
Figure 7: Comparison of Manual and Predicted rating for Subject 4
V. DISCUSSION
Figure 4: Comparison of Manual and Predicted rating for Subject 1
After the experiments, subjects’ manual feedback/rating and the rating produced by the application on the basis of user’s brain activity were analyzed, compared and graphically represented. Snapshots of comparison for all subjects are shown in Figure [4-7]. It shows a comparison of manual and predicted rating against a scale of 1-10. ‘1’ represents the least interest while ‘10’ the most. Figure 4 shows the ratings generated by the application for all four websites for subject 4. Comparing it with Figure 7, the graphical representation and comparison of users’ interest among four websites, it can be observed that the second websites (i.e. foodly) has the highest rating approaching 6. The difference between the manual and predicted rating is 1.2 exactly.
Figure 5: Comparison of Manual and Predicted rating for Subject 2
In order to observe the results more closely on the basis of like or dislike of visual contents, generated ratings were transformed into two categories. Rating below 5 were categorized as “disliked” and ratings with 5 and above were categorized as “liked”. Table III shows the comparison of manual and predicted rating on the basis of these two
cetegories. However Table IV shows the like and dislike based on each subject. TABLE III. RESULTS
RESULT MATRIX
Predicted Liked
Predicted Disliked
images, fonts, videos etc. Individual ratings will be beneficial in learning about human aesthetic sense much better. In the experiments the subject was bond to browse the website for particular time duration. In future it can be modified and the subject activity can be monitored for an extended period of time. ACKNOWLEDGMENT
Manual Liked
75%
12.5%
Manual Disliked
6.25%
6.25%
Attributes of result matrix are explained as follows: Manual Liked: Site rated 5 or above by the user. Manual Disliked: Site rated below 5 by the user. Predicted Liked: Site rated 5 or above by the application. Predicted Disliked: Site rated below 5 by the application.
This research is being conducted and supervised under the ‘Intelligent Systems and Robotics’ research group at Computer Science (CS) Department, Bahria University, Karachi, Pakistan. REFERENCES [1] Jacques Vidal, “Toward Direct Brain-Computer Communication”, Annual Review of Biophysics and Bioengineering 2: 157–80 [2] https://store.neurosky.com/pages/mindwave [3] https://en.wikipedia.org/wiki/Hans_Berger [4] S-. Y. Cho, A. P. Vinod, and K. W. E. Cheng, "Towards a Brain Computer Interface Based Control for Next Generation Electric Wheelchairs ", Int. Can! On Power Electronics Systems and Applications, pp. 1-5, 2009 [5]
75% websites that were liked by the users manually were also marked as “liked” by the application on the basis of brain data. 6.25% websites that were marked as “disliked” by the users were marked as “disliked” by the application too. According to the results a difference of 18.75% was reflected comparing the users’ manual ratings and the ones generated from their brain activity.
[6]
[7]
[8]
Table IV: Subjects Predicted and Manual Like and Unlike
K. Tanaka, K. Matsunaga, and H. O. Wang ,"Electroencephalogram Based Control of an Electric Wheelchair ", IEEE Trans. on Robotics, vol. 21,no. 4,pp. 762-766,2005 Jetsada Arnil et al. “Bci-based assistive robot arm”. In: Medical Information and Communication Technology (ISMICT), 2013 7th International Symposium on. IEEE. 2013, pp. 208–212. Taha Beyrouthy et al. “EEG Mind controlled Smart Prosthetic Arm”. In: Emerging Technologies and Innovative Business Practices for the Transformation of Societies (EmergiTech), IEEE International Conference on. IEEE. 2016, pp. 404–409. L. J. Trejo ; R. Rosipal ; B. Matthews, “Brain-computer interfaces for 1D and 2-D cursor control: designs using volitional control of the EEG spectrum or steady-state visual evoked potentials”, IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol.14 , no. 2, pp. 225-229, 2006 Guobin Wu, Zheng Xie ; Xin'an Wang, “Development of a mindcontrolled Android racing game using a brain computer interface (BCI)”, 2014 4th IEEE International Conference on Information Science and Technology, pp. 652 – 655, 2014
Subject 1: Manual: like 3, unlike 1 Predicted: like 4, unlike 0
Subject 2: Manual: like 4, unlike 0 Predicted: like 3, unlike 1
[9]
Subject 3: Manual: like 3, unlike 1 Predicted: like 3, unlike 1
Subject 4: Manual: like 4, unlike 0 Predicted: like 4, unlike 0
[10] Ananya Dutta, Sanjiv Sambandan, “Android based EEG speller for the disabled”, Electrical, Electronics, Signals, Communication and Optimization (EESCO), International Conference, pp. 1-3, 2015 [11] D. Beymer, D. M. Russell. WebGazeAnalyzer: a system for capturing and analyzing web reading behavior using eye gaze. Proc. CHI EA '05 CHI '05 Extended Abstracts on Human Factors in Computing Systems. [12] R. Atterer, M. Wnuk, A. Schmidt. Knowing the user's every move: user activity tracking for website usability evaluation and implicit interaction. Proc. WWW '06 Proceedings of the 15th international conference on World Wide Web. [13] Terezia Kvasnicova, Iveta Kremenova, Branko Babusiak, “Investigation of brain activity during browsing websites”, ELEKTRO, 2016
VI. CONCLUSION AND FUTURE WORK The proposed research work was based on determining user interest level through an EEG based application and comparing it to the subjects’ verbal interest. On the basis of a subject’s brain activity; responsive browsers can be developing which can automatically mark the website as favourite based on user interest level. The presented research could be extended by rating the contents on the basis of individual parameters like colour,
