An Investigation of Using Multimodal Metaphors in ...

1

An Investigation of Using Multimodal Metaphors in Browsing Large Amount of E-mail Data: An Empirical Approach S. Alharbi, D. Rigas Department of Computing, University of Bradford, Bradford BD7 1DP, UK

volume users can get more than hundred [6]. By leaving these Abstract— This paper reviews the previous research and studies that have been performed in order to enhance the usability of e-mail clients. It proposes a development of an e-mail tool that organises e-mail inboxes by combining the visualisation techniques with different types of multimodalities such as Earcons, auditory icons and speech sounds. This tool will show the advantages of presenting the basic properties of e-mail messages using sounds and other modalities as well as using them as main factor for visualisation. it tool will be a basis for an experimental programme. Index Terms— Auditory Icons, Earcons, e-mail, multimodal, usability, visualisation

e-mail messages in the e-mail inboxes they will be very difficult to use and they will be cluttered. The following section (II) shows the previous works and some of the studies that tried to enhance the usability of e-mail clients. Section III shows the limitations of the previous research and proposes a multimodal e-mail tool that will be a basis for the research experiments. The last section (IV) of this paper presents some conclusions of the literature survey as well as the future works that can be employed in e-mail

I. INTRODUCTION

clients.

E

-mail is the most widely used application and it has become the application that users spend much of their

time using it [1, 2]. E-mail clients are used everyday in our

II. Literature review

lives and they can be the reason of buying personal

Although e-mail applications are widely used and many

computers. Therefore, Duchenaut and Bellotti called it habitat

studies have been performed but it has not been significantly

[3]. E-mail is designed to enable none-face to face

treated and researched in term of usability aspects. Although

communication but because it is widely used nowadays it is

most of the e-mail clients use the textual table view to presents

being used for additional functions which e-mail is not

the

designed for, therefore it is called overloaded [4]. Many

browsing large amount of e-mail messages. For example users

studies showed that the number of e-mail messages rapidly

will take long time to locate a specific e-mail message.

grows. For example, the University of California stated that

Filtering and using folders has been proposed and suggested

about 31 billion e-mail messages have been sent in 2002 and

to be the way of organising the e-mail inboxes which will

this number might be doubled in 2006 [5]. In other study, it

make the navigation easier [3]. Many studies showed that

has been shown that the average user gets around 49 e-mail

using folders and filtering features has many problems [2, 4,

messages a day while high volume users can get more than

6, 7]. For example, Duchenaut stated that folders might be

hundred [1]. In another study, it has been shown that the

long nesting and might be redundant or no longer used [3].

average user gets around 49 e-mail messages a day while high

Yiu stated that filing is time consuming and it could be

e-mail messages, different problems are exist when

cognitively intensive [7]. Auto classification and automatic

2 folders creation has been proposed to be a solution, but it has proven error prone [1]. Apart of using filtering and auto

B. The use of sound One of the limitations of using graphics extensively is making

classification of e-mail messages many studies have been

the users visually overloaded. To overcome this problem,

performed in order to provide an alternative approach to the

sound can be used to convey and present information in the

textual table approach. Most of these studies used different

user interface [10-12]. Using sound in user interface enables

visualization techniques with the information hiding concept

the visually impaired users using the computer [13]. Many

in order to present the e-mail messages and data. Other studies

studies employed sounds (speech- none-speech) in browsing

used the auditory stimuli such as speech and non-speech

e-mail data in order to enhance e-mail client’s usability. The

sounds to presents the e-mail attributes. The following

basic e-mail properties such as the category of the e-mail

subsections show some of these studies.

messages can be presented using short sounds [14]. Rigas and

A. The use of graphics Information visualisation has been proposed by many researchers as solution for enhancing the usability of e-mail clients. They used the hidden information in the e-mail messages such as messages threads, social relationships, the time of receiving messages and messages contents. In the thread technique all the messages that have relationships between the senders to the recipients or the replay relationships are shown as connected tree [1, 6, 8]. Although messages thread has been used by many researchers, the important attributes of e-mail messages, such as the existence of attachments and the priority of e-mail messages, are hidden which may affect the understanding of e-mail messages. The time of receiving the e-mail messages has been used as the factor for visualising the e-mail messages [5, 7]. Yiu, et al have organised E-mail messages on Y, X axis where time was presented along the X-axis and the senders on y-axis [7]. E-mail messages were displayed as dots and they were

colleagues has performed a serious of experiments to investigate the use of various types of sound when browsing e-mail data [15-17]. They pointed out that using a combination of different types of sounds (speech, non-speech) for conveying e-mail data instead of information hiding could reduce the graphical complexity [18]. They proved that auditory stimuli such as Earcons, auditory icons and speech can be used to communicate the hidden information in e-mail data [17]. A list of guidelines for designing multimedia e-mail tools have been derived from these experiments [19]. all the experiments and e-mail tools were using auditory stimuli without considering graphical browsing and information visualisation therefore Rigas, et al developed an e-mail tool which provides audio-visual browsing of e-mail data [20]. Although the graphical browsing of e-mail data has been added in this tool it has not enhanced the organisation of e-mail inbox and the performance of the users has not been considered.

interactive for example, user can click on an e-mail message to read it. Viegas and colleagues developed a tool called “TheMail” for visualising e-mail archive but, it used different

III. THE EXPERIMENTAL TOOL

visualisation techniques which relied on the content of e-mail

As it is shown above sound has been employed in order to

messages [9]. It portrayed the relationships using the

reduce the graphical complexity without taking into account

interaction histories in e-mail archives. Many studies

the organisation of e-mail inbox. On the other hand,

combined two or more techniques in order to reduce the

Information visualisation has been used in order to organise

complexity of e-mail inboxes [2, 8]. As show above many

the e-mail inbox, but users were not able to retrieve the

efforts have been performed to enhance the e-mail clients

properties of an e-mail message unless they open it. Therefore,

using graphics but most of them have not tested with users.

an e-mail tool which organises the clutter of e-mail inbox and provides the properties of e-mail messages to the users easily

3 without overload them visually. This tool can be built by

[5]

combining the visual channel with different modalities. The

[6]

results of the experiments in [21, 22] motivated us to start the

[7]

development of e-mail tool that combines the graphical clues

[8]

with different auditory stimuli such as Earcons and auditory icons and other modalities. This tool is under development

[9]

and it has not been finished yet. It initially presents three visualisation techniques, all of them based on the basic e-mail

[10]

properties such as the sender’s e-mail address because of [11]

using the threads may lead to a graphical complexity. The other properties are presented using the different modalities such as the audio channel instead of using the information hiding concept. This tool will be a basis for our experimental

[12] [13]

[14]

programme. [15]

IV. CONCLUSION

[16]

By reviewing the pervious research and studies that have been performed in browsing e-mail data it has been inferred that

[17]

using graphics alone is not sufficient for enhancing e-mail usability. Information hiding is not good solution for reducing

[18]

the graphical complexity because it might hide critical information. This information can be presented using other modality such as the auditory channel. The combination of

[19]

different modalities can enhance the usability of any information system. This paper proposed an experimental tool

[20]

that visualises the e-mail messages based on the basic e-mail properties and presents the other using different modalities.

[21]

The anticipated benefits of this tool are expected to be significant and effective because the e-mail applications are widely used. REFERENCES [1]

[2]

[3] [4]

Rohall, S.L., Redesigning Email for the 21 stCentury Workshop Position Paper. CSCW 2002 Workshop: Redesigning Email for the 21st Century. New Orleans, LA, 2002. Sudarsky, S. and R. Hjelsvold, Visualizing electronic mail. Information Visualisation, 2002. Proceedings. Sixth International Conference on, 2002: p. 3-9. Duchenaut, N. and V. Bellotti, Email as habitat. Interactions, 2001. 8(5): p. 30-38. Whittaker, S. and C. Sidner, Email Overload: Exploring Personal Information Management of Email. 1996.

[22]

Frau, S., J.C. Roberts, and N. Boukhelifa, Dynamic Coordinated Email Visualization. WSCG05: p. 187–193. Rohall, S., et al., Email Visualizations to Aid Communications. IEEE Symposium on Information Visualization (InfoVis), 2001. Yiu, K., et al., A time-based interface for electronic mail and task management. Proc. HCI International'97, 1997. 2: p. 19-22. Venolia, G.D. and C. Neustaedter, Understanding sequence and reply relationships within email conversations: a mixed-model visualization. Proceedings of the conference on Human factors in computing systems, 2003: p. 361-368. Viegas, F.B., S. Golder, and J. Donath. Visualizing email content: Portraying relationships from conversational histories. 2006. Montreal, QC, Canada: Association for Computing Machinery, New York, NY 10036-5701, United States. Brewster, S.A., et al., The sonic enhancement of graphical buttons. Proceedings of Interact, 1995. 95: p. 43-48. Brewster, S.A., Using Non-Speech Sound to Overcome Information Overload. Displays, 1997. 17(3): p. 179-189. Brewster, S.A., Sonically-enhanced drag and drop. Proceedings of ICAD, 1998. 98: p. 1±4. Tannen, R.S., Breaking the Sound Barrier: Designing Auditory Displays for Global Usability. Proc of Human Factors & the Web, Basking Ridge, USA, June, 1998. 5. Hudson, S.E. and I. Smith, Electronic mail previews using nonspeech audio. Conference on Human Factors in Computing Systems, 1996: p. 237-238. Rigas, D.I. and D. Memery, Utilising audio-visual stimuli in interactive information systems: a two domain investigation on auditory metaphors. Information Technology: Coding and Computing, 2002. Proceedings. International Conference on, 2002: p. 190-195. Rigas, D., D. Memery, and K. Klearhou, A Multimedia E-mail Tool: An Empirical Case of using Auditory Metaphors to Communicate and Browse Data. Applied Informatics 2002, 2002: p. 199–203. Rigas and Memery, Experiments Using Combinations of Auditory Stimuli to Communicate E-Mail Data. Proceedings of the HCI International 2003, the 10th International Conference on HumanComputer Interaction, 2003. Rigas, D.I. and D. Memery, Multimedia e-mail data browsing: the synergistic use of various forms of auditory stimuli. Information Technology: Coding and Computing [Computers and Communications], 2003. Proceedings. ITCC 2003. International Conference on, 2003: p. 582-586. Rigas, D., Empirically Derived Multimedia Design Guidelines for Browsing Large Volumes. ANNALS of mathematics, computing & teleinformatics, 2003. 1: p. 77-83. Rigas, D., et al., Designing Information Systems with Audio-Visual Synergy: Empirical Results of Browsing E-Mail Data. Panhellenic Conference on Human-Computer Interaction: Advances on Human-Computer Interaction, 2001: p. 960-7620. Rigas, D. and A. Ciuffireda, Multi-Modal Web-Browsing An empirical Approach To Improve the Browsing Process of Internet Retrieved Results. SIGMAP 2006 - INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING AND MULTIMEDIA APPLICATIONS, 2006: p. 296-276.

Rigas, D.I., J.L. Alty, and F.W. Long, Can music support interfaces to complex databases. EUROMICRO-97, New Frontiers of Information Technology, 1997: p. 78–84.