ANNALS OF MATHEMATICS, COMPUTING & TELEINFORMATICS, VOL 1, NO 1, 2003, PP 77-83
77
Empirically Derived Multimedia Design Guidelines for Browsing Large Volumes of E-Mail Data Dimitris Rigas Department of Computing, School of Informatics University of Bradford, Bradford BD7 1DP, UK
[email protected]
Abstract— This paper introduces multimedia design guidelines for browsing large volumes of e-mail data. These software design guidelines derived from two sets of experiments under a prototype version of a multimedia e-mail tool. The experiments tested various forms of audiovisual communication metaphors. For example, the auditory stimuli consisted of musical sounds (earcons), compositional sounds, environmental sounds (auditory icons), stereophony, pitch, synthesised and recorded speech as well as other sound effects. The first set of experiments involved the multimedia browsing of simple and complex sets of e-mail data. The second set of the experiments involved the communication of e-mail categories and other related e-mail information such as the presence or absence of an attachment, priority status and subject of an e-mail. The results of these two sets of experiments are described and linked with other experiments in the literature. The paper integrates and discusses all these experimental results, the incorporation of auditory stimuli and the suitability of different types of multimedia metaphors for the design of audio-visual browsing techniques of large volumes of e-mail data. The totality of the multimedia design appeared to offer a meaningful, easily understood and recognised interaction mechanism and therefore demonstrated a potential way to maximise the volume of information that could be communicated to users. The application of synthesised and recorded speech, earcons and auditory icons as well as sound effects are also discussed in the light of the experimental results with emphasis upon issues of synergy, synchronisation, consistency, structure, classification and presentation of different types of communication metaphors as an integral part of the software design process. Furthermore, the suitability and use of compositional sounds are also discussed. The paper concludes with a set of practical guidelines for software and multimedia designers and a design approach for this type of data browsing.
Index Terms— Human-Computer Interaction, Multimedia Metaphors, Design Guidelines, Audio, Sound, Auditory design
I. I NTRODUCTION Raphical browsing of e-mail data produces complex graphs or displays. Visual complexity increases as more e-mail messages are introduced. In order to reduce complexity, software designers filter the presented information in a way that the displays produced are easily read and comprehended. The filtered information is therefore either hidden from the user or shown in another instance of the visual display. Both approaches introduce disadvantages. Sometimes, it is difficult for designers to decide the amount and type of information to be hidden and often this decision is left to the users. When information is hidden from the user, there is always a risk that critical information may not be displayed and alternative visual instances must be prepared so as the user could browse serially. An alternative approach is to use auditory stimuli, as part of a multimedia application, to communicate the hidden information while the user is browsing visually. This paper describes the results of experiments that investigated the auditory aspects of multimedia e-mail data browsing and discusses some guidelines for the design of the auditory stimuli as part of a multimedia browsing technique.
G
II. I NTERACTIVE S YSTEMS AND THE U SE OF AUDIO Auditory metaphors include earcons and auditory icons. Short series of musical stimuli are often referred to as earcons. Categories of earcons include oneelement, compound, inherited and transformed earcons [1]. Multimedia designers can benefit from some guidelines that are available in the literature [2], [3] and [4]. Auditory stimuli of various types have been successfully applied in many different types of applications. For example, structured musical stimuli has been used to help visually impaired users to read and draw diagrams [3], [5] and [6], to use a word processor system
c 2003 AMCT/TEI Larissa ISSN 1109-9305 °
78
ANNALS OF MATHEMATICS, COMPUTING & TELEINFORMATICS, VOL 1, NO 1, 2003, PP 77-83
that incorporated speech and non-speech sound [7], to improve the usability of a graphics package for nonblind users [8], to communicate aspects of a database [9], [10] and program execution [11]. Other examples include the development of software engineering environments that have incorporated the use of sound. LogoMedia is one of those environments that use auditory stimuli to communicate events during program creation, execution and review [12]. In another environment, InfoSound, application events can be linked with auditory messages [13]. Teleconferencing systems have also been developed, called ‘audio windows’. These systems utilised spatial sound and gesture [14]. Auditory icons are environmental sounds. A user interface that uses environmental sounds is called SonicFinder. In this system, auditory icons were communicated in addition to the visual messages in a way that each auditory icon implied the information communicated [15], [16]. Every day sounds such as tearing paper, hammering, and walking were also investigated [17]. Other experiments involved the matching of sources of sounds and the sounds themselves [18]. Some confusion did occur when similar sounds were produced from different sources and it was observed that users interpreted the sources of the sounds and not the pitch or timbre. Environmental information was also communicated using sound sources. These included physical events such as a bottle breaks and smashes or bounces when dropped on the floor, events in space such as an ambulance’s siren approaching, dynamic changes such as liquid is poured into a glass, abnormal structures such as the sound of a car engine with a fault, invisible changes such as a hollow space in a wall can be identified by tapping blocks such as rhythm, timbre, register and dynamics [19]. III. E XPERIMENTAL P LATFORM A research platform with two prototype versions of multimedia e-mail tools was developed. The first prototype investigated the use of auditory metaphors to complement visual interaction during browsing of email data by using aural stimuli to communicate parts of an e-mail message. For example, the beginning of the message is announced using an auditory signal, the rhythm of the e-mail originator is then presented, another auditory message announces that the identities of the recipients will be presented, the rhythms of the recipients are presented, a further auditory signal announces that the identities of the users in the copied section of the e-mail will be presented, the identities of those users using rhythms are presented, and the auditory message finishes with an ending auditory signal. The name of an e-mail originator can be a
particular rhythm or a speech message. The prototype was implemented with Java 1.2 and JavaMail, Java Cryptography (JCE), Java Media Framework (JMF), Java Sound, Java Communications API, JavaBeans Activation Framework (JAF), and JavaHelp. Several developmental software tools were also used in the development of the prototype. Examples of these include Kawa Pro 4.01a IDE, WithClass 2000, JET native compiler, JavaDoc, and WaveLab Lite. A. Simple and Complex Browsing Two browsing techniques were developed using audio-visual stimuli in order to communicate simple sets of related e-mail data. Identities of users were communicated in the first browsing technique using sound and stereophony. The sound associated with each user can be recorded voice (e.g., name) or musical stimuli (e.g., rhythms). An initial auditory message announces the beginning of the auditory message. This was followed by the rhythm of the sender (e-mail originator), another auditory signal announced that the identities of the recipients will follow, the rhythms of the recipients were presented, another identifying auditory signal announced the identities of the users that the e-mail was copied, the rhythms of these users were presented and an ending auditory signal indicated the auditory message was completed. In the second browsing technique, the number of the recipients, the number of people the e-mail was copied as well as the number of e-mails received with the same subject were communicated using stereophony, synthesised timbre and rhythmic pitch sequences. An e-mail with three recipients (To field), four users in the copied section (Cc field) and the user had received five e-mails with the same subject would be communicated using rising pitch notes as three notes from the left speaker using piano, four notes from the right speaker using organ and five notes from the middle stereophonic position using cello. The auditory aspects of the two browsing techniques were evaluated with 19 users. All users were given a short training at the beginning (5 min) by presenting examples. Figure 1 shows the results of the experiments using the simple and complex browsing techniques. These results were not due to chance. The results of the simple browsing technique were significant for the information communicated in the “To field” (χ2 = 21.92, df=1, critical value 10.83, p
