October 30, 2003

October 30, 2003

Designing the user experience of Web as receivers of mass communication A transdisciplinary Ph.D. research project within VUBIS

Prof. Dr. Jan Kleinnijenhuis (FSW) Dr. Gerrit C. van der Veer (FEW) Dr. Elly A. Konijn (FSW) Dr. Johan F. Hoorn (FEW)

1

Table of Contents 1

Project Information 1.1 Project title 1.2 Principal Investigators 1.3 Project Duration 1.4 Project Type

3 3 3 3 3

2

Classification

3

3

Composition of the Research Team 3.1 Promotores 3.2 Past Performance of Ph.Ds 3.3 Candidate

4 4 4 5

4

Research Schools

5

5

Description of Work 5.1 Abstract 5.2 Introduction 5.3 Research Questions 5.4 Method 5.4.1 Participants 5.4.2 Stimuli 5.4.3 Procedure 5.4.4 Measurements 5.5 Expected results and merits 5.6 Future Research: Towards True Adaptability 5.7 Work Plan

6 6 6 9 9 9 10 10 12 12 13 13

6

Relevance to Academia and Society 6.1 Relevance to Social Sciences 6.2 Relevance to Computer and Information Sciences 6.3 Societal relevance

15 15 15 16

7

Background of Related Research 7.1 DUTCH and GTA 7.2 Content Analysis 7.3 PEFiC: Agents as Fictional Characters 7.4 Reality Perception 7.5 Intelligent Multimedia @ VU 7.5.1 WASP 7.5.2 RIF

17 17 18 19 22 23 24 25

8

References

26

2

1

Project Information

1.1

Project title Designing the user experience of Web as receivers of mass communication

1.2

Principal Investigators Prof. Dr. Jan Kleinnijenhuis (FSW) E-mail: [email protected]

Dr. Elly A. Konijn (FSW) E-mail: [email protected]

Dr. Gerrit van der Veer (FEW) E-mail: [email protected]

Dr. Johan F. Hoorn (FEW) E-mail: [email protected]

1.3

Project Duration Four years, starting from January 1, 2004.

1.4

Project Type

One Ph.D. student, with supervision shared by FEW and FSW. Note that the funding for this project comes from the currently available VUBIS funding. VUBIS stands for the VU Research Center for Business Information Sciences. Being one of the new initiatives launched as part of the “VU-ster,” VUBIS is a collaboration between the faculties FEW, FEWEB, and FSW, particularly concerned with the strategic topic of informatization and digitization in society. For information on the VUBIS research aims, approach, and interfaculty collaborations, we refer to the baseline document of the VUBIS initiative (final version, June 20, 2002).

2

Classification On the basis of the report of the Verkenningscommissie Informatica (1996): 4.5. User interface systems (generic applications, integration of diverse media, design guidelines and patterns) 2.6. Tele-applications (WWW for mass communication) 3.2. Specification methods (requirements analysis for user experience) From the NOAG-i (NWO National Research Agenda for Informatics 2001-2005): 13200 30700 50503 53506 54005 54201

Multidisciplinary Sciences Interaction Computer User Mass Media Persuasion Media Studies Study of Social Sciences and Computers

3

3

Composition of the Research Team

3.1

Promotores Prof. Dr. Jan Kleinnijenhuis (FSW) Faculty of Social Sciences Department of Management and Communication Studies Dr. Gerrit C. van der Veer (FEW) Faculty of Sciences Section Information Management and Software Engineering Co-promotores Dr. Elly A. Konijn (FSW) Faculty of Social Sciences Department of Management and Communication Studies Dr. Johan F. Hoorn (FEW) Faculty of Sciences Section Information Management and Software Engineering

3.2

Past Performance of Ph.Ds Jan Kleinnijenhuis Meijer, M. M. (1999-2004). The effects of media coverage on the reputation of organizations and occupational groups. Dept. Communication Science, Vrije Universiteit. Van Noije, L. J. J. (2002-2006). Europe and the relocation of agenda setting power to the news media. Dept. Communication Science, Vrije Universiteit. Van Noort, G. (2003-2007). Trust in Web-based marketing communications. Dept. Communication Science, Vrije Universiteit. Gerrit C. van der Veer Vertegaal, R. (1998). Look who’s talking to whom – Mediating joint attention in multiparty communication and collaboration. Twente University. De Vries, A. P. (1999). Content and multimedia database management systems. Twente University. De Haan, G. (2000). ETAG – A formal model of competence knowledge for user interface design. Information Management and Software Engineering, Vrije Universiteit. Van Welie, M. (2001). Task-based user interface design. Information Management and Software Engineering, Vrije Universiteit. Van Engers, T. M. (2001). Knowledge management – The role of mental models in business systems design. Information Management and Software Engineering, Vrije Universiteit. Benz, H. (2003). Casual multimedia process annotations CoMPAs. Twente University. Chisăliţă, C. M. (2001-2005). Organizational sub-cultures in interactive system design. Information Management and Software Engineering, Vrije Universiteit. Hoorn, J. F. (2001-2005). Integrating design of business processes and task analysis. Information Management and Software Engineering, Vrije Universiteit. Kok, E. (2002-2006). Design of an information system at the Dutch police force. Information Management and Software Engineering, Vrije Universiteit.

4

Elly A. Konijn Wildschut, L. (2003). Moved by dance. The experience of theater performances by children. Dept. Media and Re/presentation, Utrecht University. De Groot, R. (2003-2007). Interpersonal communication with people via display screens. Dept. Communication Science, Vrije Universiteit.

3.3

Candidate

Drs. or MSc either with a degree in Communication Science or Computer Science – or in a closely related area, such as Cognitive Psychology. The candidate should have a broad interest in transdisciplinary work, both theoretically and empirically. Areas of interest cover, for instance, human information processing, human computer interaction, software engineering, artificial intelligence, and mediated interpersonal communication. The candidate is preferably skilled in laboratory experimentation and multivariate statistics. In addition, skills in or affinity with programming 3-D environments and multimedia are highly recommended. A good command of English both in writing and oral presentation is mandatory. The candidate will be selected on the base of an advertisement in de Volkskrant and announcements on the Web sites of the research schools SIKS and NESCoR.

4

Research Schools

SIKS1 is the Netherlands national Graduate Research School for Information and Knowledge Systems. Founded in 1996 by researchers in the fields of Artificial Intelligence, Databases & Information Systems and Software Engineering. SIKS received its accreditation by KNAW in 1998. SIKS is an interuniversity graduate research school that comprises 10 universities. Currently, over 230 researchers are active, including about 120 Ph.D. students. The Vrije Universiteit Amsterdam is SIKS’ coordinating administrative university (‘penvoerder’). The office of SIKS is located at Utrecht University. SIKS has been re-accredited in 2003 by KNAW for the period until 2009. The mission of SIKS is (1) to perform high-level fundamental and applied research in the field of information and computing science, more particularly in the field of information and knowledge systems; (2) to organize a high-quality four-year educational program for its Ph.D. students, employed at 10 different Universities in the Netherlands or at leading companies in the field of ICT; (3) to facilitate and stimulate co-operation and communication between our members (Ph.D. students, research fellows, senior research fellows and associated members) and between the School and its stakeholders, including leading (industrial) companies in the field of ICT. NESCoR,2 the Netherlands School of Communications Research, was first accredited by the KNAW in 1999. It is an interuniversity graduate research school, comprising four universities. The school aims at contributing to knowledge on the political, social, psychological, cultural, and economic aspects of communications infrastructure and production, contents, and effects, in a society characterized by the increasing importance of, and dependence on, communication and information. NESCoR aims at realizing this mission by addressing fundamental scientific research questions. The training program is for young researchers from the Netherlands and abroad and is internationally oriented. Currently, the school has 86 senior researchers (33 fte research time) and 64 Ph.Ds (40 fte). NESCoR is in the process of being reaccredited by the KNAW. NESCoR facilitates (a) the training of graduate students in the area of communication science and research, (b) the study of fundamental issues in communication science, and (c) the use of this fundamental knowledge in applied research. The Universiteit van Amsterdam is NESCoR’s coordinating administrative university (‘penvoerder’). Prof. Dr. Jan Kleinnijenhuis is member of the scientific board of NESCoR.

1 2

http://www.siks.nl http://www.nescor.nl/

5

5

Description of Work

5.1

Abstract

Designing the User Experience of Web as Receivers of Mass Communication is one of the Ph.D. projects of transdisciplinary research to be carried out within VUBIS, in this case, a cooperation between the Information Management and Software Engineering research group at the Faculty of Sciences (FEW/IMSE) and the research group Communicative Performance at the Faculty of Social Sciences (FEW/CP). The current research project combines the research tradition of communication sciencists with respect to the effects of (computer) mediated communication on human information processing, on attitude formation and on decision making with the expertise of software engineers with regard to the design of interactive software to enable users to perform tasks pleasantly and effectively. This document describes the scientific contents of the envisioned work. Organizations use Web sites to inform the public on their mission, goals, and organizational hierarchy and try to convince people into joining groups, share work, update their knowledge, or buy goods. The most important means to do so are the use of language and visuals. Yet, new developments in ICT urge for applications that feature anthropomorphic software in which computer mediated people (i.e. sources) send out the organization’s message to receivers (here, the Web users). In applications that feature anthropomorphic software, however, not only the message type that is put forward by the sources in language and visuals needs to be understood but the attitude of the receiver/user to the mediating anthropomorphic agent needs to be right as well, since information reception depends both on perceived source characteristics and on perceived media characteristics. We want to explore whether agent assistance is beneficial to task-performance of the user, which is most germane in industrial applications. Should different agent types be employed to accommodate the needs of novices as compared to experts, for example? The overall purpose of this research proposal is to improve the design of the user experience of Web-based agents in the direction of realistic face-to-face interaction so to increase efficiency in task performance, starting from the DUTCH approach (Designing for Users and Tasks from Concepts to Handles). Interaction of end-users with antropomorphic agents may consist of alternating sequences of user responses (e.g. mouse clicks) on the one hand and displayed Web pages that are accompanied by postures and suggestions from the anthropomorphic agent on the other hand. Agents may be designed also as a visualisation of much more textually oriented question-answering-software, in which the user replies to questions from a visualized anthropomorphic agent, or vice versa, the software replies to question of the user, with words or phrases taken from the user’s natural language. Strict laboratory experimentation (e.g., mouse click reaction timing in a forced-decision task) for the inquiry into efficient task execution will be combined with a textual content analysis of questionanswering sequences and with structured questionnaires to assess the end-user’s experience of the agent’s ‘personality.’

5.2

Introduction

Broadly speaking, the early years of computing were characterized by many users served by one mainframe. Today, one user is struggling with one personal computer or workstation on his or her desk. In the future, however, many computer systems will serve many users simultaneously but without being visibly present any more (cf. ubiquitous computing). In other words, the user has to deal with many wireless or satellite connected applications by means of one or two mobile devices (a GSM or PDA). It is therefore needed that users can communicate with those applications without having to learn new interfaces and interface languages all the time. The ideal interface would communicate in a human fashion with users and talk computer code with different applications. The DUTCH design approach (Designing for Users and Tasks from Concepts to Handles) offers a conceptual framework to model ‘the ideal interface of the future.’ The assumption is that no user has a completely realistic notion of the computer system s/he is working with. The fictional system the user does work with is termed the ‘User’s Virtual Machine’ (UVM). Novice as well as expert users happily mix software with hardware features and apply human capacities to them (e.g., processing is thinking, storing is remembering). As long as the mental model that the user has about the system is aligned with the system’s true functionality, this is no problem. However, as soon as the User’s Virtual Machine is not following the

6

real machine’s functionality, the usability of the system drops. DUTCH offers entrance to the information sources and tools needed to create interfaces that keep track of the UVM in relation to the actual computer system. In the present proposal, the mediating role between UVM and system should be performed by a Web-based anthropomorphic agent system that can be tailored online to user needs both in appearance as in functionality. Web agents are software systems that operate in the World Wide Web, the Internet, and related corporate or government intranets. Agents may be designed to perform autonomously, preferably platform independent, classical information processing tasks, such as searching for and retrieving information, filtering and classifying information, conversion of formats, and information storage. The use of agents in human-computer interaction engages the user in a two-way “face-to-face” dialogue where the computer interface agent provides mutual information exchange and meaningful feedback. Agents are considered anthropomorphic to the degree that they become engaging, entertaining, approachable, and understandable to the user, thus harboring potential to build trust and establish relationships with users. An anthropomorphic agent is typically presented to the user as a mediated figure or animation agent to guide the users to the information sought for, the goods they want to buy, etc. It is believed that employing humanoid communicators on the Web reduces reluctance and increases involvement and credibility to use the services offered (e.g., buy, sell, inform, distribute work, etc.). Yet, although reluctance may be decreased, is the application of agents facilitating task performance (e.g., increasing efficiency) or is it just a nice diversion? With regard to anthropomorphic software, the original work by Reeves and Nass (1996) indicates that end-users attribute personality to a computer system, which appears to be psychologically “real” to those users. Systems are seen as independent social actors and sources of information. A nice and friendly responding system would enhance appreciation and task performance as compared to curt, no-nonsense feedback. The development of intelligent agents was the direct consequence of the Reeves and Nass studies but, for example, Microsoft’s Bob (and friends) failed because end-users did not like the ‘personality’ of the characters. In this line, Dehn and Van Mulken (2000) provide an overview of empirical work in the area and found that an animated agent does not necessarily improve a user’s comprehension or recall of information. The added value has more to do with motivational aspects. For instance, learners may be willing to spend more time with the learning application when an agent is present, or may feel less anxious in a comprehension test conducted by a synthetic tutor. Whatever the effect of an agent may be, the central issue is that the goals of the user are satisfied by the agent and that the work process towards those goals is facilitated. That is, the agent should be relevant to the task at hand (be it information or entertainment seeking, communication, or composition) and should support rather than impede the work process (raising positive instead of negative valence) (cf. Frijda, 1986, p. 494, p. 463, p. 455). Commercial systems such as Microsoft’s Bob and Clippit, kiosks such as the Postal Buddy and the anthropomorphic bank teller machines, or the recent Ananova news reader have failed because they either gave irrelevant information or slowed down computer processing and therefore, transaction time. Agents that distract one’s attention from information processing tasks invariably invoke irritation and negative feelings (Catrambone, 2002; Schaumburg, 2001). Moreover, an anthropomorphic agent that only presents technical advice rather than opinions may be perceived as more unrealistic. More extremely, many IRC channels do not allow software robots or ‘bots’ because it is hard telling a good bot from a bad bot (e.g., as programmed by a hacker, Quittner, 1995). These findings are consistent with research results regarding media effects on information processing and attitude formation. Research on educational television programs, for example, indicates that realistic pictures to represent abstract knowledge usually distract attention, whereas simplified ‘iconic’ diagrams may highlight the essence of the abstract information and may enhance comprehension (Heuvelman, 1989). Although information processing may be impeded by realistic, anthropomorphic visuals, the process of attitude formation may be stimulated (cf. McGuire, 1989). Human-like creatures evoke stronger emotions than bare texts or icons. Quite predictably attitudes tend to become negative in the case of anthropomorphic agents that distract one’s attention from information processing tasks. Serious information processing is but one task of computer users, however. Information on the Web that is usually skipped by users (e.g. advertisements) may get peripheral attention due to anthropomorphic agents. The agent’s vividness may boost perceived presence, which plays a pivotal role in generating more favorable attitudes toward the advertisement in the Web site and stronger intention to revisit the Web site (Choi, Miracle, & Biocca, 2001). Web agents that help a user to navigate through a new program are deemed less irritating than Web agents that give unsolicited opinions on traditional tasks, suggesting that task variation is much more important than anthropomorphic design variation (Catrambone, 2002). Although task

7

variation is the base line for design choices, as emphasized by the DUTCH approach, design choices presumably have an independent effect on user evaluations. In health care communication, for example, interaction with patients is much more important than medical information to influence patients. Carefully designed anthropomorphic agents may add to the repertoire of interactive health care communication (H.Y. Huang, 1999; Street, Gold & Manning, 1997). The present research proposal wishes to develop design theory for (multi-) agent systems by investigating the end-user experience of mediated figures and agents as realistic personalities so as to optimize task performance. Users of Web applications experience difficulty in performing tasks, particularly in information searches that require hypertext navigation. The present generation of digital guides and assistants, however, is often rejected as task-irrelevant, unreliable, or plainly irritating (Schaumburg, 2001). One of the problems is that so-called adaptive intelligent interface agents are neither adaptive nor intelligent. For example, they cannot differentiate between a novice and an expert user, let alone that they are intelligent enough to adapt to the evolution of a novice into an expert. Yet, novices need extensive help whereas experts only need a hint for efficient task execution. However, it is no use to software engineers to have developed an agent that adapts to user needs and changes in goals, if the user still puts the agent aside as only marginally interesting or downright boring. The central issue of the present investigation is to find out how receivers/users build up a relationship with computer programs that feature agents in the communication interface to support user goals and tasks. Compared to traditional ways of interaction (command line, dialog boxes), such agents supposedly improve user involvement with the computer system (Koda & Maes, 1996) and would help reaching user goals such as handling information (e.g., filtering and forwarding e-mail, doing information searches), buy goods (e.g., autonomous negotiator agents), or to be entertained (e.g., Alexbot playing the IRC Jeopardy game). Moreover, what kind of agent is capable of increasing the efficiency of taskperformance in novices and experts as materialized in speeded Web navigation and decreases in the error rates of clicking hyperlinks. Theories of TV (unidirectional) and face-to-face (bidirectional) communication both are relevant to the questions at hand. However, agents resemble interpersonal communication more than TV but less than real life interaction. The theoretical explorations, then, should be focused on what the limitations are of theories of TV and interpersonal communication regarding agents. How to keep the user involved with the agent, independent of the level of user expertise? What makes an agent entertaining without becoming a nuisance? As mentioned, task-relevance of agent behavior is a main factor. In addition, whether the agent supports or (perhaps unintentionally) slows down user performance or induces errors may also have quite an impact on user involvement with the agent system. This may impinge positive or negative valence upon the user, setting of tendencies to approach the agent (in the case of genuine help) or to avoid activating the agent when it merely obstructs reaching user goals. Another factor may be the similarity of the user with the agent. The popularity of avatars seems to warrant the assumption that users like to converse with a digital lookalike, which may have a positive impact on user involvement. Commercial persuasion presents media products that are furnished with beautiful people. Motion pictures exhaustively repeat showing the triumph of the good over the bad. The introduction of reality TV, docusoaps, Virtual Reality, and the current investments in the realistic rendering of multimedia all direct at the assumption that adding more realism to an application will improve user involvement with the system. Confirmatory factor analysis performed in Konijn & Hoorn (2003) on viewers’ responses to film characters shows that none of these factors is redundant in explaining involvement with and appreciation of a mediated figure. We wish to investigate how the 6 factors of task-relevance, valence (user support or not), similarity, agent ethics (a good bot or a bad hacker bot, cf. Quittner, 1995), outer attractiveness or aesthetics (beautiful or ugly appearance), and reality representation or epistemics (realistic-unrealistic rendering) explain involvement with, distance towards, and appreciation of the agent in novices and expert users. A structured questionnaire with which user involvement and appreciation of mediated figures can be estimated is already available and thoroughly tested for psychometric qualities (Konijn & Hoorn, 2003). When we know what the right settings of the Web agent are to let the user experience involvement with the agent, then how do the different settings or designs of the agent affect the efficiency of task performance in terms of high speeds combined with low error rates? Two groups of users (novices vs. experts) should participate in a forced-decision reaction time experiment. The task is a simulation of navigating through a hypertext structure to find certain information by clicking the correct hyperlink or the incorrect link. Trials consist of ‘searches,’ which actually are simple decisions between two hyperlink options (e.g., Journals vs. Books), preceded by a search task (e.g., “find an article”), and after the decision,

8

followed by feedback on the selected information (correct-incorrect). Together, the sequence of simple decisions between pairs resembles an extended navigation through a (narrow) hypertext structure in which users check whether a selected page was the right choice or not. Agents of various kinds serve to either cue or wrongly cue a hyperlink. In subsequent experiments, adding hyperlink options can extend the width of the navigation structure. The dependent variables are the click speed and error percentage of decision making between the hyperlink options.

5.3

Research Questions

The following research questions serve as a list of work hypotheses to explore the theoretical fields involved in this multidisciplinary approach but can be altered in recognition of viable alternatives as derived from those theoretical explorations. I What kind of users (novices-experts) perform what tasks while surfing the Web (e.g., navigation)? Is it particularly helpful to have a mediated person or embodied agent support those tasks? II How does a user build up a relationship with an agent (cf. parasocial interaction)? How can we avoid high degrees of emotional distance towards the present generation of digital humanoids? III Which features of the agent are relevant to the user in order to get involved with the agent so to improve task performance? One can think of task-related features but also of outer appearance and degree of realism. IV What are the computational aspects of engaging with agents? Can we provide a model of the design of agents that can account for sometimes-paradoxical user behaviors (answers to research questions I-III). This, to allow design algorithms of user-tailored agents.

5.4

Method

Proposed methods and approaches are task analysis, computational content analysis, reaction time studies, and structured questionnaires. The study is performed within the DUTCH design framework for interactive systems, developed at the VU and internationally recognized (flagship textbook forthcoming at the McGraw-Hill Education). Within DUTCH, Groupware Task Analysis (GTA) will be performed to determine user tasks in different stakeholder groups of the Web system (Van der Veer, Lenting, & Bergevoet, 1996). Computational Content Analysis (CCA) is a particular area of expertise of the VU as well (Van Cuilenburg, Kleinnijenhuis, & De Ridder, 1988; Kleinnijenhuis, 2003; Kleinnijenhuis, De Ridder, & Rietberg, 1997; Lagerwerf, Spooren, & Degand, 2003). CCA will be performed to pinpoint the product features of the agent in terms of, for example, good or bad intent (cf. Quittner, 1995), attractive graphics, and degree of realistic rendering. The features of the agent are then manipulated so to satisfy or dissatisfy the user needs as derived from GTA. In formal laboratory experiments, the task performance of the user in different conditions of the agent’s support or impediment of the user can be measured by means of reaction timing (speed) and error registration (accuracy), both indicators of efficiency. The emotional involvement with and appreciation of the agent can be assessed with a structured questionnaire that is developed and pre-tested at the VU with characters in motion pictures. For both the study on user involvement as well as the efficiency study, multifactorial experimental designs are inevitable. To guarantee the feasibility of the practical work, we will propose a step by step approach in which gradually more factor levels can be systematically crossed, ideally leading to a coverage of the complete factorial design (Table 1 in Section 5.4.3).

5.4.1 Participants The experiment should be performed by novice and expert groups, which makes a total of N= 2(novices-experts) * (8 (agent types)*40 users) = 640 payed participants to cover the full factorial design

9

(Table 1 in Section 5.4.3). However, this design is simplified in Section 5.4.3, to make the investigations more manageable.

5.4.2 Stimuli Agents form the stimuli in an experiment on Web agent navigation efficiency and Web agent personality evaluation (Table 1 in Section 5.4.3). Because there are 6 factors (Task-relevance, Valence, Similarity, Ethics, Aesthetics, and Epistemics) with two levels each for creating the agents’ ‘personalities,’ all in all, 26 = 64 agent characters should be designed, that only differ from each other regarding the manipulation. That is, stimuli are graphic variants of the same blueprint, for example, a good bot versus a bad bot (factor Ethics) in a realistic and an unrealistic rendering (factor Epistemics). The manipulation consists of the following. Task-relevance is reflected by agent behavior that pertains to the task (relevant behavior is to cue the hyperlink) or showing behaviors unconnected to the task (irrelevant behavior is to offer help and tips on, for example, word processing). To increase task-relevance of the agent, users will receive a reward if they expose the fastest and most accurate performance of their group. Valence is manipulated by offering the user genuine support (positive valence induced by correct cues) or by misleading the user (negative valence induced by incorrect cues). Similarity is established by presenting an avatar of the user (similar) or a dissimilar figure. Ethics is reflected by agent behavior that is friendly (encouraging) or hostile (impolite feedback). Aesthetics is reflected by agents that are designed according to universal standards of beauty (e.g., average face shape and symmetry, see, among others, Johnston and Oliver-Rodriguez, 1997). Ugliness is induced by deviations from those standards, for instance, agents having misshapen skulls or showing signs of ‘physical’ decay. Epistemics is reflected by agents that are designed like photo-realistic mediated persons or, for example, unrealistic cartoon figures.

5.4.3 Procedure Obviously, confronting users with 64 different agents will not produce the desired results. Therefore, we propose to conduct four experiments with a subset of agent types. To do so, we suggest making a subdivision of 8 agents that are systematically crossed on ethics, aesthetics, and epistemics as between-subjects conditions. We will term this triad Agent ‘perception.’ Within each condition, agents are systematically crossed on task-relevance, valence, and similarity, making another 8 agent variants as within-subjects conditions. This triad we will term Agent ‘experience’ (Table 1). The agent’s features that impinge upon ‘perception’ and ‘experience’ refer to the sub-processes in the PEFiC-model (see Figure 3). In Experiment 1, the ethics, aesthetics, epistemics, and similarity of the agents is held constant, whereas task-relevance and valence differ. Agent ‘perception’ (between-subjects) is restricted to the goodbeautiful-realistic agent types (cells (1)-(8), Table 1). The ‘goodness’ of the agent may lie in polite and positively formulated feedback on the user’s performance. With respect to Agent ‘experience’ (withinsubjects), agents are more-or-less similar to the user in outer appearance (avatar condition), whereas the 2*2 manipulation pertains to relevant or irrelevant behavior of the agent with respect to the user task (factor Task-relevance) crossed with user support or user obstruction (factor Valence). The latter two factors are within-subjects. Thus, all agents in Experiment 1 are the same regarding the three factors of Agent ‘perception’ and the Agent ‘experience’ factor Similarity. The variation in Experiment 1 only concerns the factors Task-relevance and Valence. The tasks users are asked to perform are twofold. First, users perform a number of simulated Web navigation searches. They are to participate in a forced-decision reaction time task, in which they have to choose between two hyperlinks to find the proper information. For example, if users get the task to find a scientific article, they are confronted with two hyperlinks: Journals versus Books. ‘Journals’ is a better choice than ‘Books,’ which contain chapters instead of articles. The correct decision, then, is ‘Journals.’ In the first condition (agent type 1), the agent shows task-relevant behaviors (i.e. directly cueing the hyperlinks) and supports the user by pointing at the correct link. In the second condition (agent type 2), the agent shows task-irrelevant behaviors (i.e. indirectly cueing the hyperlinks while showing superfluous animations, such as jumping, blinking, etc.). Nevertheless, it supports the user by cueing the correct link. In the third condition (agent type 3), the agent shows task-relevant behaviors (i.e. cueing the hyperlinks) but (occasionally) misleads the user by pointing at the incorrect link. Users get feedback on their correct or incorrect choices, so to affect their attitude towards the agent. In the fourth condition (agent type 4), the

10

agent shows task-irrelevant behaviors (i.e. indirectly cueing the hyperlinks) and (occasionally) misleads the user by pointing at the incorrect link. Decision speed and number of errors will fluctuate with the proper or improper cue the agent provides and the relevant or irrelevant behaviors it exposes. After finishing the online task for one condition, users fill out an offline questionnaire to evaluate the agent’s ‘personality’ in terms of user involvement, distance, and appreciation. In total, then, users fill out the questionnaire four times. This constellation results in a 2 (Task-relevance: Relevant vs. irrelevant) by 2 (Valence: Positive vs. negative) by 2 (Users: Novices vs. experts) factorial design. Because the four agent types (relating to the four conditions) are presented within-subjects, 20 novice participants plus 20 experts suffice, making a (sub)total of n = 40. Experiment 2 is similar to Experiment 1, except that the agent is dissimilar to the user (unalike condition). This makes a subtotal of n= 80 participants. Experiment 3 repeats the same set-up as Experiment 1, so that the agents are similar to the users again (avatars). However, although user similarity is the basis, agents are edited such that they become bad, ugly, and unrealistic (cells (57)-(64), Table 1). The ‘badness’ of the agent may lie in impolite and negatively formulated feedback on the user’s performance, ugliness in showing signs of deformation and decay, and the agent may be unrealistic in being dressed up like a sci-fi figure. The subtotal of participants is n= 120. Experiment 4 is similar to Experiment 3, except that the agents do not look like the users (unalike condition) (accumulated n= 160). Taken in unison, the combined experiments make a 2 (Users: Novices vs. Experts) by 2 (Agent perception: Good-beautiful-realistic vs. bad-ugly-unrealistic) (between-subjects) by 2 (Similarity: Similar vs. dissimilar) (between-subjects) by 2 (Task-relevance: Relevant vs. irrelevant) (within-subjects) by 2 (Valence: Positive vs. negative) (within-subjects) factorial design and will be analyzed as such. In Table 1, the full factorial design is displayed if all eight combinations of factor levels of Agent ‘perception’ are explored as well. However, it is too much of a burden for the junior researcher to deal with the full of this experimental design. Nevertheless, Table 1 offers possibilities for explorations beyond Experiment 1-4 by means of so-called ‘cell grazing.’ Table 1. Full factorial design for an experiment on Web navigation efficiency and Web agent personality evaluation, using different agent types Agent ‘experience’ Relevantvalence pos.similar Irrelevantvalence pos.similar Relevantvalence neg.similar Irrelevantvalence neg.similar Relevantvalence pos.dissimilar Irrelevantvalence pos.dissimilar Relevantvalence neg.dissimilar Irrelevantvalence neg.dissimilar Subtotal Cumulative N

Agent ‘perception’ Good-beautifulBad-beautifulrealistic realistic Task order: Task order: RT RT Quest (1) Quest

Good-uglyrealistic Task order: RT Quest

Bad-uglyrealistic Task order: RT Quest

Good-beautifulunrealistic Task order: RT Quest

Bad-beautifulunrealistic Task order: RT Quest

Good-uglyunrealistic Task order: RT Quest

Bad-uglyunrealistic Task order: RT Quest (57)

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest n= 20 Task order: RT Quest (5)

RT Quest n= 20 Task order: RT Quest

RT Quest n= 20 Task order: RT Quest

RT Quest n= 20 Task order: RT Quest

RT Quest n= 20 Task order: RT Quest

RT Quest n= 20 Task order: RT Quest

RT Quest n= 20 Task order: RT Quest

RT Quest n= 20 Task order: RT Quest (61)

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

RT Quest

n= 20 n= 40 N= 40

RT Quest

n= 20 n= 40 N= 80

RT Quest

n= 20 n= 40 N= 120

RT Quest

n= 20 n= 40 N= 160

RT Quest

n= 20 n= 40 N= 200

RT Quest

n= 20 n= 40 N= 240

RT Quest

n= 20 n= 40 N= 280

RT Quest

n= 20 n= 40 N= 320

Note. RT= reaction time task, Quest= questionnaire task. Cells with horizontal borders designate between-subjects conditions; cells without horizontal borders designate within subjects conditions. Between brackets are the cell numbers. N= 640 if the full factorial design is conducted with both novices and experts

Next follows a number of general methodological remarks, which concern all four experiments. The experiments entail two tasks in a fixed order (Table 1). Each participant performs a forced-decision reaction time (RT) task that simulates Web navigation prior to the agent’s ‘personality’ evaluation. We wish to combine the online measurement (efficiency) with the offline measurement (involvement) for

11

reasons of ecological validity (active navigation goes before evaluation) and of methodological reliability (tasks are performed within subjects with a unidirectional order effect). Participants in one sample (n= 20) are pseudo-randomly assigned to 4 agent types to work with and to evaluate (N= 640 for the full factorial design, see Table 1). Yet, to control for order effects, the order of presenting the 4 agent types should be different for each participant (4*6= 24 different orders are possible). In the reaction time task, each user does 20 searches or ‘trials’ per agent type. Trials represent simple decisions between two hyperlink options, preceded by a search task (e.g., “find article”). After the decision (e.g., between Journals and Books), the user receives feedback on the selected information (correct -incorrect). Whether the agent cues the correct hyperlink or not is an aspect of the manipulation of Valence. Agent types are blocked and trials are mixed within blocks in a pseudo-random order. After users completed the 20 search trials for an agent type, they should evaluate the ‘personality’ of the agent by filling out a (digital) questionnaire. Questions have a blocked-mixed design as pointed out in Hoorn, Konijn, and Van der Veer (2003b). Then the next 20 searches with another agent type commence, after which the questionnaire is filled out again. This means that in total, each user performs 80 searches or trials (estimated maximal duration: 1 hour) and fills out the questionnaire 4 times (estimated maximal duration: 1 hour). Care should be taken to avoid fatigue and test-retest biases by making psychometric scales as short as methodologically possible.

5.4.4 Measurements Dependent variables are velocity of decision time and error rate to indicate Web navigation efficiency and user-involvement with, distance towards, and appreciation of the agent to indicate userengagement with the agent system. Independent variables are the 6 factors Task-relevance, Valence, Similarity, Ethics, Aesthetics, and Epistemics. The respective psychometric scales to assess the values of the independent variables from the user’s perspective, and thus, the checks on the manipulations, are already available and can easily be adapted from Konijn and Hoorn (2003). However, conventional software packages (e.g., ERTS, Beringer, 1992) for the behavioral sciences (i.e. reaction timing) do not support animated stimuli that can meet industrial standards. To facilitate such research, agent-based software packages that feature animated agents should be added to conventional packages for testing the usability of animated agents and environments by means of response speed in user tasks. State of the art technology developed at the VU (e.g., Huang, Eliëns, & Visser, 2003a) can easily supply the necessary technical add-ons and may well-provide for creating stimulus materials (see Section 7.5: Intelligent Multimedia @ VU).

5.5

Expected results and merits

Exploring the parameters of human-agent collaboration can lead to optimization algorithms that tailor the features and behaviors of the digital communication partner to the needs of the user. First, agents who are relatively positive on Ethics, Aesthetics, and Epistemics will evoke higher involvement and lower distance than agents who are relatively negative on these factors. Second, agents who are relatively positive on Ethics, Aesthetics, and Epistemics will evoke higher appreciation than agents who are relatively negative on these factors. However, thirdly, we believe that mixed evaluations in the appraisal domains (e.g., bad-beautiful-unrealistic) may counteract those general tendencies and lead to higher appreciation of an agent, because the agent becomes more interesting, exciting, or fun. This comes about in significant interactions (e.g., as found in Konijn & Hoorn, 2003, for film characters). Fourth, we expect that the tradeoff between involvement and distance explains the appreciation of an agent better than either involvement or distance alone. This counters the avatar-hypothesis, which claims that high similarity between user and agent predicts the highest appreciation. Finally, on the basis of the available literature, precise predictions are hard to formulate for the perceived similarity, relevance, and valence, which should be included as mediating variables in between the encoding of the stimulus and the response phase. In exploratory statistical analyses these complicated compositions will be scrutinized. In addition, we expect that efficiency of task performance as reflected in the speed-accuracy data may decrease because of task-irrelevant cues but that appreciation of the agent is increased because the agent is involving. By contrast, efficiency of novices in performing the task can increase because the agent

12

gives correct cues but this may be at the cost of emotional distance towards the expertise of the agent, due to a lack of similarity with the novice user. The merits we anticipate from these results are: 1 Improvement of the theories of parasocial interaction, reality and person perception, as well as impression formation with regard to technological innovations, such as anthropomorphic software. 2 A productive quantity of empirical data from a variety of methods and techniques (task analysis, content analysis, reaction timing, questionnaires) on how users perceive and experience mediated figures and agents on the Web, aspects of task performance, and how agents can contribute to more efficiency. 3 Improvement of the theory of character engagement and mediated interpersonal communication by confronting these theories with empirical data. 4 Contributing to user-centered design of Web-user support applications with more specific guidelines and algorithms for the communication between humans and machines.

5.6

Future Research: Towards True Adaptability

We believe that users of Web applications have at least three general goals in mind: To be informed (instruction, education, e-learning), to be entertained (e.g., e-gambling, gaming), and to be persuaded (e.g., in e-commerce or health campaigns). In future studies, we ought to focus on the first two user goals because the use of an agent to a novice will relate to being instructed while for an expert it may lie in being entertained (cf. infogames). In other words, we could anticipate effects of user expertise (novice or expert) in interaction with the priority of user goals (instruction or entertainment) on the degree of relevance of the agent to those goals. For novices, the relevance of the agent to instruction should be high and to entertainment low, otherwise the novice gets distracted from the task. For experts, relevance of the agent to entertainment should be high and to instruction low, otherwise the expert becomes irritated by receiving information s/he already knows. For the knowledgeable user, the needs of being instructed and entertained are in balance and should be about equal. The main question in future research is how an agent should adapt to the shift in relevance to user goals. While working online, we can only obtain indirect measurements because asking the user straightforwardly interferes with task performance and has methodological drawbacks (e.g., social desirability and answering tendencies). We may assume, then, that while the learning curve grows and efficient task execution increases, the call for help decreases (visible in the frequency of activating assistance functions). The speed of task execution in combination with the number of errors that are made may indicate the efficiency of task execution. By keeping track of the efficiency of the single user in comparison with the average efficiency curve of that particular application, the agent knows when to evolve from being a tutor into being an entertainer or even a game hero. Entertainment, then, may still have an educational side in that the entertainer challenges the expert to improve upon him/herself and to teach the agent new procedures.

5.7

Work Plan

For the proposed research, the following tasks should be performed. The task order is not necessarily fixed, but will be indicatively included in the time schedule below. I.

Reading DUTCH design approach; Task analysis; Theories and methods on Perceiving and Experiencing Fictional Characters; Theories of interpersonal communication; Reality perception in mediated communication; methods of content analysis

II.

Education

13

Experimental testing (manipulation procedures, designs, statistical analyses, i.e. MANOVA and SEM). Content analysis. Writing academic English. III.

Agent development Determining the contents of the PEFiC factors with regard to the Web interface agents (e.g., what are regarded realistic features, task-relevant features, etc.). Theoretical explorations and pilot studies using content analysis techniques.

IV.

Programming agent software, featuring the contents of III.

V.

Developing hyperlink and task contents Theoretical explorations and pilot studies using task analysis and content analysis. Which words should be used as primes (for stating the user’s tasks) and targets (the hyperlinks) with what semantic distances?

VI.

Programming the experimental Web site Such in accordance with the requirements of laboratory experimentation. The agents developed under IV serve as cues, the task words and hyperlink words developed under V as primes and targets.

VII.

Pretest on the PEFiC questionnaire as adapted to the user’s evaluation of the Web agents

VIII.

Running the four experiments; running a content analysis of user interactions with existing agents

IX.

Analysis of results

X.

Writing of reports, papers, and the thesis

Depending on the background and qualities of the PhD-candidate (see 3.3), the work plan can be adjusted. In general, it is to be expected that a candidate with a background in Computer Science will need more time for the social-theoretical backgrounds of the project, whereas a candidate with a Communication background will need more time for the technical aspects of the project. Once experiment 1 is set up and conducted, subsequent experiments will be easier to conduct. Therefore, the time plan is skewed with respect to the experiment’s feasibility and with respect to the PhD candidate. The numbers I, II, III, etc. refer to the above tasks. Starting January, 1, 2004, finishing December 31, 2007 (4 yr. 10/10 fte): January 2004 – December 2004: Tasks I, II, III, and X. That is, focused literature search and reading. Writing a literature report + preparatory activities regarding Exp. 1. Following courses in Experimental Testing, Content Analysis, and specific theoretical backgrounds. Jan. - July: Focused reading and reporting. Sept. - Oct.: Preparatory activities regarding Exp. 1 + additional reading, reporting, education. Nov. - Dec.: Writing a theoretical contribution (publication) based on the reports. Jan. 2005 – Dec. 2005: Tasks IV, V, VI, VII, parts of VIII, IX, and X. That is, preparing, executing, and reporting. Experiment 1 with an eye on materials for subsequent experiments. Jan. – July: Preparing the necessary materials and pre-testing; additional education. Sept. – Nov.: Conducting Exp. 1, statistical analyses, reporting, perhaps additional education. Dec.: Finishing the report on Exp. 1. Jan. 2006 – Dec. 2006: Tasks I, VIII, IX, and X. That is, extended reading, preparing, executing, and reporting Exp. 2, 3, and 4. Presentation and preparing publications of the results. January - April: Preparation, execution, analyses, and report of Exp. 2. May - July: Presentation of Exp. 1 at a conference. Finishing scientific publication Exp. 1.

14

Sept. – Dec.: publications.

Executing Exp. 3 and 4 + statistical analyses, writing reports + scientific

Jan. 2007 – Dec. 2007: Tasks IX and X. Jan. – April: Finishing reports + scientific publications on Exp.s 2, 3, and 4. May: Presentation of Exp.s 2, 3, and 4 at (a) conference(s). June – Dec. 2007: Writing the final Thesis. Where necessary, improving the foregoing (submitted) scientific publications as chapters, writing the introduction and discussion. Jan. 2008: Thesis defense in promotion ceremony.

6

Relevance to Academia and Society

6.1

Relevance to Social Sciences

From the four expected results mentioned in Section 5.5, the points 1, 2, and 3, and to a lesser extent 4, are most relevant to the communication sciences. In an information-dense society where massmediated, organizational, and interpersonal forms of communication increasingly have to compete for scarce attention of target audience members, communicative performance in Web-based applications becomes a pivotal construct with which to judge the strategic worth and viability of (mediated) messages. This viability is dependent on the extent to which various forms of communication, such as user-agent interaction, are able to gain, hold, and channel attention. It is this quality that is the essence of communicative performance and is at the heart of research in communication science at the Vrije Universiteit Amsterdam. The general research mission of the research program “Communicative Performance” is to conduct empirical (field, experimental, and content analytic) research to conceptualize this construct. Computer-mediated communication with Web agents will deepen our understanding of the potential of varying forms of communication to affect psychological, affective, cognitive, and behavioral responses of audience members (individuals and groups) in their social context. This mission manifests itself in the following main research topics that guide research at the Vrije Universiteit and that are directly addressed by the present proposal. We wish to explore the effects of specific combinations of informational contents, person portrayal, and other message elements, on the psychological, affective, cognitive, or behavioral responses of audience members (individuals and groups) in their social context. Investigation of the way agents are portrayed as good/bad, beautiful/ugly, and realistic/unrealistic and the effects on user-involvement, distance, and appreciation of the agent in novices and experts is a specification of just that. Key constructs in understanding communicative performance are the dynamics of (mediated) contents, the Web, and ‘persons,’ such as agents. Our Research Domain I, Mass-mediated communication, focuses on communicative performance in contents and persons in the media. An extension with multimedia (as applied in Web applications) is most welcome. Within the realm of mass-mediated communication, communicative performance pertains to the issue of the extent to which (fictional or real) persons, such as agents, or (informational) contents are featured in the mass media and in what way, as well as their differential effects on audience members (e.g., involvement, distance, and appreciation). Its primary sources of data are content analysis and survey data. Starting point is that increasingly the mass media determine the “public sphere” (cf. the Web as important information supplier). Research in this area aims at gaining insight in the relationship between the media and the audience. An interesting area of research by itself is the further development of automatic content analysis to tap the contents and images of the World Wide Web and electronically available media archives.

6.2

Relevance to Computer and Information Sciences

From the four expected results mentioned in Section 5.5, the points 2 and 4, and to a lesser extent 3, are most relevant to computer science. In communication science, a long tradition prevails in studying human interactions in mass communication such as newspapers, radio broadcasting, and TV. Theories and methods developed in this area can throw much needed new light on a medium that typically evolved from 15

computer science, namely the World Wide Web. The Web can be seen as a digital mix of newspaper, radio, and TV, so that the Web too should be seen as a form of mass communication in which the sender, for instance, is an organization, the Web sites’ contents are the message, and the user is the receiver. Agent technology has been concerned with developing algorithms for distributed computing (multi-agent systems), transparency (autonomous agents), and 3-D graphical rendering. Designers of agent systems have been concerned with making agents ‘emotional,’ that is, implementing affective responses to user actions (Velasquez, 1997). HCI studies that investigate the user experience of agents bring together all kinds of factors (e.g., user anxiety, task performance, and subjective evaluations in Rickenberg & Reeves, 2000). What is missing out, however, is a full-fledged model of the user in cognitive and emotional interaction with agents to understand the effects of, for example, reality perception, involvement-distance trade-offs, appreciation of but also task-relevance of the agent to user goals and concerns (cf. Konijn & Hoorn, 2003). Emotional responses per se are not necessarily task relevant. Task relevance is not necessarily improved by emotional feedback. For the present study, then, computer science is in need of a unidirectional as well as a bidirectional approach to human-agent communication. On the one hand, agents are like TV personalities, game heroes, or cartoon figures. Communication is from the agent to the user and for a large part, the agent is ‘deaf’ to what the user says. Horton and Wohl (1956) refer to this kind of communication as part of parasocial relationships (cf. parasocial interaction, Meyrowitz, 1985; Giles, 2002). On the other hand, the agent simulates to be a real person by interacting in a more-or-less natural way (cf. chatterbots, conversational agents, or voice based telephone applications). This ‘person’-to-person aspect of agents can be covered by theories of interpersonal communication, which already are applied to mediated interpersonal communication (cf. groupware such as e-mail and SMS; Barnes, 2001; Gumpert & Cathcart, 1986; Walther, 1996). Reeves and Nass (1996) claim that receivers/users treat their machines like humans, so that consequently, interpersonal communication theories would fully apply. However, we believe that state of the art agent communication is far from perfect to human standards so that theories of parasocial interaction are concurrent. The present study wishes to settle the parameters of both approaches. A third line of research that is fruitful to design the user experience is the mediating effect of ‘information reality’ on particularly the beliefs of the user about the agent and the message that source transmits (Shapiro & McDonald, 1992; Hoorn, Konijn, & Van der Veer, 2003a). Whether the agent is judged as realistic (e.g., a digital newsreader in a kiosk) or unrealistic (e.g., the Bonzi Buddy figure) makes a great difference in estimating the believability and credibility of the agent and its message.

6.3

Societal relevance

In view of the future growth of (invisible) applications that will furnish everyday life, interfaces that can communicate in a human way with ordinary people are of germane interest. The digital transformation of data and contents has placed (and will place) people with little background in computer operation and information navigation for a big societal divide: Those who can and those who cannot access information (e.g., digital form filling, info-kiosks, latest news). The digitalization of contents thus should be followed (or better anticipated) by novice support. One way of doing this, is to provide end-users of Web applications with a digital tutor or guide. However, it should not be the case that conversing and interacting with such an agent requires computer skills instead of social skills. For those who can access information, the use of agents is diminishing unless agents are capable of keeping up with the expert in a mature way. This way, expert and agent can teach one another, the interaction between which may boost productivity and global knowledge distribution. In systematically exploring agent characteristics in interaction with level of expertise, the present research proposal directly cuts into the socio-technical engineering of agent types that users of different background find agreeable as well as useful. Thus, the proposed study wishes to contribute to universal access of information. Moreover, increased efficiency and work pleasure will advance productivity, which eventually, is beneficial to the entire community.

16

7

Background of Related Research

7.1

DUTCH and GTA

As stated in the Introduction (5.2), the DUTCH design approach (Designing for Users and Tasks from Concepts to Handles) offers a conceptual framework to model ‘the ideal interface of the future.’ Moreover, it gives entrance to the information sources and tools needed to create interfaces that can align the User’s Virtual Machine with the actual computer systems users are dealing with. The DUTCH design process is task based which means that it uses the tasks of users as a driving force in the design process. The goals are to design both usable and useful systems. We think it is important to base the design on the work that has to be done by the users. Therefore, the users play an important role in acquiring knowledge about their work as well as for usability testing. Our process consists of four main activities: (a) Analyzing a “current” task situation, (b) envisioning a future task situation for which information technology is to be designed, and (c) specifying the information technology to be designed. In parallel to these activities, (d) evaluation activities make the process cyclic. In a global way, DUTCH states that the design process of a complex IT system consists of three types of activities: Analysis, specification, and evaluation. Analysis points to phases in design where an existing context of IT use (labeled “Task Model 1”) is considered and assessed on its values for all stakeholders concerned, and where a future context of use (labeled Task Model 2) is envisioned and assessed. These models are supposed to include both the situation, the technology (at a global level, as part of the situation) and the users and stakeholders. Specification indicates the phases where details of technology are elaborated (as far as the users and stakeholders are concerned). Evaluation includes all activities that are aiming at assessment of ideas, global and detail specifications, tentative proposals and refined improvements during the design process. Users are the most important element in the DUTCH design approach. In our studies, we use psychological and ethnographic techniques that allow designers to find out about the user and to obtain information about the organization and groups of users. Task analysis techniques receive special attention in our work. Groupware Task Analysis or GTA (Van der Veer & Van Welie, 2000) is the framework we use to cover the wide range of aspects to consider in the analysis of the current situation (Task Model 1) and the envisioned situation (Task Model 2). In task analysis we include modeling the organization, the work situation, and the history of both. Figure 1 depicts an ontology of the task world of a user. In GTA, the word ‘agent’ covers all entities, whether human or technological, that can perform tasks. Users, then, are agents in the form of human actors. Software agents are agents in the form of technological actors. The present research proposal focuses on the interaction between human agents and software agents. Users and software agents perform tasks like searching, navigating, and exchanging information. They use objects such as databases and hardware devices to perform those tasks. The roles software agent and user play often is of sender and receiver, of supply and demand, of helper and helped, respectively. However, these roles may switch during the process when for instance the software agent needs more information or the user interferes with a software agent’s task. This is called interaction. All this action and interaction serves to an end, which is the goal of the user, for instance, to find information, to be entertained, to reach a business goal (cf. Marketing Information Systems), or to streamline a work process (e.g., tips and suggestions by Microsoft’s Clippit).

17

Software

User Figure 1. The task world of the user (Van der Veer & Van Welie, 2000). External and internal events may induce the need for new tasks, for setting new goals, and inventing new work processes. By pressure from the market, for instance, a digital library can develop into an e-commerce company. The change in business model from an educational to a commercial company has an impact on the business goals that are set and the business processes that should help achieving those goals. Offering high quality contents and keeping an exhaustive stock may be of interest to an educational institution but may be cost-intensive in the eyes of a commercial service. This changes the user’s requirements of a computer system or software application, in our case, the software agents. A library would want a software agent to immediately point out new works that might be of personal interest to the user or improve the user’s comprehension of a scientific problem. Commercial industries, however, would hope that an animated agent attracts more traffic on their pages because it is fun to interact with a somehow life-like entity. They hope for ease of communication because the software agents to a certain extent emulate human-human communication. Concepts used in the DUTCH design framework to relate methods and different aspects of the design process are extracted from theories in cognitive psychology, distributed cognition, ethnography, HCI, graphical representation, and multimedia. Handles and affordances, and in general, solutions for ordinary people, are the design products in which we are mainly interested. The specification of the user interface covers all that is relevant to the user in context. The main domains we explore in our studies are cultural domains, Web sites, walk-up-and-use systems, and complex systems. At this point of the design process, to go into the details of technology means to focus on what users need to know about the functionality, the representation and the way to communicate with the envisioned system. The associated evaluation refers back to the users with scenarios, mock-ups, and simple prototypes that allow us to start an evaluation very early in the design process. The DUTCH-framework will be used as a point of departure for the selection of tasks that will be used as stimuli in the experimental design and the content analysis. Tasks such as serious information processing, that are incompatible with anthropomorphic agents that distract one’s attention (e.g., Catrambone, 2002; Schaumburg, 2001; Nowak, 2003) will not be studied here. The DUTCH framework will be used in future research for the design of anthropomorphic Web agents on the basis of the research findings.

7.2

Content Analysis

Content Analysis is defined as any research method to extract data from texts or visuals in a systematic, replicable way. Methods for discourse analysis in linguistics are closely related (Lagerwerf, Spooren, & Degand, 2003). Apart from extracting information on the occurrence or the frequency of topics,

18

determining the tenor of texts with respect to these topics is a central aim. By using statistical approaches based on occurrences and co-occurrences of specific keywords (in specific grammatical constructions), judgements of diverse aspects of texts may be delivered automatically. Together, these approaches make it possible to build information structures of texts, make abstracts automatically, or disclose tendencies in the contents of multiple texts. Content analysis based on occurrences and co-occurrences can be almost completely automated by applying Single Value Decomposition (“latent semantic analysis,” e.g. Foltz, 2003) and “support vector” software (Joachims, 2002). Content Analysis starting from semantic similarities, i.e. the extraction of subject-object-predicate-triples, is more difficult (Kleinnijenhuis, 2003), but the VUBIS-project of Van Harmelen (FEW) and Kleinnijenhuis (FSW) intends to shed light on this research area. Content Analysis will be used in this research project for a large scale analysis of verbal logs of user interactions with existing Web agents that employ verbal exchanges with users (typically questionanswering-sequences in addition to mouse clicks). Which Web agents will be selected, depends on the task analysis. In the case of corporate or government Web sites, their cooperation will be asked, under a strict privacy guarantee. The design of the content analysis is to compare a few comparable Web agents that are used for the automation of structurally different tasks, as well as a few different Web agents for more or less the same task. The logs of the interaction sequences will be compared regarding the topics of discussion (who – agent, user – did respond to what?) and with regard to verbal indicators of epistemic, aesthetic, and ethic (evaluative) appraisals of the users. The content analysis should reveal data on the interaction of Web surfers with lifelike agents that can be compared to the experimental data.

7.3

PEFIC: Agents as Fictional Characters

Although the user may feel interacting with a real person, in fact all embodied agents are fictional characters. Against Reeves and Nass’s (1996) position of the computer as a real person, one may argue whether users actually think there is another person or just apply the same social and communication rules to the computer because the computer roughly fakes human interaction? Sometimes people talk to their pen or pat their car on the dashboard as a form of self expression but that does not mean they mix up fiction with reality (Hoorn, Konijn, & Van der Veer, 2003a). Although FitzGerald et al. (1998) avow that “… creating audience empathy with the agent through manifestations of personality can create motivation in the student learner that would not otherwise exist,” most agents do not really emotionally absorb the user but remain quite empty puppets (Bonito, Burgoon, & Bengtsson, 1999). Much of what an agent should offer depends on the task at hand. In the best tradition of role-playing games, characters for entertainment are multi-sided, have contradictory features, and cope with exciting situations. If agents should entertain and motivate, they perhaps should be designed as characters of a computer game. Would not it be exciting to discover that our animated pedagogical agent is a dark-haired, gothic girl of about 18 years old, with a bloody apron covered with symbols and tied together with a skull on her back (Alice, American McGee, Figure 2)? To date, agents are boring, slick good guys serving as obedient slaves. Much of the distance they evoke, comes from their stupidity in executing tasks. Konijn and Hoorn (2003), however, show that characters that should entertain raise distance that is based, for instance, on ethical badness. This distance, however, contributes to the overall appreciation of the character. People like role-playing games because they want to escape from daily life and identify or have empathy with an incredible hero. If agents should have an entertaining side, we perhaps might want them to be empathetic. However, notions such as empathy and identification have many meanings. If we really completely identify with the character, then the user suffers as much pain as the blood and gore that Hana TsuVachel experiences in Fear Effect 2: Retro Helix (Kronos). Game after game, an empathetic player would experience the real terror of detective Edward Carnby, who finds his friend Charles Fiske being dead in Alone in the Dark 4: The New Nightmare (DarkWorks). In their theory on Perceiving and Experiencing Fictional Characters (PEFiC), hence, Konijn and Hoorn (2003) rather speak of a measure of involvement with the character, which co-occurs with a measure of distance; a healthy distance, which protects the human emotional system against too threatening but also too desirable situations and characters. Konijn and Hoorn provide evidence on film characters that involvement and distance are not two mutually excluding experiences. On the contrary, high appreciation of a fictional character flows from a ratio between involvement and distance. Such apparent contradictions often are found in game reviews as well:

19

Konoko, the girly heroin of the action game Oni has everything to become a new icon in the game world. Not like Lara Croft because of her spectacular looks but exactly through the combination of youthful innocence and the brute violence she is capable off. That developer Bungie did not choose for the obvious way to make her super-sexy only increases the sympathy for this girl. (Bartelson, 2001) (author translation and italics)

Figure 2. Alice, but not in Wonderland (left panel). Konoko, brutal force and youthful innocence (right panel) (http://www.3dactionplanet.com/oni/images/konoko_rogue.jpg). Two models for more motivating types of agents?

Figure 3 summarizes the PEFiC theory in explicating how involvement and distance transpire and how their interrelationship determines the appreciation of the character. Relying on cognitive memory theories (Raaijmakers & Shiffrin, 1981) and similarity studies, Konijn and Hoorn claim that not only recognition of the self in the fictional character but also the memory of one’s own vicissitudes, or situations one has seen or heard off, bring the observer closer to the character. The measure of (inner) resemblance between observer and character, including their respective situations, partly mediates the measure of involvement. By the same token do the differences partly mediate distance. Partly, because it is not as simple as counting the similarities and differences. In the light of their personal goals and concerns, observers determine how relevant particular events concerning and features of the fictional character are. That Hana is bisexual is probably more important to people with the same preference than to others. That Masami von Weitsaecker (Ring of Red, KCE Studios) is from Japanese-German descent leaves everybody indifferent, unless you underwent the attack on Pearl Harbor. Subsequently, the features of and the events concerning the fictional character are appraised for their ethics (a good hero or a bad villain) (Paiva et al., 2001), their aesthetics (is she a beauty or an ugly witch), and their epistemic value (how realistic or unrealistic are they?) (Shapiro & McDonald, 1992). Like mediated persons, voice-based information agents remain in the area of realistic representations. Programmable plush dolls and stuffed animals that control synthetic characters in animated environments (Umaschi, 1996; Johnson, Wilson, Blumberg, Kline, & Bobick, 1999) recede at the unrealistic end of the scale. It is easy to imagine that the tennis star Anna Kournikova evokes involvement because of her attractive looks, but when she is the cover-up of a virus (subject: Here you have ;-o) in an e-mail attachment (Anna-Kournikova.jpg.vbs), she yields quite as much distance. Likewise, a character like Black Magician Vivi (Final Fantasy IX, SquareSoft) may or may not be appreciated for her highly imaginative or otherwise unnatural peculiarities. But that’s not all there is to it. Nobody lives in isolation and no one is so self-willed that s/he is immune to the opinion of others. Social psychology teaches that people like to belong to particular groups (e.g., Black versus White Hat hackers) and adhere to the (cultural) norms of those groups even when they do not agree on them all that much (Grey Hats). The same is valid, of course, for Internet surfers, game addicts, UNIX gurus, film freaks, art lovers, and other connoisseurs. In the PlayStation camp (Sony), it is not allowed to positively value the Pokémon characters (Nintendo). As a child psychologist, you must cry out against the violence of shooter games and yet buy your kid Point Blank 3 (Namco) for Xmas. Thus, the socially accepted judgment over a character (“Dan & Don are a bunch of aggressive yahoo’s”) may contest with what one silently thinks

20

(“Dan & Don are my quirky heroes”). Normative behavior of agents (Machado et al., 2001) and social competence (Prendinger & Ishizuka, 2001), then, could be part of an agent’s design. All those dimensions of appraisal can lead to a tendency to approach a fictional character (“youthful innocence” hints at a friend) and a tendency to avoid (“brute violence” hints at a foe). This simultaneous occurrence of a positive and negative ‘valence’ (that is, ‘ambivalence’) suggests that selfrecognition without more does not necessarily awaken involvement with the character. The lesson learned from this for avatars is that mere resemblance is insufficient for user involvement. If someone recognizes him or herself in the revenge of Edward Carnby, the resemblance may lead to involvement but that the trait is negatively appraised leads to that extraordinary contradictory experience of distance. With the PEFiCmodel, it can now be explained why, for example, the wish ‘to be as cool as Lara Croft’ stirs up mixed emotions, which all too often may confuse young adolescents. The fact that you are not as cool (the differences) contributes to the experience of distance, whereas the positive appraisal of those differences leads to involvement.

ENCODE

COMPARE

Involvement

Features of situation and Fictional Character

% Ethics

good

dissimilar

beautiful

irrelevant

realistic

negative valence

Aesthetics

Epistemics

Norm

RESPOND

Appreciation bad

similar

ugly

relevant

unrealistic

positive valence

%

Distance

Figure 3. Model of perceiving and experiencing fictional characters, including agents. Features that are relevant to the goals and concerns of the user evoke emotions. Distinctive features that are appraised positively and intersecting features that are appraised negatively increase both involvement and distance (see %-sign). If bad features are appraised positively, an approach tendency may occur, whereas a negative appraisal of good features may cause an avoidance tendency. The entire process is doubled if the norms of the individual are contrasted with the norms of the socio-cultural group (“What do significant others think of it?”).

The PEFiC-model gives an integrative account of how appreciation of fictional characters in literature, film, TV, and digital media is a trade-off between the parallel processes of involvement and distance. Given the relevance and valence (positive-negative) of a situation, involvement and distance supposedly flow from ethic (good-bad), aesthetic (beautiful-ugly), and epistemic (realistic-unrealistic) appraisals of the features of the fictional character, thereby considering the (dis)similarity between the fictional character and the observer (Figure 3). PEFiC can be considered an overall model of character engagement in virtual or fictional environments. In the case of agents, it is the application that matters and makes the difference. Involvement with the character may be the goal in a game more than in a training environment such as a driving simulator, where keeping your cool (distance) is more important. In tele-presence involvement probably is

21

more important than in remote-control scenarios such as long-distance surgery, where a physician should keep his/her mind together. Yet, PEFiC can account for these different constellations of involvementdistance trade-offs. The main shifts probably can be traced back in the change of goals for using the application, which will narrow down the task-relevance of the agent.

7.4

Reality Perception

The ASPASIA-project Reality Perception, Affect, and Information Value of Mediated Characters is a personal grant awarded to Elly A. Konijn at FSW (NWO File number: 015.000.019). It concerns the perception of reality and the evoked emotions in response to mediated persons, such as characters in soap operas and docudramas. It is expected that, in general, exposures to media fare might get a high information value for its viewers as its perceived realism and/or the emotional response increases. This will probably hold for fiction in particular. With the introduction of reality TV and emotion TV, media broadcasters attempt to increase emotional involvement of the receiver with the program so to raise appreciation and average viewing time. The assumption supposedly is that realistic representations and emotional expressions enhance emotions or involvement in its receivers and the better it will be appreciated. However, expressed emotions by senders do not necessarily evoke (the same) emotions in receivers. On the other hand, experiencing emotions, particularly physiological changes accompanying them, are seen as ‘proof of reality’ by individuals (Izard, 1984; Lazarus, 1991; Reeves & Nass, 1996). Furthermore, studies reveal that receivers’ emotional experiences frame subsequent information (Nabi, 2003), lead to recall errors (Brosius, 1993), motivate selective information processing (Forgas, 1995), and cause information processing errors in taking fiction for real (Prentice & Gerrig, 1999). Therefore, this intriguing issue and the effects of (experienced vs. expressed) emotions on the perceived reality and subsequent information processing have demanded further investigation. The current mediated society underscores the relevance of the research question. After all, information delivered via the mass media influences our real life knowledge structures, even if it concerns fiction (e.g., Busselle, 2001; Prentice & Gerrig, 1999). Focusing on mediated persons, several authors argued that mediated persons (presenter, actor, fictional character) are able to set up (parasocial) or unidirectional relationships with the receiver (e.g., Horton & Wohl, 1956; Rosengren & Windahl, 1972; Meyrowitz, 1985). A recent review of the literature argues that the concept of parasocial interaction (PSI) should be theoretically and empirically elaborated and taken up by psychologists (Giles, 2002). Although several authors discussed the correspondence between PSI and interpersonal communication (e.g., Rubin, Perse, & Powel, 1985; Perse & Rubin, 1989), an integrative account of the relevant (psychological) literature on interpersonal communication with mediated persons has not yet been found. Hoorn and Konijn (2003) integrated and refined important factors from interpersonal communication into a unifying model of engaging mediated persons as a framework for future experimental work. Their findings concerning fictional characters (Konijn & Hoorn, 2003) and politicians (Konijn, 2003) are in line with Rosengren and Windahl (1972), who saw PSI not as a separate variable but as part of involvement, in which ‘being captured’ is the highest state (cf. identification; see also Cohen, 1999). Involvement and distance appeared parallel processes and together explain appreciation for the mediated person. In computer-mediated communication and human-computer interaction, the concept of mediated interpersonal communication seems well-established (Barnes, 2001; Gumpert & Cathcart, 1986; Walther, 1996). Reeves and Nass (1996), for instance, claim and provide ample evidence that people treat their computers, television, and new media like real people and places. However, even realistic mediated persons are mediated, and thus, liable to editing, cut, focal plane, and directors’ choices; they are not entirely real but partly fictional and partly construed by the laws of fiction. The ASPASIA studies address the following questions: When receivers attribute more reality to the mediated persons, do the get more emotionally involved with and attribute more information value to the mediated persons? Do emotions ‘expressed’ by mediated persons affect the perceived reality, the emotional experiences, and information value of a program’s contents? Is a documentary perceived as more realistic when we feel fear and anxiety than when we are in a tranquil, reflective state? The ASPASIA studies combine media effects research into TV with theories of computer-mediated communication and human-computer interaction. These disciplines integrate and apply cultivation, uses & gratifications, parasocial, and interpersonal communication theories to mediated, unidirectional communication with mediated persons. To prepare stimuli and structured questionnaires for experimental purposes, in depth interviews are carried out while participants watch televised segments of reality-based

22

and fictional genres. Four experimental designs are testing the effects of the manipulated reality status of mediated persons, including their emotional expressions, on the levels of involvement, reality perception, and information value. Exp. 1 concerns manipulating ‘reality contexts’ (cf. framing or priming) within which the mediated persons operates. Information presented within a fictional context (compared to a realistic) might have higher information value when subjects are unfamiliar with the represented persons and contents (cf. Slater, 1991). Exp. 2 concerns manipulating reality aspects or cues (based on the aforementioned interviews) within fictional contexts of mediated persons (e.g., mixing genre-conventions). It is expected that the reality perception will be judged relative to the genre’s conventions, which may explain curious findings such as a lower reality perception for politicians than for Hollywood characters (Konijn, 2003). Further, more realistic mediated persons, within fictional contexts, should evoke stronger emotional involvement and higher information value. Exp. 3 concerns manipulating emotional expressions of mediated persons. Assumingly, ‘emotional’ mediated persons positively affect reality perception, emotional involvement and information value. Exp. 4 manipulates the emotional experiences of receivers, with contrasting expectations. Emotionally involved receivers are motivated to centrally process the mediated message (Prentice & Gerrig, 1999). Thus, they should be better able to scrutinize the reality status of the arguments. Conversely, emotionally involved receivers are considered less capable, i.e. make more errors (cf. limited capacity theory) in taking fiction for real and correspondingly enhance information value.

7.5

Intelligent Multimedia @ VU

For the technical realization of the experiments, it is essential that the VU research group of Intelligent Multimedia (IM) is involved in the project. This group is developing a high-level platform for 3D and rich media virtual environments based on agent-technology, using the languages DLP, Java, and VRML. On top of this platform, this VU research team has developed a scripting language STEP for specifying humanoid movements and gestures, based on dynamic logic. The goal of IM is to study aspects of the deployment and architecture of virtual environments as an interface to multimedia information systems, such as the Web. The platform supports embodied conversational agents, see Eliëns, Huang, & Visser (2002). As demonstrators, IM has developed a distributed soccer-game prototype with intelligent autonomous avatar-embodied agents as players (Huang, Eliëns, & Visser, 2002), a humanoid animation demonstrating Tai-Chi (Huang, Eliëns, & Visser, 2003a), avatars presenting a dialog in a mixed media presentation environment (Eliëns, Dormann, Huang, & Visser, 2003), a domestic agent that can be addressed in natural language (Hildebrand, Eliëns, Huang, & Visser, 2003), an avatar reaching for objects that uses reasoning and inverse kinematics (Huang, Eliëns, & Visser 2003b), and an avatar conductiong music (Ruttkay, Huang, & Eliëns, 2003). As a student project in the Casus Practicum, a 3D presentation for INCCA is developed, the International Network for the Conservation of Contemporary Art, in cooperation with ICN, the Dutch Cultural Heritage Institute. The research has been supported by two NWO projects, WASP (Web Agent Support Program) and RIF (Retrieval of Information in virtual worlds using Feature detectors). The combined effort of these projects led to the DLP+X3D platform and the development of the STEP language. The work on embodied conversational agents is being done in cooperation with Z. Ruttkay from CWI. The work on the cultural heritage application is done in cooperation with T. Scholte from ICN. In preparation is a prestigious book on Life-like Characters, for which the VU contributed a chapter describing the platform and STEP (Huang, et al., 2003a). Complex humanoid gestures are of a highly parallel nature. The STEP scripting language supports a direct way of modeling parallel gestures by offering a parallel construct (par), which results in the simultaneous execution of (possibly compound) actions (Figure 4). To avoid unconstrained thread creation, the STEP engine makes use of a thread pool, containing a fixed number of threads, from which threads are allocated to actions. Once the action is finished, the thread is put back in the pool. This approach works well for most examples. However when many threads are needed, as in the conductor example (which requires approximately 60 threads), problems may occur, in particular when there are may background jobs.

23

Figure 4. STEP Testbed consists of an action builder and a script text area. Retrieved September 10, 2003, from http://wasp.cs.vu.nl/step/images/toolrm.jpg

The agent model may be characterized as a BDI-model, extended with sensors and effectors needed for the interaction in a virtual environment. The STEP scripting language has been developed to facilitate the specification of communicative acts, like gestures. However, it would be worthwhile to explore text-to-speech synthesis as an extra modality of communication as well. Another interesting research issue is how to specify a reusable library of gestures, accomodating for differences in (personal) style. This is currently being investigated by Z. Ruttkay from CWI. Yet another issue is the use of inverse kinematics to grasp objects. To solve the problem of reliable timing would require not only a modification of the STEP engine, but also a rather different implementation of the DLP threads supporting the parallelism in STEP. Currently, the implementation only allows for best effort parallelism and does not provide the means for deadline scheduling. However, it seems that the utmost efficiency feasible has been reached within the Java platform. Therefore, the research team considers to redevelop the DLP+X3D platform in a .NET environment. An additional advantage of migrating to the .NET environment would be the possible integration of functionality such as text-to-speech synthesis which is not readily available in the Java environment.

7.5.1 WASP WASP (Web Agent Support Program) is an NWO project to enable average users to keep track of relevant information on the Web. In a relatively short time, the Web has become a de facto standard for the dissemination and retrieval of information. Due to the exponential growth of the Web and the information it provides, finding relevant information has become more and more difficult. In particular, browsing is in most cases no longer appropriate for the user searching for specific information. It is our view that, in the near future, access to the Web will increasingly be mediated by intelligent helper applications, software agents, that assist the user in finding relevant and interesting information. The goal of the WASP project is to provide support for developing such agents. By combining our joint expertise, we plan to develop a framework that encompasses modeling aspects as well as the realization of software assistants for intelligent Web access. As testified by our publications, our expertise covers the realization of Web-aware applications, client- and server-side support

24

for information retrieval and information maintenance, and modeling and specification of cooperative agents in a multi-agent framework. Combining our experience and the technology that resulted from our previous research activities enables us to develop a practical framework for designing and implementing intelligent Web agents and to tackle the research issues involved in the definition and realization of personal software assistents that help the average user to disentangle the complexities of the Web. The WASP project is envisaged to result in a framework providing support for: Intelligent navigation and information retrieval, information and document maintenance, user interfaces for Webaware applications, dynamic documents with user-defined applets, declarative descriptions of agentbehavior based on user-preferences, declarative modeling of coordinated and cooperative behavior of software agents, and programming single and multi-agent systems. As a target product for the WASP project, which allows us to demonstrate our results to the scientific community and other interested parties, we envisage to develop Pamela (Personal Assistant for Maintaining Electronic Archives), an application combining the functional and architectural features mentioned above.

7.5.2 RIF RIF, Retrieval of Information in Virtual Worlds Using Feature Detectors, is an NWO project (# 612-61-607) that is transferred to the VU in January, 2001. Based on the observation that multi-user virtual worlds are becoming an increasingly important phenomenon in the Internet-based Cyberspace (see for example www.blaxxun.com and www.colonycity.com), RIF proposes to investigate the indexing and retrieval problem for multi-user virtual worlds. Our research is meant to extend the ACOI framework for the indexing and retrieval of multimedia objects by providing special purpose feature detectors for virtual worlds, that fit within the ACOI architecture. The ACOI architecture is built around a Monet database that stores information concerning the structure and contents of multimedia objects available on the World Wide Web. The assumption underlying the indexing and retrieval model underlying the ACOI framework is that the structure and contents of multimedia objects may be expressed by a grammar that is augmented with media-specific detectors for analysing the object. These detectors act as token generators, providing the information to be stored in the Monet database. For the construction of feature detectors for multi-user virtual worlds, we need to develop the technology to analyse world description files and VRML encodings of the geometric structure and objects contained in a world, and an ontology or knowledge representation to store and reason about the contents of the world and its relation to other worlds. Further we need to investigate how we can assist the end user in formulating and refining queries with respect to locations within a world, informational resources contained within a world, and navigational preferences, that is the wish to migrate to different worlds. The initial approach will be to extend the musical feature detector, see MIDI, which is based on a combination of feature grammars and descriptive logic, to the contents of virtual world. In a later stage, we will combine our virtual world feature detector with the other multimedia feature detectors developed within the ACOI framework. To validate our approach, we intend to create a small virtual world ourselves, in order to establish the effectiveness of our indexing and retrieval method. RIF research will result in actual feature detectors running within the ACOI framework, a knowledge representation or ontology for collecting information about virtual worlds, and appropriate applets to formulate queries and deliver a report of the outcome of consulting the ACOI index database.

25

8

References

Barnes, S. (2001). Online connections: Internet interpersonal relationships. Cresskill, NJ: Hampton Press. Bartelson, E. (2001). Review Oni. PlayNation 3, 18, 58-61. Beringer, J. (1992). Entwerf einer Anwendersprache zur Steuerung psychologischer Reaktionszeitexperimente. Peter Lang, Frankfurt am Main. Bonito, J. A., Burgoon, J. K., and Bengtsson, B. (1999). The role of expectations in human-computer interaction. In Proceedings of the international ACM SIGGROUP conference on supporting group work (pp. 229-238). New York: ACM Press. Brosius, H. (1993). The effects of emotional pictures in television news. Communnication Research, 20, 105124. Busselle, R. W. (2001). Television exposure, perceived realism, and exemplar accessibility in the social judgment process. Media Psychology, 3, 43-67. Catrambone, R. (2002). Anthropomorphic agents as a user interface paradigm: Experimental findings and a framework for research. In: Proceedings of the 24th Annual Conference of the Cognitive Science Society (pp. 166-171), Fairfax, VA, August 2002. Choi, Y. K., Miracle, G. E., & Biocca, F. (2001). The effects of anthropomorphic agents on advertising effectiveness and the mediating role of presence. Journal of Interactive Advertising, 2 (1). Cohen, J. (1999). Favorite characters of teenage viewers of Israeli serials. Journal of Broadcasting and Electronic Media, 43, 327-345. Dehn, D. M., & Van Mulken, S. (2000). The impact of animated interface agents: A review of empirical research. International Journal of Human-Computer Studies, 52, 1, 1-22. Eliëns A., Huang Z., & Visser C. (2002). A platform for embodied conversational agents based on distributed logic programming. In AAMAS Workshop - Embodied conversational agents - let’s specify and evaluate them! Bologna, Italy. Eliëns A., Dormann C., Huang Z., & Visser C. (2003), A framework for mixed media - emotive dialogs, rich media and virtual environments. In S. Göbel, N. Braun, U. Spierling, J. Dechau, & H. Diener H. (Eds.), Proceedings of TIDSE03, 1st Int. Conf. on Technologies for Interactive Digital Storytelling and Entertainment, March 24-26, 2003. Darmstadt, Germany: Fraunhofer IRB Verlag. Frijda, N. H. (1986). The emotions. New York: Cambridge University. Foltz, P. (2003). Content processing with Latent Semantic Analysis: Essays, teams and clinical interviews. In L. Degand (Ed.), Determination of information and tenor in texts: Multidisciplinary approaches to discourse (pp. 179-188). Münster: Nodus. Forgas, J. P. (1995). Mood and judgment: The affect infusion model (AIM). Psychological Bulletin, 117, 39-66. Giles, D. C. (2002). Parasocial interaction: A review of the literature and a model for future research. Media Psychology, 4, 279-305. Gumpert, G. & Cathcart, R. (1986, 3rd ed.). Inter media: Interpersonal communication in a media world. New York: Oxford University Press. Heuvelman, A. (1989). Buiten beeld: Onderzoek naar de invloed van vormgevingsaspecten van educatieve tvprogramma's op cognitieve processen bij de kijkers. Enschede: proefschrift Universiteit Twente. Hildebrand, M., Eliëns A., Huang Z., & Visser C. (2003). Interactive agents learning their environment. In R. Aylett, D. Ballin & T. Rist (Eds.), Proceedings of the 4th International Working Conference on Intelligent Virtual Agents (IVA’03), Lecture Notes in Artificial Intelligence 2792 (LNAI 2792). New York, Heidelberg, Germany: Springer-Verlag. Hoorn, J. F., & Konijn, E. A. (2003). Perceiving and experiencing fictional characters: An integrative account. Japanese Psychological Research, 45, 4, 221-239. Hoorn, J. F., Konijn, E. A., & Van der Veer, G. C. (2003a, February 15). Virtual reality: Do not augment realism, augment relevance. Upgrade - Human-Computer Interaction: Overcoming Barriers, 4, 1, 18-26. Retrieved February 15, 2003, from http://www.upgrade-cepis.org/issues/2003/1/up4-1Hoorn.pdf Hoorn, J. F., Konijn, E. A., & Van der Veer, G. C. (2003b). Rationale for REquest, a structured questionnaire for general requirements validation. Manuscript submitted for publication. VU, Amsterdam. Horton, D. & Wohl, R. R. (1956/1986). Mass communication and para-social interaction. Observation on intimacy at a distance. In G. Gumpert & R. Cathcart (Eds.), Inter media: Interpersonal communication in a media world (3rd ed., pp. 185-206). New York: Oxford University Press. Huang, H.-Y. (1999). The persuasion, memory and social presence effects of believable agents in human-agent communication. Proceedings of the Third International Cognitive Technology Conference CT'99 (San Francisco/Silicon Valley, 1999). Huang, Z., Eliëns, A., & Visser, C. (2003a). STEP: A scripting language for embodied agents. In H. Prendinger, & M. Ishizuka (Eds.), Life-like characters, tools, affective functions and applications. New York, Heidelberg, Germany: Springer.

26

Huang, Z., Eliëns, A., & Visser, C. (2003b). “Is it within my reach?” - An agents perspective. In R. Aylett, D. Ballin & T. Rist (Eds.), Proceedings of the 4th International Working Conference on Intelligent Virtual Agents (IVA’03), Lecture Notes in Artificial Intelligence 2792 (LNAI 2792). New York, Heidelberg, Germany: Springer-Verlag. Huang Z., Eliëns A., & Visser C. (2002). 3D agent-based virtual communities. In W. Wagner & M. Beitler (Eds.), Proceedings of the International Web 3D Symposium (pp. 137-144). New York: ACM Press. Izard, C. E. (1984). Emotion-cognition relationships and human development. In C. E. Izard, J. Kagan, & R. Zajonc (Eds.), Emotions, cognition, and behavior. New York: Cambridge university Press. Joachims, T. (2002). Learning to classify text using support vector machines. Boston/Dordrecht/London: Kluwer. Johnson, M. P., Wilson, A., Blumberg, B., Kline, C., & Bobick, A. (1999). Sympathetic interfaces: Using a plush toy to direct synthetic characters. In Proceeding of the CHI 99 conference on Human factors in computing systems: The CHI is the limit (pp. 152-158). New York: ACM Press. Johnston, V. S., & Oliver-Rodriguez, J. C. (1997). Facial beauty and the late positive component of event-related potentials. Journal of Sex Research, 34, 2, 188-198. Kleinnijenhuis, J. (2003). Automatic retrieval of semantic (dis)similarities? In L. Lagerwerf, W. M. P. S. Spooren, & L. Degand (Eds.), Determination of information and tenor in texts: Multidisciplinary approaches to discourse (pp. 203-214). Münster: Nodus. Kleinnijenhuis, J., de Ridder, J. A., & Rietberg, E. M. (1997). Reasoning in economic discourse: An application of the network approach to the Dutch press. In C. W. Roberts (Ed.), Text analysis for the social sciences: Methods for drawing statistical inferences from texts and transcripts (pp. 191-207). New York: Erlbaum. Koda, T., & Maes, P. (1996). Agents with faces: The effect of personification. Proceedings of HCI’96. Konijn, E. A. (2003). Interpersonal judgments of Bush, Blair, Bin Laden, and Hussein at the eve of the Iraqi war 2003. Fitting the main characters on the world stage to the PEFiC-model [work title]. Amsterdam: Dept. of Communication Science, Faculty of Social Sciences, Free University. (manuscript in progress). Konijn, E. A., & Hoorn, J. F. (2003). Some like it bad: Testing a model for perceiving and experiencing fictional characters. [Manuscript accepted for publication after revision]. Lagerwerf, L., Spooren, W. M. P. S., & Degand, L. (Eds.). (2003). Determination of information and tenor in texts: Multidiscplinary approaches to discourse. Münster: Nodus. Litkowski, K. C. (2000). Syntactic clues and lexical resources in question-answering. Ninth Text REtrieval Conference(TREC-9). Gaithersburg, MD. November 13-16, 2000. Lazarus, R. S. (1991). Emotion and adaptation. New York: Oxford University Press. Machado, I., Paiva, A., & Prada, R. (2001). Is the wolf angry or… just hungry? In Proceedings of the fifth international conference on autonomous agents, 2001, Montreal, Quebec, Canada (pp. 370-376). New York: ACM Press. McGuire, W. J. (1989). Theoretical foundations of campaigns. In Rice, R. E., & Atkin, C. K. (Eds.), Public communication campaigns (2nd ed.) (pp. 43-65). Newbury Park, CA: Sage Publications. Meyrowitz, J. (1985). No sense of place. New York: Oxford University Press. Nabi, R. L. (2003). Exploring the framing effects of emotion. Do discrete emotions differentially influence information accessibiliy, information seeking, and policy preference? Communication Research, 30, 2, 224247. Nowak, K. L. (2003). The effect of the agency and anthropomorphism on users’ sense of telepresence, copresence, and social presence in virtual environments. Presence, 12 (5). Perse, E. M. & Rubin, R. B. (1989). Attribution in social and parasocial relationships. Communication Research, 16, 59-77. Prentice, D. A., Gerrig, R. J. (1999). Exploring the boundary between fiction and reality. In S. Chaiken, & Y. Trope (Eds.), Dual-process theories in social psychology (pp. 529-546). New York: Guilford. Quittner, J. (1995, April). Autonata non grata. Wired, 119-122, 172-173. Raaijmakers, J. G. W., and Shiffrin, R. M. (1981). Search of associative memory. Psychological Review, 88, 93134. Reeves, B., & Nass, C. (1996). The media equation: How people treat computers, television, and new media like real people and places. Cambridge, UK: Cambridge University Press. Rickenberg, R., & Reeves, B. (2000). The effects of animated characters on anxiety, task performance, and evaluations of user interfaces. Proceedings of the CHI 2000 conference on Human factors in computing systems April 2000. Rosengren, K. E. & Windahl, S. (1972). Mass media consumption as a functional alternative. In D. McQuail (Eds.). Sociology of mass communication. (pp. 166-194). Middlesex, England: Penguin. Rubin, A. M., Perse, E. M., & Powel, R. A. (1985). Loneliness, parasocial interaction, and local television news viewing. Human Communication Research, 12, 155-180.

27

Ruttkay Z., Huang Z., & Eliëns A. (2003). The conductor: Gestures for embodied agents with logic programming. In Joint Annual ERCIM/CoLogNet Workshop on Constraint and Logic Programming, 30 June - 2 July, 2003. Budapest, Hungary. Schaumburg, H. (2001). Computers as tools or as social actors? The users' perspective on anthropomorphic agents. International Journal of Cooperative Information Systems. 10, 1, 2, 217-234. Shapiro, M. A., & McDonald, D. G. (1992). I’m not a real doctor, but I play one in virtual reality: Implications of virtual reality for judgments about reality. Journal of Communication, 42, 4, 95-114. Street, R. L., Gold, W. R., & Manning, T. (Eds.) (1997). Health promotion and interactive technology: Theoretical applications and future directions. Mahwah, NJ: Erlbaum. Umaschi, M. (1996). SAGE storytellers: Learning about identity, language and technology. In ICLS 96 Proceedings, AACE, 526-531. Van Cuilenburg, J. J., Kleinnijenhuis, J., de Ridder, J. A. (1988). Artificial intelligence and context analysis. Quality and Quantity, 22, 65-97. Van der Veer, G. C., Lenting, B., & Bergevoet, B. (1996). GTA: Groupware Task Analysis - Modeling complexity. Acta Psychologica, 91, 3, 297-322. Van der Veer, G. C., & Van Welie, M. (2000). Task based groupware design: Putting theory into practice. In Conference proceedings on designing interactive systems: Processes, practices, methods, and techniques, 17-19 August 2000, New York (pp. 326-337). New York: ACM Press. Velasquez, J. (1997). Modeling emotions and other motivations in synthetic agents. Proceedings of the AAAI Conference 1997, Providence, RI, 10-15. Walther, J. B. (1996). Computer-mediated communication: Impersonal, interpersonal, and hyperpersonal interaction. Communication Research, 23, 1, 3-43.

28