Towards a Design Methodology for Adaptive Applications - CiteSeerX

Towards a Design Methodology for Adaptive Applications Malcolm McIlhagga, Ann Light and Ian Wakeman School of Cognitive and Computing Sciences University of Sussex Brighton BN1 9QH August 21, 1998

Abstract

rise of middleware has generated renewed interest in providing generic adaptation policies, but we believe that the interfaces to manipulate resource requirements are phrased in the wrong vocabulary [4]. Recently there has been an attempt at defining object design patterns [19], but these do not address the holistic application. The work on the Glomop [3] and Odyssey[17] architectures are closest in approach to our work. However, we have focussed on developing a policy control architecture rather than developing a salable solution, and in particular in generalizing to an architecture of adaptive applications. In this paper, we provide a number of design choicw which should be explicitly addressed when designing m adaptive application. We believe these design choice are common across all adaptive applications, and are fundamental in determining how successful the application is in adapting to changing resources yet still providing utility to the user. ~er explaining the design choices, we illustrate their use in the design of an Image Proxy for the WWW. A feamre of adaptive applications is that they are no longer purely systems design - they interact with users directly, and wc believe that user studies must be undertaken to determine the appropriate interface and behaviors for the application. We finish by d~cribing some of the political issues that arise when adaptive applications are used. As a side product of this work, we have empirically determined appropriate quality factors for JPEG images, which are much lower than those produced by standard scanner software. We show that if JPEG images are reduced to the threshold quality factors, then image file sizes can be significantly reduced, lightening the ever-increasing load on the Internet.

We describe an abstract architecture of adaptive applications, and indicate where we believe crucial design decisions must be made. We illustrate the use of the abstract model in the design and implementation ofan image proxy, and describe the studies required in determining the appropriate design points. In particular, even though adaptation to resource constraints is generally considered a systems probleu the adaptation is visible to the user in changes in utility, and so the user must be involved in designing the application. Finally, we discuss the politics that creep in when dwigners change the semantics of applications.

1

Introduction

h an age of cheap mobile compute devices, we would like an ubiquitous computing environment [21] in which applications can follow us around as we move from computing interface to interface, and from device to device. However, because of the well-known constraints of battery power, display capabilities and network characteristics for mobile computing, the resources are not equally available across all platforms. To allow applications to scale across the platforms, we need the applications to adapt to the changing constraints of the resources available in the environment. There have been a number of adaptive applications, ranging from those attempting to alleviate the effects of congestion and limited bandwidth, such as vat [5], vic [15], ivs [6] and rat [18], to User Interface Management Systems such as Amulet [16] which attempt to adapt to the display capabilities through battery management schemes within the operating system. The

2

Pemlissionto makedigitalor hardcopiesof all or part of this !vorkfor perjonalor classroomuse is granted \\ithout fee provided that copies

Adapting to Constraints: Choices in Application Design An adaptive application is one in which the application changes its behaviour according to the perceived constraints in Ille envitnnnlent, so fl,rto nlaintain the semantics of the application for the lise~

are not made or distributed for prolit or commercial advantage and that copies bLwrthis notice and tbc full citation on the first page. To copy othen~ise. to republish. to post on senrers or to redistribute to lists, requires prior specific permission and~ora fee.

NfOBICOM 9S Dollas Texas USA Copfight AChf 19981-581 13435-ti98/10... S00OO

If we take the above definition as our starting point, then we 133

—---

-

..-.

..

.. . . .

....

...—

I

open implementation [7] from software engineering enables the designers of components to open up the implementation of their component so that it can be adjusted and adapted to suit various needs. The behaviour of a component should be described by its interface abstractions, but the implementation is generally hidden behind the interface. Following the design guidelines of open implementation (01) [13], the designer of the component will offer other meta interfaces through which the programmer can adjust the implementation of the component. If we are aiming to produce distributed applications which scale across network and display capabilities, then 01 offers a suitable sofhvare engineering approach to enable this scalabili~. Simply put, the 01 part of a software component exposes the network requirements of a particular implementation of a component. By allowing the adaptation controller to manipulate the implementation of a component, the controller can adjust the network requirements of the component and thus adapt the component to the prevailing nehvork conditions. Following the guidelines in [S], the designer of an open implementation interface should attempt to:

User Utility

Application Semantics

I I I

t

o%

-----r

------

------

01

01

I A&ptio r I ontroll 1 I I I Application

I

s----

-----

01

u

Componenb -----

I

I I

I I

I I I I

I I

__/

Resource Constraints (network, display, battery...

1. Define the abstract black box intetiace, which instantiates the “usefi~’ behaviour of the component.

Figure 1: A Layered View of the Adaptive Application

2. Using the domain knowledge of both the inherent implementation of the component and of how clients will use the component, define interfaces which allow the client to control implementation strategies.

can immediately break the problem of designing an adaptive application into various stages. The constraints in the environmentfor the application should first be determined. For mobile computing applications, the likely constraints are network quality of service, battery power and display capabilities. We must next define the semantics of the application. What is the application trying to achieve? This is user and task specific, and can vary from context to context. As we will argue, the more generic the application, the more difficult it is to determine the application semantics. Having made a best guess at the semantics, a design goal is to ensure that the semantics of the application remain invariant across the constraint-reacting behaviors of the application. By doing so, we hope that the utility of the application to the user remains high. In order to maintain the semantics, the changm in behaviour of the application should ensure that the application continues to work whilst adapting to the changing constraints. If the behaviour is chrmging to adapt to a restriction of some needed resource, such as bandwidth, the change should itiict the minimum damage upon the essential functionality of the application. This leads us to a solution in which the implementation of the application should be opened up so as to allow the choice of some equivalent but less resource-hungry implementation. We thus come to the first design point in our methodologyDesign Choice: Providing an Open Implementation of the components in the Underlying Technology The concept of

The inherent implementation of the component is based around the abstractions that will be used to implement the component. For the case of the file caching mechanism described in [ 12], it is the disk buffers and caches of a file system. For the network retrieval architecture described in Section 3, the abstractions are the compression schemes and representations of the various forms of multimedia. Since the abstractions used in understanding the implementation of a component are closer to the domain of the application, the designer will be able to better tune the adaptation of the application. Design Choice: Determining the Degrees of Freedom in degradation trajectories The 01 interfaces of the components will normally provide a discrete set of implementations upon some variation. Imagine a simple speech tool which provided a set of encodings to use for speech. As the degree of compression in the encodings gets higher, the bandwidth requirements decrease but the quality of the resultant speech is decreased, thereby decreasing the utility of the application. For more complex applications, each component may provide other choices of implementation which are orthogonal to each other in that they can reduce resource usage in different ways. If we can add redundancy to the speech encodings, using some of our bandwidth to provide data to repair lost packets at the expense of greater latency, then we provide another set of choices of about how to use valuable bandwidth.

134

-

..—- -

.-%-— ~

\-, .

,,

.-.,-

.

..-.

———

If we regard ~ch component as supplying an 01 interface which can be adjusted independently of the other components, the designer thus has a choice over each component of an implementation. The number of independent axes which contain implementation choices is the degree of freedom of the application. Adjusting the Open Implementation of a component can be viewed as placing the component at some new point in the space of (implementation choice, resource usage, application utili~) space. As we select a new set of implementation choices to decrease the resource usage of a component, we generally decrease the utility of the overall application. Thus for each of the possible implementations of the component, we could theoreticrdly maure the utility of the resultant application. The designer must understand this space of possible implementations, for it is from this set of implementation choices that the designer chooses a path of degradation in the face of resource constraints. Design Choice: How transparent should the degradation be? Having plotted the implementation space, the designer must next decide how the resource usage is to be monitored and controlled. For networked applications, this has generally been the congestion signal such as a packet loss, or for processor sensitive applications, by examining the process queue length [9]. In our abstractions, the control is through a generic adaptation controller, but this is generally integrated with some other component. The designer must next decide what the resource availabilities upon which to switch implementations, and whether this should include the user within the loop or not. Systems which exclude the user and use closed loop feedback need to worry about the stability of the control loop, and the effect that varying application utility will have upon the user. Systems which include the user must be certain that the user is educated about the need for their collusion in adapting the application. Systems in the latter case include batte~ monitors querying the user about whether to close down applications, or video conferencing system in which adaptation would reduce the quality to less than the minimum demanded [6]. Dwign Choice: Selecting Run-time versus Dwign-time Bhaviour Specifications Having investigated the implementation space and decided upon how degradation should be controlled, the designer must next decide when the possible trajectories through the implementation space are decided – when designing the application or through interpreting some later set of instructions. Design-time behaviors include the use of reactive protocols in the face of congestion, such as in the video conferencing tools vie and ivs. Th=e are fixed policies of degradation, or fied trajectori~ through the implementation space. In general design-time policies are easier to encode. However, as the applications become more generic and are used for more disparate tasks, then the semantics of the application are increasingly determined by the context of use, 135

———

and the utility of the application is difficult to pinpoint. An application which simply reports the state of some object has simple semantics and will only be used in a very limited set of contexts, and thus the behaviour can be pre-determined. The user can either have an update or not. On the other hand, a web browser is used by users in many different ways [10, 14], and the semantics of the retrieved pages depend highly upon context. As dwigners, we can only make best gumses about the utility of various implementations, and so we can only select trajectories through implementation space that approximate the profile of some imaginary user. In real situations, we must allow users to override profiles to determine which aspects of the application semantics are importat to them at run-time. Design Choice: Selecting Fixed Trajectory Behaviors Users of applications don’t care about the subtleties of network and display performance – they just want the software to work. Thus whether the behaviors are fied at design-time or run-time, the user should be able to use the application without fiddling with editors to set values up. So the designer must determine and install a set of trajectories through the implementation space that correspond to some expected path through resource availability. We thus provide some fied trajectories which give the application a behaviour when degrading in the face of resource shortage. In doing so, the designer has to determine the likely contexts in which the application is to be used, so that they can guess at the likely semantics, and then experiment and select appropriate behaviors. The choice of fixed trajectories has a large impact on whether users will use an application and so is a ve~ importrmt part of the design process. We believe that the context and associated semantics m only be determined by users, so as designers we must bring users into the design process. If possible, we should study them in situ, but if this is not faible due to project budgetary constraints they should at lmst be used in experimental mock-ups. Design Choice: Designing Appropriate Interfaces for Customisation Traditionally, customisation of networking and distributed applications has been via dialogues which provide direct access to actual variables within the program. However, Ms is is of limited use when the abstractions are themselves complex and of no meaning within the experience of the user. The open implementation of the components provides abstractions within the application domain to allow manipulation of behaviour. However, these abstractions maybe far removed from the experience and understanding of the user. In particular, the various nuances and abstractions of networked and distributed applications are rarely understood by programmers, so it is unlikely the the user will be able to comprehend behaviour specification in terms ofnctworking abslractious. Instmd, the abstractions of the application must be mapped from the system image onto metaphorical controls which build

upon the experiences of the user. By using metaphors from the cxpcricnce of the user, we can educate the user to have an appropriate men(al model ofthc application [1].

3

Technological Choice: The Lowband Multimedia Information Retrieval Architecture

veloped a test bed application that acts as an image proxy for web pages, providing similar functionality to [2]. When a page is rcqucstcd by a browser, the proxy edits the html of returned pages to replace image tags by applet tags for the Iowband applet. Whhin the browser, the applets tdk to the proxy holding the image and remotely invoke the transformations before downloading the image. The decision about what set of transformations to apply to a given piece of multimedia is made within the applet. ~Is decision comes from the interpretation of an instance of what we have termed the Media Policy Language mpl. This script is at the heart of the cotigurability of our system, it weaves the transformations and media together. As the name suggests, mpl defines policies on what to do to media in particular situations of bandwidth availability and display characteristics described in Section 5.

We use active multimedia objects to adapt themselves to the available bandwidth and display characteristics. An active multimedia object is some form of multimedia such as an image or a video with an associated piece of code that can control various aspects of the object either at the point it is served, cached or displayed. These aspects are such things as the level oflossy compression in a JPEG image, the frame rate of video, or the CO1OW depth and size of an image adapted to a specific Plotting the Implementation Space: display device. We use the obvious media hierarchy to repre- 4 sent the media, as in Figure 2, and provide control interfaces Experiments on Images to the media, as in Table 1. These control interfaces can be remotely called across the network, transforming the media For the image proxy, the key points in the implementation before it is downloaded. As long as the overhead of the re- space occur when the user detects a reduction in quality due to mote calls plus transformed media size is less than the original the adaptation ttilng place in the proxy. For the scale method, media size, then the transformations may considerably reduce this is too dependent upon context to determine - the authors the time taken to download the media, yet be more suited to of the HTML may have fied on a particular width of images the display. me control interfaces are the Open Implementa- and any deviation will reduce the quality. However, for the tion of the components in our technology. Full details of the reduce method operating on JPEGs, we can generally reduce network aspects of the architecture can be found in [4]. the quality factor without impact upon the user. We could find no hard numbers within the literature, and from observation, Applicable to most JPEGs were encoded with too high a quality factor, so Description Apply a Iossless compres- uncompressed we designed an experiment to determine the limits of the qualcompress sion to the media ity factor on user perceptions. We could then relate this to the reduce media to display in Video, Images corresponding files sizes. scale a smaller size all Apply Iossy compression reduce 4.1 Perceptual Limits on JPEGs Quali@ Translate media into some all toAscii ASCII equivalent, typiThe primary aim of this experiment was to determine the JPEG cally a description of what quality factor’ at which users can no longer perceive the difthe media represents ference between a high quality image and the same image reConvert an image to an X- image toBmp duced in quality through JPEG compression – this point we Bitmap shall call the just perceivable dlfierence or JPD. When comConvert an image to a Giff image toGiff bined with knowledge about the approximate relationship to Convert a media to some all toHtml file sizes, we can plot the quality threshold for the various imhti representation plementations of JPEG images. It is asserted that: toProgressive Convert a JPEG to JPEG progressive scan a The mean of the JPD from a sample of Web usem representation plus twice the standard deviation of the JPD bedescribe JPEG using Huff- JPEG toBaseline hveen an image of high quality and the same image man coding only compressed, represents the cut-off point to which all JPEG images can be compressed in a default polTable 1: Control intetiaces icy, since given a normal distribution 98% of users To demonstrate the viability of the architecture, we have de-

I\Veare usingthe IndependentJPEG Group’s source code, which scales quantimtiontablesas a percentageoverthe recommendedtables

136

—T

—

T--

-

-- -s..

==

.

-.

.

,.

. ...%:

. ..*..

. ..--.

—

““—

Media Audio

AAA\

PCM

LPCADPCM M-JPEG

Text

Image

Video

MPEG XBM

GIF TIFF

JPEG

ASCII

Figure 2 A partial media hierarchy

offer across section of the kind of images that users would encounter on the Web and also those with differing susceptibility to the JPEG compression algorithm.

would see no perceivable difference in image quality. Those who notice a difference would only notice a slight variation. In this study it was hypothmised that a difference would be found between JPDs for different types of images used in the experiment. 4.1.1

Methodology in determining the Just Perceivable Difference

We used four different images of a face, a tree, an architectural shot of a room and a map graphic. For each of three images, we presented two representations to the subject, one as a full quality JPEG image, the other as a scaled JPEG image. In the first phase, we decreased the quality of the image, till the subject could notice a difference, and in the second phase, we started at the lowest possible quality and increased till the subject could not notice a difference. The high quality image was presented for one second and then removed. After a gap of another second, the variable quality image was presented for one second. We used twen~ subjects, of varying age (14 to 40) and sex, all of whom were regular web users. Statistically conservative cut off point is one where no percentage of the population see a difference beween the image they thought that they were downloading and the image that they receive via the Iowband proxy server. We have defined this as the mean plus mice the standard deviation of a sample of JPDs. However, a less conservative fi~re might be the m- plus one standard deviation; time issu= are discussed below in section 4.2.

Image presentation time is bounded Presentation Criteria: by two factors: the visual persistence effect [11] and subject strategy development. The visual persistence effect is found when wc look at an object for a short period of time; for a short period of time after a visual stimulus has been removed we can recall nmly all of the detail of the stimulus, but after that time recall becomes much worst. That is to say that we have a short term visual buffer. The persistence effect is typically measured at around 250ms. If the presentation time and the period of the delay between presentation was lower than 1000ms then the persistence effect would represent a more significant part of the subjects recall mechanism, producing unwanted and uncontrolled for effects. In pre-experiment trials subjects reported that they developed strategies to determine image quality difference when the image was displayed for more than about a second. In the actual experiment, subjects still probably developed strategies, but the effect is less marked with a shorter presentation time. We therefore decided that a pr=entation time of 1000ms for each image and delay of 1000ms between imag- combined with a random presentation order would be the best presentation criteria for the experiment.

The images: Since it was predicted that there might be a difference in the effects of compression on different kinds of image, and also in the detail users observe when looking at different kinds of image, it was important to show several different types of image to the subjects. Four kinds of image were selected: a face, a tree, a map and an interior scene. The second two pictures included several straight lines, whereas the former two were structurally more natural. Ml of them contained areas of detail and areas of block colour, This combination of image types was deemed to 137

4.1.2

Results

Table 4.1.2 combines both sets of scores for each picture used in the experiment. All figures ~e measure in the units ofJPEG quality factor. Table 4.1.2 shows the improvement in compr=sion ratio when different cut-off points are used; a JPEG quality factor of 75 was used as the baseline. Finally table 4.1.2 shows the probabilities and significance that the scores generated across subjects and bctwccl~pictures are draw from the same population. Clearly, in all cases but one the images can be said to have different JPDs.

4.2

Image Quali@ Choices

We can use the results from this experiment in two ways: in defining the users’ perceptual limits, and their tolerance to visibly degraded images under differing conditions and in differI Ist and 2nd Set Combined (REG index) I ent contexts. Establishing the mean ED over a number of I I wse I wie 1 sd II mimages not only allows the calculation of a cut-off point for face II 27.13 I 8.37 I 8.22 I 13.65 default policies, but also delimits the variation that can meantree I 18.10 I 10.18 t 7.48 I 18.43 ingfully offered to users for customisation of their policy. The hypothesis proposed that the mean of the sample (25.42) plus two times the standard deviation (8.48) giving a further compression of around 57% would be practicable. An ~ wse~ithin subject mean squared err alternative would be to adopt the mean plus one standard deviwie~ithin image mean squared error ation, a value of 25.42. This would also be meaningful – 16% of users would notice only slight degradation in image quality Table 2: Mean Scores Of Both Sets For All Subiects And All and the compression would be higher at 78°/0.However, there Pictures. are highly significant differences between each of the images – except behveen the face and the map. Since there is considerable variation behveen image types, we have chosen the conservative approach of opting for 2 standard deviations. Authors should be made aware of these findings. In particular, their tools should be designed to encourage the use of a default setting that reflects the cut-off point established here. Images intended for use in web pages generated by popular graphic and art packages can be saved at a WEG quality factor of 42. Finally, authors should be encouraged to write policies in which the quality of the image reflects its importance.

5

Run-time Policies: The Media Policy Description Language

Table 3: Mean Compression Ratios For Meaningful cut-off Points

We have chosen to implement the policy of degradation at runtime for the following reasonx

sets face-tree

tree-map tree-room ma~om

●

The underlying technology has four main methods through which the implementation of the transferred media can be manipulated. The application thus has at least four degrees of freedom in which we can change the implementation, and so providing a wide choice of trajectories of degradation.

●

The application using the media is very generic - the Web model of documents is being pushed by Microsoft as others as the basic metaphor for the next generation of machine interfaces. Since the uses of the application are legion, we as designers cannot constrain the choice of policy degradation.

●

The ability to transfom the media entirely across ~es implies an even wider implementation space. Since transformations may be chosen as more convenient by the user, eg. a partially sighted user may transform text to audio, we believe that allowing users to determine policies of adaptation will provide more exciting uses of the technology.

I P of Similarity I Significance I I 3.26e-09

I High

2.41e-05 5.( __ _. 1.21e-06

, --.o.I High

I

Table 4 Results of pair wise t-test for each picture pair

138

_—

—.

----

m

---

--—

.. .

.

●

In downloading a particular media instance, there are two associated actors who are interested in how the media is downloaded and displayed – the person initiating the download and the author of the media. fich of these actors may have preferences about how media should arrive – users may want download as fast as possible, yet authors may require a colour display. To allow each of these actors to defie requirements, it is easier to resolve run-time rmolution of policies.

or ifa user has a monochrome browser media.

image.

* : true

: toMono;

or for a more complicated expression when the browsers is always used over low bandwidth links.

media. image. jpeg: meta (progressive) == false: toprogressive, stimit default; media. * : true: scale 75, stimit author; media. * : true: reduce 50, stimit author; Policy degradation objects are written in the Media Policy media. * : true: compress 10; Language mpl and are interpreted within the application and the server. They can be combined with each other to form Policies w be combined with information to the environment to sel&t paths of degradation depending upon the available new policy objects representing the combined needs and expeconstraints, so if the user wishes to ensure that all download rience of user, author and system designer. The policy objects times are less than five seconds, they first attempt to reduce have been used to control the preferences users have in how the quality, then if this fails, they sale the object, using the imag= are viewed across the Web. However, the principle resolution procas in Section 5.1, applies equally to other multimdia application (networked or otherwise) and indeed to any application that wants to scale it’s media. *: SIZE/B~DWIDTH>5: reduce 50; interface. User, authorial and default policis could be devised media. *: SIZE/B~DWIDTH>5: scale 75; true: compress 10; to allow an application’s widgets to prtient themselves in a media.*: sensible manner on differing platforms and visual displays. An author’s policy to make sure that a JPEG fits in the availThe mpl is a rule based language that allow the mapping of able space certain actions or mutations to specfic groups or subgroups of media according to the various current networking and display media .image. jpeg.’’wwsi tetorgrmypicij peg”g” : conditions. The geneti format is: MEDIA-WIDTH > DISPLAY-WIDTH : path

: condition

: action

scale (~DIA-WIDTH

[, stimission]

/ DIsPLAY-wIDTH);

Path m-s apply action to the spectied media type if the 5.1 Poficy Resolution condition is true and some rule belonging to a different rule group doesn’t override the path. Conditions are legal boolean me policy scripts are compiled together to create a single exexpressions. ecutablepoli~. The policy can act on the multimedia obWe use enviwnment vatiablm to hold values of networking ject with which it is associated through the various interfaces and display conditions and attributes of the medi% as deter- discussed earlie~ transformation, reduction, scaling and commined by the run-time environment [20]. Each environment pression. When the policy is activated (asked to transform its variable has a unique name and has a ~pe. Currently variables media), it goes through a number of parsing and resolution are phases. Th=e phases determine which rulw are relevant to the associated multimedia object, which rules can be removal - int FILESIZE or overridden by others and they establish a definitive prece- int MEDIA-HEIGHT dence order between rules from different policy sources. Ini- int MEDIA-WIDTH tially any mle which applies to media other than that of the - int MEDIA-DEPTH attached multimedia object are removed. Then rules which - int DISPLAY-WIDTH clash are resolved according to the following criteria DISPLAY-HEIGHT - int DISPLAY-DEPTH B~WIDTH ~TE

- int - int - int

MIME META

- string - string

1. user rules have precedence over authorial and default rules, except when 3. or 4. is in place,

(in Bytes/second) (stream flow rate) (mime type of media) (anY other info)

2. authorial rule have precedence over default rula, except when 3. is in place.

Actions are compress (lossless), reduce (quality lossy compression), scrde (Dimension), and transform (from one media to another) etc, as in Table 1. For ins(ancc, if a user wishes to conlprcss all objcc[s over 10k media.

*

: SIZE >= 10*1024 : compress;

3. user and authorial rules can speci~ that they submit to default rules. 4. user rules can specify that they submit to authorial rules. Four phases of multi-pass rule activation then take placa

139

m.

-.,-

.

.

.

,.

___

. .

.

.

.—

Transformation involves firing roles that transform the mul- quality of a WEG or the fme rate of 30 @s video (see sectimedia object from one type in the media hierarchy (fig- tion 4.1). Finally, the default policy is the basis for the user to ure 2). Rules that utilise the transformation interface arc incrementally develop their own policies. key to the writing of policies that cope with difficult disIn an ideal world in which we had usable technology, we play attributes and location dependant data. would have studied users in context, using browsers connected to the lowband proxy. Unfortunately, the performance of the Reduction involves fing rules that utilise the reduce inter- proxy was stymied by the performance problems of W and face, so reducing the quality of the attached multimedia in particular the Object Serialisation. Wllst we wait for Sunobject and thus improving down load time. The reduc- sofi to fix thae in a new release of the ~~ we have conducted tion phase is a multi-pass operation. Each pass of the exploratory interviews to try to capture user opinions about acrules ~er reduces the multimedia object. Passes are ceptable trajectories of degradation, and interface behaviour. repeated until no rule is fired, that is, all of the media’s size and quality requirements are met. Of course this may never happen! So, the multi-pass mechanism is con- 6.1 Image QuaIi@ Tolerance in Context strained by a set of heuristics which can identifi looping, It is relatively straightforward to determine the point at which the exhaustive limits of compression and rules which re- images of reduced quality become perceptibly different from duce the multimedia object to the edge of our perception. the fill-quality counterpart. The point varies between users, Scaling is the process of altering the dimension of the Media. Images are scaled in the X,Y dimension as are Video (although they also have the frames per second dimension). For some media types scaling is not meaningful or takes on a different meaning. For example it is not meaning to scale ASCH and it only meaning to scale audio in terms of amplitude or tone – qualities that m be adjusted to suit the user, but do not effect the download time, compression is a simple one-pass treatment of using nonIossy compression where possible. Media that benefit from this are those which, unlike ~EG and MPEG, do not support their own compression technique. Anything under the mime type mime: text /* can benefit from the use of non-destructive compression. This multi-phase multi-pass process of resolution is at the heart of what makes the policy work. It combines the disparate needs of user, author and the system designers.

6

Setting the Fixed Trajectories: Determining a Default User Policy

There are three good reasons for the existence of a default policy. First, it is necessary to provide a reasonable behavior for an object if no policy is provided by either user or author. Second, it is sensible to insist on certain behaviors unless the user or author overrides those behaviors. It is sensible to use lossless compression on objects that implement no comprwsion of their own, such as text, html, postscript, etc. It is also usefil to be able to provide default policy decisions that only effect the quality of the media slightly but massively improve down load time or display usability, such as a small reduction in the z~e Gzlp compressionutility is used as a publically amilable nondestmcti},ealgorithm

The image

The chosen picture was presented to participants, without context, in four states of degradation the fill Web

image of WEG quality factor 75 and reduced images of quality factors 18, 12 and 63. Afier looking at the full quality image (see below), users were given the three degraded images and asked whether they would accept “all or any of these” in the context of a web-page they had requested. The picture presented to them showed a room with ornamentaldetail in old fimiture and a series of antique portraits. 3eorrespondingto approximately73,22, 15and 9 Kb

140

———

affected by physical factors, but the range is measurable. However, the same standards cannot be applied to the issue of users’ tolerance. Tolerance is a notional concept of image acceptability and, clearly, a number of variables complicate any measurement of this. It can be predicted that there would be difference in how individuals regard the issue based on personal taste and dso what their relationship with the image might beat the time: do they need information from it? Do they like it? Do they even notice its presence? k order to understand a little more of the interaction between these factors, we undertook an exploratory study. The study was not be expected to generate any defiitive design criteria, rather to explore whether such criteria were appropriate or not. Obviously, a thorough test of quality tolerance would have involved working with several different pictures drawn from different genres, presented in different contexts and evaluated in the pursuit of different tasks. Since at this point of development, the purpose in involving users was to gather indicative data, a much simpler method was sufficient. It involved presenting users with one picture in seve~ conditions. Semistructured questioning optimised the amount of information that could be gleaned. There were four groups of questions user-related, image-related, time-related and product-related.

These details were fairly clear in the original image, but by the third degraded image (of quality factor 6), nothing except shape and disintegrated colour were evident. The image was picked as holding enough detail -in its full state - to arouse detached interest or curiosity, but as being emotionally neutral. This was confirmed by the results of the fist question, which asked for users’ r~ponse to the full qurdity picture’s content. From the user’s point of view, when using the Web there are two categori= of image the on= that the user actively swchti to see and those that just arrive as part of pages (which can be firther broken down into those which happen to arouse interest and those which do not). A “neutra~’ image modelled the haph-d condition and allowed for a similar response between users in terms of engagement. Other conditions were discussed. The Users 10 people participated in this study. At the time of the interviews they were all using the Web at l=t 3hrs a week with some up to 10hrs. Ody one browsed regularly with images disabled, but another occasionally used a textonly browser. One had been using a text-only browser extensively to get “a gist” of pages but abandoned this tactic in the last year as images had become more integral to pages. Othemise users viewed the Web through a NetsCape browser of 2.2 or above; one using a black and white ~U and another a smrdl screen. Most had access to fast network connections for which they did not pay. Users’ response to the three degraded images in the second study varied widely. Most users qualified their answers even before committing themselves to considering the images in front of them. As expected, context and purpose were determining factors: “it depends on what the image is fofl, “surely depends on the relevance it has to the page being caller, while one person identified the size at which the picture appeared as the major issue. A majority agreed the best quality image of the three (22Kb) was “acceptable”. However, everyone also agreed that there were times when tils was not good enough. If the purpose of tiding the picture was to study the detail or to print it off, then higher quality was needed. Users also identified other kinds of images that would require better rendering images of fac~ or fie art in particular. Opinion split about the cut-off of this notional acceptability, witil an incrcasiug uumbcr rcjcctiug the image as qualily wcut down. Some people felt that no picture was preferable to a degraded one, while others felt that general information could still be gleaned from it. Most users mentioned either context or purpose again at this point. One user accepted any quality of image. “I just woul~t look at it as long”, she said of the poorest. Another user rejected all the images: “I’d find any quite fmstrating if I was trying to look at the picture for some reason”. So a difference in general tolerance became apparent, even in a small sample.

Waiting for images Users were vague as to how long they would wait for images to appear on a page. Seved said it was relative to what they were used to. T~icdly, it was “not very long”. One user had measured his tolerance and identified it as 15 sees. Other figures, where h-ded, stretched from a few seconds to a couple of minutes. Answering the qumtion appeared difficult for most users: it was accompanied by long pauses, claims that it was a difficult question and one person watched a second go by on his watch before answering. There was also an implicit acknowledgement that patience varies from occasion to occasion. Of course, context was a determining factor too if users knew that they wanted to see the picture, then they would wait considerable longer than if they had no interest. And one pointed out that if she knew how long the wait was likely to be, she could plan round it. Several other interesting behaviors were referred to: browsing as a background activity while doing something else browsing with more than one browsing window open so that multiple web activities could be conducted reading text as the rest of the page (ie images) downloads clicking on links in the text in the appearing page without wa~lng for the image to download-~’clicking through pagw”) D~pite these mechanisms to avoid waiting, fmstration with slow dotioadlng images was apparent. The reasons for this varied: for instance, there might be no interest in the particular image or there might be no interest in the page - given the exploratory nature of much web activity and the poor description of links, it was not surprising that several people said they wanted to see quickly if they had found something interesting or not. Reference was made to the cost of waiting for images, cspecidly across a slow modem link. There was consensus that, dl other things being equal, they would prefer images that arrived in 2 seconds rather than half a minute. Preview mechanism Users were then asked whether they would appreciate the option of previewing images fast at low quality. They were told that: the quality would be around that of the most degraded image they had just been shown it would take a few seconds or lms to arrive under usual circumstances the full image would be easily summoned, if they wanted to see (and wait for) this.

141

----

z#. --.42..%-

*

c

. >--a, -.

Wch user of the system begins with a default mpl file, which All of the sample welcomed the idea enthusiastically. They were then asked whether a “point and click’ mech- provides basic facilities to improve the web experience over anism would be appropriate for summoning the fill image. low bandwidth links, or rwourcc poor displays, ~ey are then Most people felt that this would be intuitive, though several allowed access to concentric shells of editors which adjust the parameters according to their expertise. However, since little points were raised: is known about the appropriate models upon which to build ● This could be confised with hitting a liti especially as interfaces for distributed applications, the designs of the intersome pictures were links. faces allow for the collection of information about the minimum levels of quality users wish to accept. ● Newcomers might not know that clicking for a fill quality The prototype offers two interfaces to the usen simple and image was an option. advanced. A first dialog box, common to both, allows the user ● An icon should accompany degraded images to indicate to select details of constraints and persistent preferences in their status. viewing from a seri= of choices. This creates a policy which never overestimates the capacity of the display screen and can ● A slider mechanism would allow a choice of how much make maximal use of other resources, such as bandwidth. The quality was returned. advanced screen offers a mechanism for further customizing ● Dragging an icon over the picture to be upgraded would the display We anticipate that most people will opt to use the allow for several to be chosen and reloaded at the same simple controls @igure 3 presented here), a series of buttons time. and a slider which allow the user to determine change in terms of outcome (faster or smaller imaga) rather than method (e.g. ● A menu choice could allow all the picturw on a particular compression and transformation). The system gives immedipage to be upgraded simultaneously. ate feedback by representing the prospective chang~ on an HowIn summary, users’ tolerance of visibly degraded images is image in the dialog box before a decision is cotied. dependent on their purposes in viewing the image, the kmd of ever, another advanced button gives access to the code which image viewed and the context in which the image is placed, describes the policies (see below) and cofident users can adalthough there are also observable differences in people’s un- just settings here directly. The interfaces were developed directly with users, who derlying tolerance levels. Despite this, users would seem to appreciate a f~t and degraded “previeti’ image in place of a chose the terminology in which the outcomes were described. fill quality picture on a web-page as long as they knew that We gave this aspect of the work particular attention as concepts of bandwidth image file size and compression are not they could summon the full picture easily. These findings suggest that pushing compression to the edge familiar to some of the users who would benefit from employof perceptibility and beyond would be acceptable in writing ing the architecture, while the titrations of delay and display default policies as long as users could easily amend them. But are common to all. would they? There is no convenient correlation between tolIt is easy to determine if the user is viewing a web page from erance and readiness to customise. Some of the most critical a mobile device such as a laptop or palmtop – the user can users are impatient with software controls, while others would speci~ a policy that accounts for the display and bandwidth attributes of their devicq indeed they could select a predefine be happy to adjust them at length. The findings also suggest that units of time are not an intu- policy in the same way device drivers are selected by user for itive measure for users. Relative terms such as “fastefl might new hardware. be more appropriate to describe speed of delive~ than actual timings. Since bandwidth varies during downloading and timings cannot be guaranteed, this strategy is supported by local 8 Policy politics: Use and Usability circumstances. As system designers, we expected that developing an application that attempted to maximise utility under resource con7 User Customisation: The Policy Edi- straint would unconditionally be “a good thing”. However, for shared applications in which the parties do not necessarily tors share the same goals, issues arise about which of the parties The Media Policy Language would be sufficient for an expert should retain power. programmer to configure their browser using a raw text editor. The degree of control that authors of Web pages have over However, users cannot be expected to write mpl script. Insttid presentation compares unfavorably with that in most other ‘we have constructed a series of layers around the policy de- medi% as the standard layout tool, HTML, is a logical markscriptions, transmogri&ing the concepts of mpl to abstractions up language. H~L is changing slowly to take in more optical and language that are more meaningful to the user. factors, such as specified fonts and styles, because of produc142

,-

.-

,- —

.-.

.

Image Appeannce nese controlsdetermineftredelay beforeimagesappearin yourbrowser ‘ By reducingimagequalityor size,ate Imagetill downloadfaster ‘

Figure 3: The simple editor

crs’ dissatisfaction with this state of at~airs. I+owcvcr, these ncw lags do not guarantee that what is produced is what the user SCCS.because of the variation in browser’s interpretations. And users continue to have the opporttrnityto override some of tbc layout tags provided. to switch off images and Java applcts if they so choose. The balance of power bctwccn user. technology and author is in continual flux. remaining a negotiable feature of the Wcb as originally intended. On the face of it. policies strengthen the user’s hand and contribute to this uncertain environment for the author. If some authors continue to show the same cavalier attitude to image file size and the provision of “Alt” [ags as a text altcmativc. then at least users have redress now. But policies have been designed to allow either side to claim or relinquish control. An author must defer to user preferences where they exist and cannot insist on certain standards ofprcscntation, but only request thcm. Conversely. a user may define a policy that is in part overridden ifan author has produced a policy and in part defines their definite display and download needs. Thus, they can receive something close to the intentions of the author (if they so wish) or something that is “now readable” on their palm top. In the final analysis, if both parties have conflicting policies, then the user’s will triumph. After all, it is the user that must wait while images download and the user who has such special needs as small displays or disabilities that ncccssirdtc diffcriug policies. .~bcproto[ypc software offers a workable alternative to browsing with the images disabled and therefore gom a long way to ensuring that the author’s vision is delivered, if in slightly modified form. A philosophical view upon the conflict bctwccn the author and user policies is that we are seeing a concrctc representation of the post-tnodcm clash about control over the text. Dis[ribu[ion of media over ]lctworks hns provided allolhcr form of distance between the author and the reader, where the nct~vork and display may change the cxpcricncc wished for by

the author into something completely different. even before the reader brings thcmsclvcs to bear. By making the cbangcs forced by networked delivery explicit and configurable, wc can enable the author and the user to enter into a dialogue about what the ntcdia is intended for. and usc the policy prcccdcncc to resolve the asynchronous dispute harmoniously. But this (iialoguc can only occur if authors rccognisc they must take account of the various alternate rcprcscntation of their creations.

9

Conclusion and Future Work

Wc have presented a series of design points in producing an adaptive application. using Ihc design and implementation of our image proxy as an example. The itnportancc of tbc design guidelines is in cmphasising the need to involve the user in the design of adaptive applications. Too many adaptive applicat ions have been designed by systems crtginccrs without thought about the needs of the user. Our other contributions are in re-cvaluatittg the necessary quality factors for JPEG cornpr~ssion, and in designing a II~edia poliq Iangl[agc which dcscribcs user prefcrcnccs in how media should be adapted to restricted rcsourccs. We arc currently extending the Image proxy to a general proxy architecture in which any sncdia can become active. WC arc also invcstiga(ittg aulhoring tools lbal cncouragc the generation of altcmate representations of multimedia pr~sentations, and that allow authors to express their policies. The proxy and software arc available from .

References

143

[1] 1..Clark and M. A. Sossc. Conceptualdesign rcconsidcrcdthe case of the in(crnctsessiondirecto~ tool. In Pee/)/e aIfd

, I

I I I

[17] B. Noble, M. Satyanarayanan, D. Narayanan, J. Tilton, J. Flinn, and K. Walker. Agile application-aware adaptation for mobilAugust1997. ity. In Proceedings of the 16d1ACM Symposium on Operating [2] ArrnandoFoxand EricA. Brewer.Reducingwwwlatencyand SysremPrinciples, St Male, France, October 1997. bandwidthrequirementsvia real-timedistillation.In Pmt. Ffih [18] Colin Perkins, Vicky Hardman, Isidor Kouvelas, and Angela InIl. WWWConference, Paris, May 1996. Sasse. Multicast audio The next generation. In Pmceedingsof [3] Amando Fox, Steven D. Gribble, Eric A. Brewer, and Elan INET97,, Kuala Lumpur, Malaysia, June 1997. Amir. Adapting to nework and client variability via ondemand dynamic distillation. In Pmt. Seven(hlntl. Conr on Arch. Stip- [19] Edward J. Posnak. R. Greg Lavender, and Harrick M. Vin. An adaptive framework for developing multimedia sofiare comport for Ping. Lang. and Ope~ Sys. (ASPLOS-V14, Cambridge ponents. CACM, 40(10)4=7, October 1997. Ma,October1996. Computers XII: Proceedings of HCI’97, pages 67–85, Bristol,

[4] MalcolmMcI1haggaIan Wakemanand Andy Orrnsby. Signaling in a component based world. In Proceedings of the First IEEE Open Architecturesfor Signaling, San Francisco, Ca., April 1998. [5] VanJacobson and Steve McCanne. Visual audio tool-vat Manual Pages, 1992.

[20] Nick Sharples and Ian Wakeman. Netbase: Gaining access to

intemet quality of service from an application. Technical Report CSRP 476, School of Cognitive and Computing Science, University of Sussex, 199S. [21] M. Weiser. Some computer science issues in ubiquitous com-

puting. Communication of the ACM, 36(7)7$85,

July 1993.

[6] Ian Wakeman Jean Bolot and Thierry Turletti. Multicast congestion control in the distribution of variable bit rate video in the inteme~ In PmceedingsACMSIGCOMM94. ACM, August 1994. [7] G. Kiczales. Beyond the black box IEEESo@are, January 1996.

Open imPlemenmtion.

[8] Gregor Kiczales, John Lamping, Cristina Mdeira Lopes, Anurag Mendhekar, and Gail Murphy. Open implementation design guidelines. In Proceedings of International Conference on Sofmare Engineering, Boston Ma, May 1997. [9] [sidor Kouvelas and Vicky Hardman. Overcoming workstation

scheduling problems in a real-time audio tool. In Proceedings of Usenk Annual TechnicalConference, Anaheim Ca, January 1997. Web Document Interactivity on the web. ihttp//www.cogs.susx.ac.uWusedannU~x.h~l~, April 1998.

[10] Ann LighL

[11] Gemld M Long. Iconic memo~

A review and critique of the study of shoti term memory. Psycho/ogicalBuiltin, 88(3}787– 820,1980.

[12] Chris Maeda. A metaobject protocol for controlling file buffer caches. In Proceedings of ISOTAS ’96, 1996. [13] Chris Maeda, Arthur Lee, Gail Murphy, and Gregor Kiczales. Open implementation analysis and design. In Symposium on Sofware Reusability (SSR97), Boston Ma, May 1997. ACM. Browsing Not lazy searching. [14] Gary Marchionini. Slides from talk given at ASIS 96 Panel on Browsing ih@//www.ee.umd.edti march/asis96/teal.htrnl~, 1996. [15] Steven McCanne and Van Jacobson. vie: A flexible timework for packet video. In Proceedings ofACMMuItimedia, San Francisco Ca, November 1995. [16] Brad A. Myers, Richard G. McDaniel, Robert C. MilIer, Alan S. Femency, Andrew Faulring, Bruce D. Kyle, Andrew Mickish, Alex Klimovitski, and Patrick Deane. The amulet environment New models for effective user interface sotiare development IEEE Transactions on Sofm’are Engineering, 23(6)347–365, June 1997.

144

-- ----- —

—.=

—