Disconnected operation in the Coda File System - CiteSeerX

Kistler92 Disconnected System JAMES

J. KISTLER

Carnegie

Mellon

Operation

in the Coda File

and M. SATYANARAYANAN

University

Disconnected

operation

data during

temporary

is a mode of operation that enables a client to continue accessing critical failures of a shared data repository. An important, though not exclusive,

application of disconnected operation is in supporting portable computers. In this paper, we show that disconnected operation is feasible, efficient and usable by describing its design and implementation in the Coda File System. The central idea behind our work is that caching of data, now widely used for performance, can also be exploited to improve availability. Categories and Subject Descriptors: D,4.3 [Operating Systems]: Systems]: Reliability— Management: —distributed file systems; D.4.5 [Operating D.4.8 [Operating Systems]: Performance —nzeasurements General

Terms: Design, Experimentation,

Additional Key Words and Phrases: reintegration, second-class replication,

Measurement,

Performance,

Disconnected operation, server emulation

File fault

Systems tolerance;

Reliability

hoarding,

optimistic

replication,

1. INTRODUCTION Every serious user of a distributed system has faced situations work has been impeded by a remote failure. His frustration acute when his workstation is powerful has been configured to be dependent inst ante

of such dependence

Placing users, growing

data

and

is the use of data

in a distributed

allows

popularity

them

enough to be used on remote resources.

file

to delegate

of distributed

file

from

standalone, but An important

a distributed

system

simplifies

the

administration

systems

where critical is particularly

collaboration

such as NFS

of that

file

syst em. between

data.

[161 and AFS

The [191

This work was supported by the Defense Advanced Research Projects Agency (Avionics Lab, Wright Research and Development Center, Aeronautical Systems Division (AFSC), U.S. Air Force, Wright-Patterson AFB, Ohio, 45433-6543 under Contract F33615-90-C-1465, ARPA Order 7597), National Science Foundation (PYI Award and Grant ECD 8907068), IBM Corporation (Faculty Development Award, Graduate Fellowship, and Research Initiation Grant), Digital Equipment Corporation (External Research Project Grant), and Bellcore (Information Networking Research Grant). Authors’ address: School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. @ 1992 ACM 0734-2071/92/0200-0003 $01.50 ACM Transactions

on Computer Systems, Vol. 10, No. 1, February 1992, Pages 3-25.

J J. Kistler and M, Satyanarayanan

4.

attests to the compelling users of these systems critical

juncture

How

may

nature of these considerations. Unfortunately, have to accept the fact that a remote failure

seriously

can we improve

the benefits when

of a shared

that

repository

inconvenience

this

state

data

repository,

them.

of affairs?

Ideally,

but

is inaccessible.

We

the at a

we would

like

be able to continue call

the

latter

to enjoy

critical

mode

work

of operation

disconnected operation, because it represents a temporary deviation from normal operation as a client of a shared repository. In this paper we show that disconnected operation in a file system is indeed feasible, efficient and usable. The central idea behind our work is that caching

of

exploited

now

data,

to enhance

widely

used

availability.

to

improve

have

We

performance,

implemented

can

tion in the Coda File System at Carnegie Mellon University. Our initial experience with Coda confirms the viability operation.

We have

to two days.

successfully

operated

For a disconnection

disconnected

of this

duration,

and propagating changes typically takes 100MB has been adequate for us during

2. DESIGN Coda

is

servers.

Unix and

for

an

1 clients

The design

academic

half

lasting

one

of reconnecting

A local disk of of disconnection. that

size should

be

OVERVIEW

designed

untrusted

the process

of about workday.

be

opera-

of disconnected

for periods

about a minute. these periods

Trace-driven simulations indicate that a disk adequate for disconnections lasting a typical

also

disconnected

environment and

a much

is optimized

research

consisting smaller

of a large

number

for the access and sharing

environments.

It

is

collection

of trusted patterns

specifically

not

of

Unix

file

typical

of

intended

for

applications that exhibit highly Each Coda client has a local

concurrent, fine granularity disk and can communicate

data access. with the servers

over a high bandwidth network. ily unable to communicate with

At certain times, a client may be temporarsome or all of the servers. This may be due to

a server or network failure, or due to the detachment from the network. Clients view Coda as a single, location-transparent

shared

tem.

servers

The Coda namespace

larity

of subtrees

called

is mapped volumes.

At

to individual each

of a portable

file

client,

a cache

The first mechanism, replicas at more than

server one

allows

replication,

server.

The

set

sys-

( Venus)

to achieve volumes

of replication

high

to have sites

for

is

protocol

based

1Unix

its

volume

on callbacks

is a trademark

ACM Transactions

storage

of AT&T

group

( VSG).

[9] to guarantee

Bell Telephone

that

subset

a

of a VSG that is accessible VSG ( A VSG). The performance currently accessible is a client’s cost of server replication is kept low by caching on disks at clients and through the use of parallel access protocols. Venus uses a cache coherence volume

The

file

at the granu-

manager

dynamically obtains and caches volume mappings. Coda uses two distinct, but complementary, mechanisms availability. read-write

Unix

client

an open file

Labs

on Computer Systems, Vol. 10, No 1, February

1992,

yields

its latest

Disconnected

Operation m the Coda File System

.

Icopy in the AVSG. This guarantee is provided by servers notifying when their cached copies are no longer valid, each notification being to as a ‘callback

break’.

all AVSG sites, Disconnected

Modifications

and eventually operation, the

mechanism While

Venus cache.

services file system requests Since cache misses cannot application

Venus

propagates

depicts

a typical

and disconnected Earlier discuss cantly

this

3. DESIGN At

a high

used

by

disconnected,

by relying solely on the contents of its be serviced or masked, they appear and

and

users.

reverts

involving

When

to server

transitions

[18, 19] have restricts

replication

disconnection replication.

between

server

ends, Figure

1

replication

its

only

our design

described

server

attention

in those

to

areas

for disconnected

replication

disconnected

where

in depth.

In

operation.

its presence

We

has signifi-

operation.

RATIONALE level,

we

empty.

to

operation.

influenced

First,

scenario

paper

server

becomes

programs

modifications

Coda papers

contrast,

the

clients referred

in parallel

AVSG

takes

to

when

are propagated

to missing VSG sites. second high availability

Coda,

as failures

effect

in Coda

5

two

wanted

our system.

factors

to

Second,

use

influenced

our

conventional,

we wished

strategy

off-the-shelf

to preserve

for

high

availability.

hardware

throughout

by seamlessly

transparency

inte-

grating the high availability mechanisms of Coda into a normal environment. At a more detailed level, other considerations influenced our design. the advent of portable workstations, include the need to scale gracefully,

These the

resource, integrity, and very different clients and servers, and the need to strike

about and

We examine

consistency.

3.1

each of these

security

a balance

issues

assumptions made availability between

in the following

Unix

sections.

Scalability

Successful Coda’s

distributed

systems

tend

ancestor,

AFS,

had impressed

rather

than

treating

a priori,

it

to

grow

upon

in

size.

Our

experience

us the need to prepare

as an afterthought

[17].

with

for growth

We brought

this

experience to bear upon Coda in two ways. First, we adopted certain mechanisms that enhance scalability. Second, we drew upon a set of general principles to guide our design choices. An example of a mechanism we adopted for scalability is callback-based whole-file caching, offers the cache coherence. Another such mechanism added occur

advantage of a much simpler failure on an open, never on a read, write,

substantially partial-file [1] would

model: a cache seek, or close.

miss This,

can only in turn,

simplifies the implementation of disconnected operation. A caching scheme such as that of AFS-4 [22], Echo [8] or MFS have complicated our implementation and made disconnected

operation less transparent. A scalability principle that has had considerable of functionality on clients rather than the placing

influence on our design servers. Only if integrity

ACM Transactions on Computer Systems, Vol

is or

10, No 1, February 1992

6.

J J. Kistler and M Satyanarayanan

‘m

_.___—l

ACM TransactIons on Computer Systems, Vol. 10, No 1, February

1992

Disconnected security Another rapid

would

been

compromised

scalability principle change. Consequently,

or agreement algorithms consensus

Powerful,

by

large

have

we have adopted we have rejected

numbers

such as that on the current

Portable

3.2

have

Operation in the Coda File System

of nodes.

this

principle.

is the avoidance ofsystem-wide strategies that require election For

example,

used in Locus [23] that depend partition state of the network.

lightweight

and compact

to observe

how

laptop a person

we

have

on nodes

avoided achieving

computers

are commonplace

with

in a shared

uses such a machine. Typically, he identifies them from the shared file system into the When

he returns,

he copies

substantially simplify use a different name propagate

today.

file

system

files

back

manual that

into

the

caching,

shared

with

disconnected

file

write-back

operation

could

the use of portable clients. Users would not have to space while isolated, nor would they have to man-

changes

champion application The use of portable

data

files of interest and downloads local name space for use while

modified

system. Such a user is effectively performing upon reconnection! Early in the design of Coda we realized

ually

violated

7

Workstations

It is instructive

isolated.

we

o

upon

reconnection.

Thus

portable

for disconnected operation. machines also gave us another

machines

insight.

The

are

fact

a

that

people are able to operate for extended periods in isolation indicates that they are quite good at predicting their future file access needs. This, in turn, suggests

that

it

is reasonable

cache management policy involuntary Functionally,

to

seek

user

for disconnected disconnections

uoluntary disconnections from Hence Coda provides a single

assistance

in

augmenting

operation. caused by failures

caused by unplugging mechanism to cope with

are no different

portable computers. all disconnections. Of

course, there may be qualitative differences: user expectations as well extent of user cooperation are likely to be different in the two cases. First- vs. Second-Class

3.3

If

disconnected

Clients unattended

as the

Replication

operation

all? The answer to this assumptions made about

the

why

is server

replication

question depends clients and servers

is feasible,

critically in Coda.

on the

needed very

at

different

are like appliances: they can be turned off at will and may be for long periods of time. They have limited disk storage capacity,

their software and hardware may be tampered with, and their owners may not be diligent about backing up the local disks. Servers are like public utilities: they have much greater disk capacity, they are physically secure, and they It

are carefully

is therefore

servers,

and

monitored

appropriate

second-class

and administered

to distinguish replicas

(i.e.,

cache

by professional

between copies)

first-class

on clients.

staff. replicas

on

First-class

replicas are of higher quality: they are more persistent, widely known, secure, available, complete and accurate. Second-class replicas, in contrast, are inferior along all these dimensions. Only by periodic revalidation with respect to a first-class replica can a second-class replica be useful. ACM Transactions

on Computer Systems, Vol. 10, No. 1, February 1992.

J. J. Kistler and M. Satyanarayanan

8.

The function and

of a cache

scalability

first-class

coherence

advantages

replica.

When

protocol

is to combine

of a second-class disconnected,

replica

the quality

with

in

for degradation. the

face

ability.

Hence

quency

Whereas

of failures,

and

server

duration

server

replication

in

operation,

because

contrast,

quality

because

operation,

which

little.

it

of data for

reduces

fre-

viewed

additional Whether

avail-

the

is properly

it requires

costs

of a replica

the quality

forsakes

is important

quality

which it is contingent is the greater the poten-

preserves

operation

of disconnected

a measure of last resort. Server replication is expensive Disconnected

replication

disconnected

the

of the second-class

may be degraded because the first-class replica upon inaccessible. The longer the duration of disconnection, tial

the performance

hardware.

to

use

server

replication or not is thus a trade-off between quality and cost. Coda permit a volume to have a sole server replica. Therefore, an installation rely exclusively on disconnected operation if it so chooses.

3.4

Optimistic

vs. Pessimistic

By definition,

a network

replica

and

all

replica

control

to the design ing

strategies,

danger

occurrence. A pessimistic

all reads

control

by

towards

a disconnected

choice

strategy and

provides

conflict-

deals

them

operation

would

of a cached reconnection. preclude

reads

much

and

of

central

avoids

resolving

disconnected

would

families

or by restricting

everywhere,

detecting

client

strategy

writes

and writes

second-class

two

[51, is therefore

A pessimistic

partitioned

exclusive control such control until

by a disconnected

between

optimistic

An optimistic

of conflicts

client to acquire shared or disconnection, and to retain exclusive

The

operation.

partition.

approach

between

and

pessimistic

by permitting

attendant

exists

by disallowing

does can

Control

associates.

of disconnected

to a single

availability

partition

its first-class

operations

and writes

Replica

as

higher

with

after

the their

require

a

object prior Possession

to of

reading

or writing

at all other replicas. Possession of shared control would allow reading at other replicas, but writes would still be forbidden everywhere. Acquiring control prior to voluntary disconnection is relatively simple. It is more

difficult

when

have to arbitrate

disconnection

among

needed

to make

system

cannot

a wise predict

is involuntary,

multiple

requesters.

decision

is not

which

requesters

because

Unfortunately,

readily would

available. actually

the

system

may

the information For use the

example, object,

the when

they would release control, or what the relative costs of denying them access would be. Retaining control until reconnection is acceptable in the case of brief disconnections. But it is unacceptable in the case of extended disconnections. A disconnected client with shared control of an object would force the rest of the system to defer all updates until it reconnected. With exclusive control, it would even prevent other users from making a copy of the object. Coercing the client

to reconnect

ACM Transactions

may

not be feasible,

since

its whereabouts

on Computer Systems, Vol. 10, No 1, February

1992

may

not be

Disconnected


.

9

known. Thus, an entire user community could be at the mercy of a single errant client for an unbounded amount of time. Placing a time bound on exclusive or shared control, as done in the case of leases [7], avoids this problem but introduces others. Once a lease expires, a disconnected client loses the ability to access a cached object, even if no one else in the

system

disconnected already

is interested

operation

made

while

An optimistic disconnected

which

in it.

is to provide

disconnected

approach client

This,

have

in turn, high

conflict

the

availability.

purpose

Worse,

of

updates

to be discarded.

has its own disadvantages.

may

defeats

with

An update

an update

at another

made

at one

disconnected

or

connected client. For optimistic replication to be viable, the system has to be more sophisticated. There needs to be machinery in the system for detecting conflicts, age and manually

for automating resolution when possible, and for confining dampreserving evidence for manual repair. Having to repair conflicts violates transparency, is an annoyance to users, and reduces the

usability

of the system.

We chose optimistic replication because we felt that its strengths and weaknesses better matched our design goals. The dominant influence on our choice an

was the low degree

optimistic

strategy

of write-sharing

was

likely

to

typical lead

to

of Unix.

This

relatively

few

implied

that

conflicts.

An

optimistic strategy was also consistent with our overall goal of providing the highest possible availability of data. In principle, we could have chosen a pessimistic strategy for server replication even after choosing an optimistic strategy for disconnected But that would have reduced transparency, because a user would the anomaly of being able to update data when disconnected, unable

to do so when

the previous replication. Using system

connected

arguments

an optimistic from

strategy

the user’s

data in his accessible everyone else in that set of shrinks

updates

universe

universe.

contact. universe

become

4. DETAILED

throughout

strategy

presents

At any time,

Further, also apply

a uniform

visible

model

and his updates are immediately His accessible universe is usually

of

of the

visible to the entire

When failures occur, his accessible universe he can contact, and the set of clients that they, in

throughout

his now-enlarged

disconnected, reconnection,

accessible

his his

universe.

AND IMPLEMENTATION

In describing our implementation client since this is where much the physical structure of Venus, and Sections

many

to server

he is able to read the latest

In the limit, when he is operating consists of just his machine. Upon

DESIGN

tion of the server Section 4.5.

of the servers.

of an optimistic

perspective.

servers and clients. to the set of servers

turn, can accessible

to a subset

in favor

operation. have faced but being

of disconnected of the complexity

of a client, 4.3 to 4.5

support

needed

operation, we focus on the lies. Section 4.1 describes

Section 4.2 introduces the major states discuss these states in detail. A descripfor disconnected

ACM Transactions

operation

is contained

in


10

.

J. J Kistler and M Satyanarayanan

IT

—

Venus

I

1

Fig.2.

4.1

Client

Because than

Structure

ofa Coda client.

Structure of the

part but

debug.

Figure

Venus

complexity

of Venus,

of the kernel.

mance,

would

The latter

have

been

2 illustrates

intercepts

Unix

we made approach

less portable

the high-level file

system

it a user-level

may

have

access,

are handled A system

disconnected

MiniCache. If possible, the returned to the application. service returns

operation

entirely by Venus. call on a Coda object

calls

via

or server

is forwarded

from

our

for good performance

Venus

Logically, reintegration.

Venus always

rather

better

perfor-

more

difficult

to

widely

used

Sun

Vnode

performance

overhead MiniCache to filter contains no support

replication; by the

these

Vnode

on out for

functions

interface

to the

call is serviced by the MiniCache and control Otherwise, the MiniCache contacts Venus

is to

the call. This, in turn, may involve contacting Coda servers. Control from Venus via the MiniCache to the application program, updating

Measurements

4.2

process

of a Coda client.

the

MiniCache state as a side effect. MiniCache initiated by Venus on events such as callback critical

yielded

and considerably structure

interface [10]. Since this interface imposes a heavy user-level cache managers, we use a tiny in-kernel many kernel-Venus interactions. The MiniCache remote

to Coda servers

implementation

state changes breaks from

confirm

that

the

may Coda

also be servers.

MiniCache

is

[211,

States Venus operates in one of three Figure 3 depicts these states

is normally on the alert

emulation,

between

and them.

in the hoarding state, relying on server replication but for possible disconnection. Upon disconnection, it enters

the emulation state and remains Upon reconnection, Venus enters ACM Transactions

states: hoarding, and the transitions

on Computer Systems, Vol

there for the the reintegration 10, No 1, February

duration state, 1992

of disconnection. desynchronizes its

Disconnected


.

11

r)

Hoarding

Fig. 3. Venus states and transitions. When disconnected, Venus is in the emulation state. It transmits to reintegration upon successful reconnection to an AVSG member, and thence to hoarding, where it resumes connected operation.

cache

with

its

AVSG,

and

then

all volumes may not be replicated be in different states with respect conditions in the system.

4.3

reverts

to

across the to different

the

hoarding

state.

same set of servers, volumes, depending

Since

Venus can on failure

Hoarding

The hoarding state

state

is to hoard

not its only

is so named

useful

data

responsibility.

because

Rather,

reference

predicted

with

behavior,

manage

is

its cache in a manner

and disconnected operation. a certain set of files is critical

the implementation especially

and reconnection

–The true cost of a cache hard to quantify. an object

this

must

For inbut may

in

of hoarding: the

distant

future,

cannot

be

certainty.

–Disconnections

—Activity

in this

However,

files. To provide good performance, Venus must to be prepared for disconnection, it must also cache

the former set of files. Many factors complicate —File

of Venus

of disconnection.

Venus

that balances the needs of connected stance, a user may have indicated that currently be using other cache the latter files. But

a key responsibility

in anticipation

at other

clients

miss must

are often while

unpredictable.

disconnected

be accounted

is highly

for, so that

variable

the latest

and

version

of

is in the cache at disconnection.

—Since cache space is finite, the availability of less critical to be sacrificed in favor of more critical objects. ACM Transactions

objects

may

have


12

J J. Kistler and M Satyanarayanan

. # a a a

Personal files /coda /usr/jjk d+ /coda /usr/jjk/papers /coda /usr/jjk/pape

# # a a a a a a a a a a a

100:d+ rs/sosp 1000:d+

# a a a a a

System files /usr/bin 100:d+ /usr/etc 100:d+ /usr/include 100:d+ /usr/lib 100:d+ /usr/local/gnu d+ a /usr/local/rcs d+ a /usr/ucb d+

XII files (from Xll maintainer) /usr/Xll/bin/X /usr/Xll/bin/Xvga /usr/Xll/bin/mwm /usr/Xl l/ bin/startx /usr/Xll /bin/x clock /usr/Xll/bin/xinlt /usr/Xll/bin/xtenn /usr/Xll/i nclude/Xll /bitmaps c+ /usr/Xll/ lib/app-defaults d+ /usr/Xll /lib/font s/mist c+ /usr/Xl l/lib/system .mwmrc

(a) # # a a a

Venus source files (shared among Coda developers) /coda /project /coda /src/venus 100:c+ /coda/project/coda/include 100:c+ /coda/project/coda/lib c+

(c) Fig.

4.

Sample

hoard

profiles.

These are typical

hoard

profiles

provided

hy a Coda user, an

application maintainer, and a group of project developers. Each profile IS interpreted separately by the HDB front-end program The ‘a’ at the beginning of a line indicates an add-entry command. Other commands are delete an entry, clear all entries, and list entries. The modifiers following some pathnames specify nondefault priorities (the default is 10) and/or metaexpansion for the entry. ‘/coda’.

To

Note that

address

the pathnames

these

concerns,

and periodically cache via a process known algorithm,

4.3.1

plicit

Prioritized

sources

Cache

manage

Management.

of information

of interest front-end

we

with

reevaluate which walking. as hoard

rithm. The implicit information traditional caching algorithms. hoard database per-workstation fying objects A simple

beginning

in

its

‘/usr’

the

(HDB),

cache

objects

Venus priority-based

consists Explicit

are actually

symbolic

using

merit

a

links

into

prioritized

retention

in the

combines implicit and excache management algo-

of recent reference history, as in information takes the form of a whose entries are pathnames identi-

to the user at the workstation. program allows a user to update

the

HDB

using

hoard profiles, such as those shown in Figure command scripts called Since hoard profiles are just files, it is simple for an application maintainer

4. to

provide a common profile for his users. or for users collaborating on a project to maintain a common profile. A user can customize his HDB by specifying different combinations of profiles or by executing front-end commands interactively. To facilitate construction of hoard profiles, Venus can record all file references observed between a pair of start and stop events indicated by a user. To reduce the verbosity of hoard profiles and the effort needed to maintain them, Venus if the letter ACM Transactions

meta-expansion of HDB supports ‘c’ (or ‘d’) follows a pathname, on Computer Systems, Vol

entries. As shown in Figure the command also applies

10, No, 1, February

1992

4, to

Disconnected immediate

children

(or all


descendants).

A ‘ +‘

following

the

.

13

‘c’ or ‘d’ indi-

cates that the command applies to all future as well descendants. A hoard entry may optionally indicate

as present children or priority, with a hoard

higher priorities indicating more critical The current priority of a cached object

of its hoard

well

as a metric

representing

ously in response no longer in the victims when To resolve imperative

usage.

The

latter

priority

is updated

all

ensure

the

ancestors

that

a cached

of the

object

directory

while

of objects chosen as

disconnected,

also be cached.

is not

as

continu-

to new references, and serves to age the priority working set. Objects of the lowest priority are

cache space has to be reclaimed. the pathname of a cached object

that

therefore

recent

objects. is a function

purged

it

is

Venus

must

any

of its

before

hierarchical cache management is not needed in tradidescendants. This tional file caching schemes because cache misses during name translation can be serviced, albeit at a performance cost. Venus performs hierarchical

cache management by assigning infinite priority to directories with children. This automatically forces replacement to occur bottom-up. 4.3.2

Hoard

Walking.

say that

We

a cache

is in

cached

signifying

equilibrium,

that it meets user expectations about availability, when no uncached object has a higher priority than a cached object. Equilibrium maybe disturbed as a result the

of normal cache

activity.

mentioned its priority equilibrium,

as a hoard

walk.

implementation, bindings

replacing

an

suppose object,

B.

an object,

A, is brought

Further

suppose

restores but

one

disconnection. of HDB

equilibrium

A hoard

walk may

The

entries

occurs be

by performing every

explicitly

walk

occurs

are reevaluated

Coda clients. For example, new tory whose pathname is specified

required in

an operation

10 minutes by

in

is

First,

to reflect

update

activity

been created in the HDB.

a key characteristic

data is refetched only a strategy may result

current

a user

phases.

children may have with the ‘ +‘ option

callback-based caching, break. But in Coda, such

being unavailable should it. Refetching immediately ignores

B

known

our

two

priorities of all entries in the cache and HDB are reevaluated, fetched or evicted as needed to restore equilibrium. Hoard walks also address a problem arising from callback traditional callback

into

that

in the HDB, but A is not. Some time after activity on A ceases, will decay below the hoard priority of B. The cache is no longer in than the uncached since the cached object A has lower priority

object B. Venus periodically

voluntary

For example,

on demand,

prior the

to

name

by other in a direcSecond, the and

objects

breaks.

on demand in a critical

In

after a object

a disconnection occur before the next reference to upon callback break avoids this problem, but of Unix

it is likely to be modified many interval [15, 6]. An immediate

environments:

once an object

is modified,

more times by the same user within a short refetch policy would increase client-server

traffic considerably, thereby reducing scalability. Our strategy is a compromise that balances scalability. For files and symbolic links, Venus ACM Transactions

availability, consistency and purges the object on callback

on Computer Systems, Vol. 10, No. 1, February 1992

14

.

break,

and refetches

occurs

earlier.

J. J. Klstler and M. Satyanarayanan it on demand

If a disconnection

or during were

the next

to occur

hoard

before

walk,

whichever

refetching,

the

object

would be unavailable. For directories, Venus does not purge on callback break, but marks the cache entry suspicious. A stale cache entry is thus available should a disconnection occur before the next hoard walk or reference.

The

callback entry

acceptability

semantics.

y of stale

A callback

has been added

to or deleted

other directory entries fore, saving the stale nection

causes

directory break

and copy

consistency

the and

data

follows

on a directory

from

from

typically

the directory.

its

particular

means

It is often

that

objects they name are unchanged. using it in the event of untimely

to suffer

only

a little,

but

increases

actions

normally

an

the case that Therediscon-

availability

considerably. 4.4

Emulation

In the

emulation

servers.

For

semantic

state,

example,

checks.

Venus

performs

Venus

now

many

assumes

It is also responsible

full

responsibility

for generating

handled

by

for access and

temporary

file

identi-

(fids) for new objects, pending the assignment of permanent fids at updates reintegration. But although Venus is functioning as a pseudo-server, accepted by it have to be revalidated with respect to integrity and protection by real servers. This follows from the Coda policy of trusting only servers, not clients. To minimize unpleasant delayed surprises for a disconnected user, it

fiers

behooves

Venus

Cache algorithm

management used during

cache freed ity

entries

to be as faithful

of the

immediately,

so that

default request 4.4.1

they

objects

but are

not

version

in its emulation. is

involved.

Cache

of other

modified

those

purged

before

done with operations entries

the same priority directly update the

of deleted

objects

assume

reintegration.

objects

infinite

On a cache

are

prior-

miss,

the

behavior of Venus is to return an error code. A user may optionally Venus to block his processes until cache misses can be serviced. Logging.

During

emulation,

to replay update activity when in a per-volume log of mutating contains

as possible

during emulation hoarding. Mutating

a copy state

of the

corresponding

of all objects

Venus

records

sufficient

information

it reinteWates. It maintains this information log. Each log entry operations called a replay

referenced

system

call

arguments

as well

as the

by the call.

Venus uses a number of optimizations to reduce log, resulting in a log size that is typically a few

the length of the replay percent of cache size. A

small log conserves disk space, a critical resource during periods of disconnection. It also improves reintegration performance by reducing latency and server load. One important optimization to reduce log length pertains to write operations on files. Since Coda uses whole-file caching, the close after an open of a file for modification installs a completely new copy of the file. Rather than logging the open, close, and intervening write operations individually, Venus logs a single store record during the handling of a close, Another optimization for a file when a new

consists of Venus discarding a previous store record one is appended to the log. This follows from the fact

ACM TransactIons on Computer Systems, Vol. 10, No. 1, February

1992,

Disconnected that

a store

record

renders

all


previous

does not contain

versions

of a file

a copy of the file’s

.

superfluous.

contents,

but

15

The

merely

store

points

to the

copy in the cache. We

are

length

currently

of the replay

the previous earlier would

implementing

two

log.

generalizes

paragraph

The

such that

operations may be the canceling

second optimization the inverting and own log record 4.4.2

chine

further

as well

cancel the of a store

as that

to reduce

optimization

which

the

described

in

the effect

overwrites

of

corresponding log records. An example by a subsequent unlink or truncate. The operations to cancel both of inverse For example, a rmdir may cancel its

of the corresponding

A disconnected

user

and continue

where

a shutdown

optimizations the

any operation

exploits knowledge inverted log records.

Persistence.

after

first

mkdir.

must

be able

he left

to restart

his

ma-

off. In case of a crash,

the

amount of data lost should be no greater than if the same failure occurred during connected operation. To provide these guarantees, Venus must keep its cache and related data structures in nonvolatile storage. Mets-data,

consisting

of cached

directory

and symbolic

link

contents,

status

blocks for cached objects of all types, replay logs, and the HDB, is mapped virtual memory (RVM). Transacinto Venus’ address space as recoverable tional

access to this

Venus.

memory

The actual

contents

local Unix files. The use of transactions

is supported of cached to

by the RVM

files

library

are not in RVM,

manipulate

meta-data

[13] linked but

into

are stored

simplifies

Venus’

as job

enormously. To maintain its invariants Venus need only ensure that each transaction takes meta-data from one consistent state to another. It need not be concerned with crash recovery, since RVM handles this transparently. If we had files,

chosen

we

synchronous RVM trol

over

the

would

writes

supports the

obvious

have

An

alternative to

follow

and an ad-hoc

local,

basic

serializability.

had

non-nested

recovery

RVM’S write-ahead bounded provides flushes. Venus

exploits

log from persistence,

the

allows

commit

latency

a synchronous the application

time to time. where the

capabilities

and

of atomicity,

reduce

thereby avoiding commit as no-flush, persistence of no-flush transactions,

of RVM

in

local

accessible

when

emulating,

When bound

con-

permanence

and

by

the

labelling

write to disk. To ensure must explicitly flush

used in this manner, RVM is the period between log

to provide

Venus

is more

the log more frequently. This lowers performance, data lost by a client crash within acceptable limits. 4.4.3 volatile

Resource

storage

Exhaustion.

during

emulation.

It

is possible The

timed

independent

good performance

constant level of persistence. When hoarding, Venus initiates log infrequently, since a copy of the data is available on servers. Since are not

Unix

of carefully

algorithm.

properties can

meta-data

discipline

transactions

transactional application

of placing a strict

two

for

conservative but

Venus

significant

keeps

at a flushes servers

and flushes the

to exhaust instances

amount

its of this

of

nonare

ACM ‘l!ransactions on Computer Systems, Vol. 10, No. 1, February 1992

16

.


the file allocated

cache becoming filled to replay logs becoming

with full.

modified

files,

Our current implementation is not very graceful tions. When the file cache is full, space can be freed modified

files.

reintegrate always

When

ion

log space is full,

has been

no further

performed.

and

the

in handling by truncating

mutations

Of course,

RVM

non-rout

these situaor deleting

are allowed

at ing

space

until

operations

are

allowed.

We plan emulating.

to explore at least three alternatives One possibility is to compress file

to free up disk space while cache and RVM contents.

Compression trades off computation time for space, and recent work [21 has shown it to be a promising tool for cache management. A second possibility is to allow users to selectively back out updates made while disconnected. A third

approach

is to allow

out to removable 4.5

media

portions

of the

such as floppy

file

cache

and RVM

to be written

disks.

Reintegration

Reintegration

is a transitory

state

through

which

Venus

passes

in changing

roles from pseudo-server to cache manager. In this state, Venus propagates changes made during emulation, and updates its cache to reflect current server state. Reintegration is performed a volume at a time, with all update activity

in the volume

4.5.1

Replay

suspended

in two

for

and

This

new

objects

step

steps.

in many

independently

executed. protection,

to replace cases,

of changes

step, Venus temporary

since

Venus

from

obtains fids

client

in the

obtains

to AVSG

permanent

a small

fids

replay

log.

supply

of

fids in advance of need, while in the hoarding state. In the second replay log is shipped in parallel to the AVSG, and executed at each

member.

single transaction, which The replay algorithm parsed, locked.

In the first

uses them

is avoided

permanent step, the

completion.

The propagation

Algorithm.

is accomplished

until

Each

server

performs

the

is aborted if any error is detected. consists of four phases. In phase

replay one

within the

log

a is

a transaction is begun, and all objects referenced in the log are In phase two, each operation in the log is validated and then The validation consists of conflict detection as well as integrity, and disk space checks. Except in the case of store operations,

execution during replay is identical to execution in connected mode. For a file is created and meta-data is updated to reference store, an empty shadow it, but the data transfer is deferred. Phase three consists exclusively of performing

these

data

transfers,

a process

known

as bac?z-fetching.

The final

phase commits the transaction and releases all locks. If reinte~ation succeeds, Venus frees the replay log and resets the priority of cached objects referenced by the log. If reintegration fails, Venus writes file in a superset of the Unix tar format. out the replay log to a local replay The log and all corresponding cache entries are then purged, so that subsequent references will cause refetch of the current contents at the AVSG. A tool is provided which allows the user to inspect the contents of a replay file, compare entirety.

it

to the

ACM Transactions

state

at

the

AVSG,

and

replay

on Computer Systems, Vol. 10, No. 1, February

it 1992,

selectively

or in

its

Disconnected


Reintegration at finer granularity than perceived by clients, improve concurrency reduce user effort during manual replay. implementation

to reintegrate

dent

within

operations

using

the

precedence

Venus

will

pass them

to servers

along

The tagged

with

with

operations

clients. conflicts.

for conflicts a storeid

of subsequences

subsequences

of Davidson

precedence the replay

The

relies

that

only

Read/

system model, since it of a single system call.

check

granularity

graphs

of depen-

can be identified

[4]. In the

revised

during

emulation,

imple

-

and

log.

Our use of optimistic replica control means that of one client may conflict with activity at servers

Handling.

or other disconnected with are write / write Unix file boundary

maintain

17

a volume would reduce the latency and load balancing at servers, and To this end, we are revising our

Dependent

approach

graph

mentation

ConfZict 4.5.2 the disconnected

at the

a volume.

.

class of conflicts we are concerned conflicts are not relevant to the

write

has

no

on the

fact

uniquely

notion that

identifies

of atomicity each replica

the

last

beyond

the

of an object

update

to it.

is

During

phase two of replay, a server compares the storeid of every object mentioned in a log entry with the storeid of its own replica of the object. If the comparison the mutated

indicates equality for all objects, the operation objects are tagged with a new storeid specified

is performed and in the log entry.

If a storeid comparison fails, the action taken depends on the operation being validated. In the case of a store of a file, the entire reintegration is aborted. But for directories, a conflict is declared only if a newly created name collides with an existing name, if an object updated at the client or the server has been deleted by the other, or if directory attributes have been modified

at the

directory

updates

server

and the

is consistent

client. with

This our

strategy

strategy

and was originally suggested by Locus [23]. Our original design for disconnected operation replay files at servers rather than clients. This damage forcing

to

be

manual

confined repair,

by

marking

conflicting

as is currently

of resolving in server

Today, laptops

[11],

called for preservation of approach would also allow replicas

inconsistent

done in the case of server

We are awaiting more usage experience to determine the correct approach for disconnected operation. 5. STATUS

partitioned

replication

whether

and

replication. this

is indeed

AND EVALUATION

Coda runs on IBM RTs, Decstation such as the Toshiba 5200. A small

3100s and 5000s, and 386-based user community has been using

Coda on a daily basis as its primary data repository since April 1990. All development work on Coda is done in Coda itself. As of July 1991 there were nearly 350MB of triply replicated data in Coda, with plans to expand to 2GB in the next A version

few months. of disconnected

operation

with

strated in October 1990. A more complete 1991, and is now in regular use. We have for periods

lasting

one to two days. ACM Transactions

Our

minimal

functionality

version was functional successfully operated

experience

with

on Computer Systems, Vol

was demonin January disconnected

the system

has been

10, No 1, February 1992.

18

.

J. J. Kistler and M. Satyanarayanan Table I.

Elapssd

Time

(seconds)

Andrew

Make

Reintegration Totaf

Time

AllocFid

Size of Replay

(xconds)

Replay

con

Records

Log

Data

Back-Fetched (Bytes)

Bytes

288 (3)

43 (2)

4 (2)

29 (1)

10 (1)

223

65,010

1,141,315

3,271 (28)

52 (4)

1 (o)

40 (1)

10 (3)

193

65,919

2,990,120

Benchmark

Venus

Time for Reintegration

Note: This data was obtained with a Toshiba T5200/100 client (12MB memory, 100MB disk) reintegrating over an Ethernet with an IBM RT-APC server (12MB memory, 400MB disk). The values shown above are the means of three trials, Figures m parentheses are standard deviations.

quite

positive,

and we are confident

that

the refinements

under

will result in an even more usable system. In the following sections we provide qualitative and quantitative to three important questions pertaining to disconnected operation. (1) How

long

(2) How

large

(3) How

likely

5.1

Duration

does reintegration a local

disk

development answers These are:

take?

does one need?

are conflicts? of Reintegration

In our experience, development lasting To characterize

typical disconnected a few hours required

reintegration

speed more

sessions of editing and program about a minute for reintegration. precisely,

we measured

the reinte-

gration times after disconnected execution of two well-defined tasks. The first task is the Andrew benchmark [91, now widely used as a basis for comparing file system performance. The second task is the compiling and linking of the current version of Venus. Table I presents the reintegration times for these tasks. The time for reintegration consists of three components: the time to allocate permanent fids, the time for the replay at the servers, and the time for the second phase of the update protocol used for server replication. The first component

will

be zero for many

disconnections,

due to the

preallocation

of

fids during hoarding. We expect the time for the second component to fall, considerably in many cases, as we incorporate the last of the replay log optimizations described in Section 4.4.1. The third component can be avoided only if server replication is not used. One can make some interesting secondary observations from Table 1. First, the total time for reinte~ation is roughly the same for the two tasks even though the Andrew benchmark has a much smaller elapsed time. This is because the Andrew benchmark uses the file system more intensively. Second, reintegration for the Venus make takes longer, even though the number of entries in the replay log is smaller. This is because much more file data is ACM Transactions


1992

Disconnected


o

19

7J50$

— – – -----

540 g -cl ~ m 30 s ~

Max Avg Min

s 20 & .Q) z 10 -

—--———

/“--

.........

. . . . . . . . . . ------0

4

2

‘“

12 10 Time (hours)

8

6

,. . . ..-. -.

Fig. 5. High-water mark of Cache usage. This graph is based on a total of 10 traces from 5 active Coda workstations. The curve labelled “Avg” corresponds to the values obtained by “Max” and “ Min” plot averaging the high-water marks of all workstations. The curves labeled the highest and lowest values of the high-water marks across all workstations. Note that the high-water mark does not include space needed for paging, the HDB or replay logs.

back-fetched in the third phase of the replay. Finally, neither any think time. As a result, their reintegration times are

task involves comparable to

that

session

after

a much

longer,

but

more

typical,

disconnected

in

our

environment.

5.2

Cache

Size

A local disk capacity of 100MB on our clients has proved initial sessions of disconnected operation. To obtain a better the cache size requirements for disconnected operation, reference menting

traces from workstations

adequate for understanding we analyzed

our environment. The traces were obtained to record information on every file system

by instruoperation,

regardless of whether the file was in Coda, AFS, or the local file system. Our analysis is based on simulations driven by these traces. Writing validating Venus Venus driven

a simulator

that

precisely

models

the complex

caching

our of file

and

behavior

of

would be quite difficult. To avoid this difficulty, we have modified to act as its own simulator. When running as a simulator, Venus is by traces rather than requests from the kernel. Code to communicate

with the servers, as well as code to perform system are stubbed out during simulation. Figure

5 shows

The actual

disk

both the explicit From our data operating

the

high-water

size needed

mark

physical

of cache

for disconnected

1/0

usage

operation

on the

as a function

for a typical ACM Transactions

workday.

Of course,

user

file

of time.

has to be larger,

and implicit sources of hoarding information it appears that a disk of 50-60MB should

disconnected

local

since

are imperfect. be adequate for activity

that

is


20

J. J. Kkstler and M. Satyanarayanan

●

drastically significantly We plan First,

different from what different results. to extend our work

we will

investigate

disconnection.

Second,

cache we will

was

recorded

in

on trace-driven

our

could

simulations

size requirements be sampling

traces

in three

for much

a broader

longer

range

traces from many more machines in our environment. the effect of hoarding by simulating traces together

profiles

have

5.3

Likelihood

been specified

virtually

different

no

network

Second,

server

conflicts

replication due to While

partitions.

perhaps grows

will lead to many To obtain data mented 400

the AFS

computer

program cant

conflicts

will

larger.

Third,

in Coda for nearly

multiple gratifying

servers science

amount

faculty,

the same kind

mate

frequent

of usage

conflicts

a user

would

modifies

an

at

assumptions,

file

or

we instru-

for

research,

includes

a signifi-

is descended

were

user

are used by over

we can use this

be if Coda

users are system. Coda

scale,

profile

Coda

in is

disconnections

students

usage

Since

AFS

larger

graduate

Their

as the

servers

we have

an object observation

voluntary

These

and

activity.

only

extended

of conflicts

staff

a year,

updating us, this

a problem

perhaps

and education.

and makes

users to

is possible that our with an experimental

in our environment.

of collaborative

environment. Every time

become

more conflicts. on the likelihood

development,

how

Third, we with hoard

by users.

ex ante

subject to at least three criticisms. First, it being cautious, knowing that they are dealing community

of

activity

of Conflicts

In our use of optimistic seen

ways.

periods

of user

by obtaining will evaluate that

produce

to replace

directory,

we

from

AFS

data

to esti-

AFS

in our

compare

his

identity with that of the user who made the previous mutation. We also note the time interval between mutations. For a file, only the close after an open for update is counted as a mutation; individual write operations are not counted.

For

directories,

as mutations. Table II presents is classified

by

all

operations

our observations

volume

type:

user

that

modify

over a period volumes

a directory

of twelve

containing

are counted

months. private

The data user

data,

volumes used for collaborative work, and system volumes containing program binaries, libraries, header files and other similar data. On average,

project

a project volume has about volume has about 1600 files

2600 files and 280 and 130 directories.

directories, and a system User volumes tend to be

smaller, averaging about 200 files and 18 directories, place much of their data in their project volumes. Table II shows that over 99% of all modifications

because were

users

by the

often

previous

writer, and that the chances of two different users modifying the same object less than a day apart is at most O.75%. We had expected to see the highest degree of write-sharing on project files or directories, and were surprised to see that it actually occurs on system files. We conjecture that a significant fraction of this sharing arises from modifications to system files by operators, who change shift periodically. If system files are excluded, the absence of ACM Transactions


1992

Disconnected


.

2I

m “E! o

v

w

ACM Transactions

on Computer

Systems,

Vol.

10, No.

1, February

1992

22


.

write-sharing the previous

is even more striking: more than 99.570 of all mutations are by writer, and the chances of two different users modifying the

same object within ing from the point would not environment.

be

6. RELATED

a week are less than 0.4~o! This of view of optimistic replication.

a serious

problem

if

were

replaced

by

Coda

in

our

WORK

Coda is unique in that it exploits availability while preserving a high no other

AFS

data is highly encouragIt suggests that conflicts

system,

published

caching for both performance and high degree of transparency. We are aware of

or unpublished,

that

duplicates

this

key aspect

of

Coda. By providing system

[20]

since this hoarding,

tools

to link

provided

local

and

rudimentary

was not transparent

its

remote

support

name

for

spaces,

disconnected

the

Cedar

operation.

file But

primary goal, Cedar did not provide support reintegz-ation or conflict detection. Files were

for ver-

sioned and immutable, and a Cedar cache manager could substitute a cached version of a file on reference to an unqualified remote file whose server was inaccessible. However, the implementors of Cedar observe that this capability was not often exploited since remote files were normally referenced by specific Birrell

version and

availability more recent

number. Schroeder

pointed

out

the

possibility

of “stashing”

data

for

in an early discussion of the Echo file system [141. However, a description of Echo [8] indicates that it uses stashing only for the

highest levels of the naming hierarchy. The FACE file system [3] uses stashing but does not integrate it with caching. The lack of integration has at least three negative consequences. First, it reduces transparency because users and applications deal with two different name spaces, with different consistency properties. Second, utilization of local disk space is likely to be much worse. Third, recent information from cache management is not available to manage the The

available

literature

integration detracted An application-specific the PCMAIL manipulate nize with

on FACE from

system at MIT [12]. existing mail messages

a central

does not

report

on how

the usability of the system. form of disconnected operation

repository

much

the

usage stash. lack

of

was implemented

in

PCMAIL allowed clients to disconnect, and generate new ones, and desynchro-

at reconnection.

Besides

relying

heavily

on the

semantics of mail, PCMAIL was less transparent than Coda since it required manual resynchronization as well as preregistration of clients with servers. The use of optimistic replication in distributed file systems was pioneered by Locus [23]. Since Locus used a peer-to-peer model rather than a clientserver model, availability was achieved solely through server replication. There was no notion of caching, and hence of disconnected operation. Coda has transparency

benefited in a general sense and performance in distributed

from file

the large body of work on systems. In particular, Coda

owes much to AFS [191, from which it inherits its model of trust integrity, as well as its mechanisms and design philosophy for scalability. ACM

TransactIons

on

Computer

Systems,

Vol.

10,

No.

1, February

1992.

and

Disconnected 7. FUTURE

section

operation

sections

optimization, additional

.

23

development.

In

WORK

Disconnected earlier

Operation m the Coda File System

in

of this

granularity work is also

we

consider

broadened. An excellent

Coda

paper

Explicit

dreds

or thousands

under

work

active

in progress

in the areas of log

of reintegration, and evaluation of hoarding. being done at lower levels of the system.

two

ways

opportunity

Unix.

is a facility

we described

in

exists

transactions

which

and

scope

in Coda for adding

become

of nodes,

the

more the

work

transactional

desirable

informal

of our

as systems

concurrency

In

Much this

may

be

support

to

scale

control

to hunof Unix

becomes less effective. Many of the mechanisms supporting disconnected operation, such as operation logging, precedence graph maintenance, and conflict checking would transfer directly to a transactional system using optimistic concurrency control. Although transactional file systems are not a new idea, no such system with the scalability, availability and performance properties of Coda has been proposed A different opportunity exists in nected

in

operation,

low bandwidth. lines, or that mask

failures,

connectivity.

partial

as provided But

opportunities aggressive file

where

Coda

to support

connectivity

weakly

con-

is intermittent

or of

Such conditions are found in networks that rely on voice-grade use wireless technologies such as packet radio. The ability to

weak cation more

environments

or built. extended

by disconnected

techniques

which

operation, exploit

is of value

and adapt

at hand write-back

are also needed. Such techniques policies, compressed network and caching at intermediate levels.

transfer

even

with

to the communi may

-

include

transmission,

8. CONCLUSION Disconnected

operation

preload

cache

one’s

connection,

log all

reconnect ion. Implementing

is a tantalizingly

with

critical

changes

made

disconnected

simple

data,

continue

while

operation

idea.

disconnected is not

are dynamic

Only in hindsight traditional caching of a lifeline

choices

to be made

about

one has to do is to operation

and replay

so simple.

modifications and careful attention to detail in many agement. While hoarding, a surprisingly large volume lated state has to be maintained. When emulating, integrity of client data structures become critical. there

All

normal

It

until them

involves

disupon major

aspects of cache manand variety of interrethe persistence and During reintegration,

the granularity

of reintegration.

do we realize the extent to which implementations of schemes have been simplified by the guaranteed presence

to a first-class

replica.

Purging

and

refetching

on demand,

a

strategy often used to handle pathological situations in those implementations, is not viable when supporting disconnected operation. However, the obstacles to realizing disconnected operation are not insurmountable. Rather, the central message of this paper is that disconnected operation is indeed feasible, efficient and usable. One way to view our work is to regard it as an extension of the idea of write-back caching. Whereas write-back caching has hitherto been used for ACM

TransactIons

on Computer

Systems,

Vol.

10, No

1, February

1992.

24


.

performance, failures too. transitions system. remote

we have A broader between

shown that it can be extended to mask temporary view is that disconnected operation allows graceful

states

of autonomy

and

in a distributed

interdependence

Under favorable conditions, our approach provides all the benefits of data access; under unfavorable conditions, it provides continued

access to critical data. We become increasingly important and vulnerability y.

are certain that disconnected operation will as distributed systems grow in scale, diversity

ACKNOWLEDGMENTS We wish

to thank

Lily

and postprocessing

Mummert

the file

for her

reference

traces

invaluable

assistance

used in Section

Varotsis, who helped instrument the AFS servers which yielded ments of Section 5.3. We also wish to express our appreciation present

contributors

Mashburn,

Maria

to the Okasaki,

Coda

project,

and David

especially

in collecting

5.2, and Dimitris

Puneet

the measureto past and Kumar,

Hank

Steere.

REFERENCES 1. BURROWS, M.

Efficient data sharing, PhD thesis, Univ. of Cambridge, Computer Laboratory, Dec. 1988. 2. CAT~, V., AND GROSS,T. Combinmg the concepts of compression and caching for a two-level of the 4th ACM Symposium on Architectural Support for file system. In Proceedings Programm mg Languages and Operating Systems (Santa Clara, Calif., Apr. 1991), pp 200-211. 3. COVA, L. L. Resource management in federated computing environments. PhD thesis, Dept. of Computer Science, Princeton Univ., Oct. 1990. 4. DAVIDSON, S. B. Optimism and consistency in partitioned distributed database systems. ACM Trans. Database Syst. 9, 3 (Sept. 1984), 456-481. networks. 5. DAVIDSON, S. B., GARCIA-M• LINA, H., AND SKEEN, D, Consistency in partitioned ACM Comput. Suru. 17, 3 (Sept. 1985), 341-370. in distributed file systems. Tech. Rep. TR 272, Dept. of 6. FLOYD, R. A. Transparency Computer Science, Univ. of Rochester, 1989. fault-tolerant mechanism for 7. GRAY, C. G., AND CHERITON, D. R. Leases: An efficient of the 12th ACM Symposwm on Operating distributed file cache consistency. In Proceedings System Principles (Litchfield Park, Ariz., Dec. 1989), pp. 202-210. Availability and 8. HISGEN, A., BIRRELL, A., MANN, T., SCHROEDER, M., AND SWART, G of the Second consistency tradeoffs in the Echo distributed file system. In Proceedings Workshop on Workstation Operating Systems (Pacific Grove, Calif., Sept. 1989), pp. 49-54. M., 9. HOWARD, J. H., KAZAR, M. L.j MENEES, S, G., NICHOLS, D. A., SATYANARAYANAN, SIDEBOTHAM, R. N., AND WEST, M. J. Scale and performance in a distributed file system. 10.

ACM Trans. Comput. Syst. 6, 1 (Feb. 1988), 51-81. KLEIMAN, S. R. Vnodes: An architecture for multiple Summer

11. KUMAR,

system, 1991.

Usenix P.,

Conference

Proceedings

AND SATYANARAYANAN,

Tech. Rep. CMU-CS-91-164,

tile system types in Sun UNIX. In (Atlanta, Ga., June 1986), pp. 238-247, M. Log-based directory resolution in the Coda file School of Computer Science, Carnegie Mellon Univ.,

mail system for personal computers. DARPA-Internet 12. LAMBERT, M. PCMAIL: A distributed RFC 1056, 1988, memory user man13. MASHBURN, H., AND SATYANARAYANAN, M. RVM: Recoverable virtual ual. School of Computer Science, Carnegie Mellon Univ., 1991. ACM Transactions

on Computer

Systems, Vol. 10, No 1, February

1992

Disconnected


.

25

R. M., AND HERBERT, A. J. Report on the Third European SIGOPS Workshop: “Autonomy or Interdependence in Distributed Systems.” SZGOPS Reu. 23, 2 (Apr. 1989), 3-19. OUSTERHOUT,J., DACOSTA, H., HARRISON, D., KUNZE, J., KUPFER, M., AND THOMPSON, J. A trace-driven analysis of the 4.2BSD file system. In Proceedings of the 10th ACM Symposium on Operating System FYinciples (Orcas Island, Wash., Dec. 1985), pp. 15-24. SANDBERG, R., GOLDBERG,D., KLEIMAN, S., WALSH, D., AND LYON, B. Design and implemenUsenix Conference Proceedings (Portland, tation of the Sun network filesystem. In Summer Ore., June 1985), pp. 119-130. of SATYANARAYANAN, M. On the influence of scale in a distributed system. In Proceedings the 10th International Conference on Software Engineering (Singapore, Apr. 1988), pp. 10-18. SATYANARAYANAN, M., KISTLER, J. J., KUMAR, P., OKASAKI, M. E., SIEGEL, E. H., AND STEERE, D. C. Coda: A highly available file system for a distributed workstation environment. IEEE Trans. Comput. 39, 4 (Apr. 1990), 447-459. SATYANARAYANAN, M. Scalable, secure, and highly available distributed file access. IEEE Comput. 23, 5 (May 1990), 9-21.

14. NEEDHAM,

15.

16.

17.

18.

19.

D., GIFFORD, D. K., AND NEEDHAM, R. M. A caching file system for a of the 10th ACM Symposium on Operating workstation. In Proceedings System Principles (Orcas Island, Wash., Dec. 1985), pp. 25-34. 21. STEERE, D. C., KISTLER, J. J., AND SATYANARAYANAN, M. Efficient user-level cache file Usenix Conference Proceedings management on the Sun Vnode interface. In Summer (Anaheim, Calif., June, 1990), pp. 325-331. 20.

SCHROEDER, M.

programmer’s

22. Decorum File System, 23. WALKER, B., POPEK, operating system. In ples (Bretton Woods,

Transarc Corporation, Jan. 1990. G., ENGLISH, R., KLINE, C., AND THIEL, G. The LOCUS distributed Proceedings of the 9th ACM Symposium on Operating System PrinciN. H., Oct. 1983), pp. 49-70.

Received June 1991; revised August

1991; accepted September

ACM Transactions

1991


Disconnected operation in the Coda File System - CiteSeerX

Disconnected operation in the Coda File System - CiteSeerX

Suggest Documents