Functionality of Gray- Scale Display Workstation

Functionality of GrayScale Display Workstation Hardware and Software in Clinical Radiology1 Brent K Stewart, PhD Thurman Gillespy HI, MD Thomas A. Spraggins, PhD Samuelj Dwyer HI, PhD

This

article

sion

from

examines

the

functional

factors

radiography

film-based

crucial

to radiologic

for

the

gray-scale

successful display

conversystems,

in-

cluding hardware architecture and software requirements, radiologic workstation operations, and a multilayered intelligent user interface. Radiologic workstation operations are logically decomposed into case preparation, case selection, case presentation, case interpretation, and documentation and presentation of the diagnosis. A multilayered software architecture for an adaptive, intelligent user interface is proposed: a hardware interface layer, an object-onented layer, and a knowledge-based layer. The knowledge-based layer is cornposed of three elements: image presentation based on context-dependent models

of diagnostic

requirements,

in diagnostic ologist

decision

to potential

making, lesions

U INTRODUCTION Radiology departments tous

radiographic

knowledge-based and

computer-assisted

systems

for

diagnosis

assistance

to alert

the

radi-

or abnormalities.

are at the film

expert

that

has

center

been

of a massive

the

basis

change

of image

in technology.

management

The

for

almost

ubiqui100

years

is being displaced by digital image management systems, which are known as picarchiving and communication systems (PACS) and image management and communication systems (IMACS). Radiology departments now perform 18%-20% of their examinations with digital imaging modalities. These modalities include computed tomography (CT), magnetic resonance (MR) imaging, scintigraphy, ultrasonography, digital fluorography, and computed radiography performed with storage phosphor imaging ture

Abbrevladons:

API

munication Index logical

system, terms: facifitles

RadioGraphics I

From

T.G.)

the and

assembly. Partially C

RSNA,

RAM

Computers, #{149} Video

1994; Departments University Received

supported

application

program

= randorn

access

diagnostic systems

interface, memory,

aid

GUI RGB

= graphical

= red,

#{149} Computers,

green,

examination

user

interface,

PACS

= picture

archiving

and

corn-

blue control

#{149} Radiology

and

radiologists,

design

of radio.

14:657-669 of Radiology, ofVirginla,

December by a National

University

Charlottesville 20,

1993; Cancer

ofWashington (T.A.S.,

revIsion

requested

Institute

grant

Sj.D.).

Medical Presented February

Center,

1959

as a refresher 1 and

received

R01-CA-51 198. Address

reprint

NE Pacific, course February requests

Seattle, at the

1993

WA

98195

RSNA

12; accepted

(B.K.S.,

scientific

February

17.

to B.K.S.

1994

657

plates.

Furthermore,

dalities

are

applications

are

angiography,

with

the

patient

tions.

developed,

table

In these

be

department

radiologic

digital

tem,

and

the

one

tion

than

other

data

bases),

tion.

Once

formation

has

ar-

must

be able

from

the

display

have

been

The

tive

workstation, report.

many

the

in-

task

of

a consulta-

radiologists

en-

counter preting terpreting

difficulties when they shift from interanalog images on a film alternator to inimages on an interactive gray-scale

display

workstation.

Interactive play

and

workstations

manipulate

are

used

gray-scale

There

sistent

display

ment,

such

tions;

and

as those (d) analysis

for

the

assessment

dalities

that

of the

images.

in intensive

The

emulates

the

quantitative and

must

provide

resolution

mo-

four

by the radiologist to These viewing stasuffi-

number

monitors,

& Therapeutic

Technology

must

from must

be.

workstation

number light

fmally,

the should

video

amount

screen

of light

to extract

data.

radiologist’s

To

emit-

and

performance,

provide

to the

diagnostic

enhance

the

the

requisite

radiologist,

to aid

the

and auxiliary document

cmi-

and

organize

present the information so that it may prehended easily, provide enhancement tools

of

and flexible

to reduce

the

should

cal information

the

Ambient

the

be able

from

the

smaller

monitors is limited compared film alternators. data have been generated, the

image

maximize

a much

smaller

since

information

either

inteffigent

to a minimum

that

x 2,048 highrows of economic or

By far the most widely has been two video moni-

The

especially

like

in two

dictate

software

video

2,048

often

the more

ted from

and

Imaging

on-line

designer would film alternator,

eight

wide,

used configuration tors side by side.

the image

U

to ap-

of items,

stacked

of monitors.

analysis

658

require

considerations

radiologist

contrast

scanning

performed

Although the conventional

monitors

are quite distinct in utilization. For viewing sta-

and

to the

visual

selection

monitors

space

with

or film

for

would

Once

box

the

resolution

of

light

be paid

radiologic

of operations

interface

glare,

evaluation

performance

of the

be kept

applica-

imaging

the

conventional

depart-

designed

design

alternator commonly used review film-based images. tions

care

of specialized

require

the

workstations

these types of workstations because of their differences example, the ultrahigh-resolution tion

outside

must

help, and other general operations. Other design considerations are the number of video monitors to use and ambient lighting

plays)

areas

efficient

environ-

process and the sequences of operations commonly performed during the diagnostic process. The designer also needs to consider the volume of cases interpreted per unit time and

which

in clinical

attention

understanding

are four classes of interactive workstations: those attached to a specific imaging modality (eg, independent console of an MR imager); (b) ultrahigh-resolution workstations (2,048 x 2,560 x 1 2-bit displays with 72-Hz refresh rates), which are used in an image management network for department use; (c) high-resolution workstations ( 1 ,024 x 1 ,024 x 8- to 1 2-bit dis-

(a)

used

clinical

details of workstation design because it is the radiologist’s sole interface with the resources of the digital image management network. Workstation design requires a comprehensive

conditions. to mimic

to dis-

images.

for more

busy

preciate the importance of throughput efficiency. Some ergonomic rules to integrate into the design include (a) providing some type of sensory feedback for successful operations; (b) placing items in a search area where they are most likely expected; (c) providing an easy-to-use interface requiring minimal memorization; (CI) improving target area visibility through highlighting; and (e) presenting a con-

diagnostic

ultimate

to allow

in the very

be re-

the frequency

worksta-

is to generate

However,

to a minimum

of the

should

and

the radiologist, whether with a conventional film-based system or a high-resolution gray-scale display

duced

manipulations

the users

alien-

generated,

to extract

data.

sys-

less

networks

gray-scale data

also remay be conwith the use component and

management

(eg,

image

imaging

received

elements

is the

radiologist

image

that

and

from

Exceptional im-

film.

transmitted,

workstation

required

quality of conof interactions

ment.

analog

digital

the

image utilization

cx-

remain

chived, displayed and manipulated, corded on film. Analog radiographs verted into digital representations of film digitizers. The most crucial in the

with

examina-

a latent

digitally

CT

of the

radiographic

on radiographic data from the

can

as in MR

spiral

80%

examinations,

is recorded The acquired

new

a continuously

Approximately

screen-film

modalities

such

transiting

cient to match the diagnostic ventional films. The number

mo-

and

and

in a radiology

traditional

imaging

refined,

spectroscopy,

gantry.

aminations

digital

being

being

MR

rotating

age

these

continually

be

and

viewer

in interpreting

clinical

information,

and

present

Volume

the

14

corn-

fmal

Number

di-

3

Six

Categories

of Radiology

Category

Operations

Workstation Description

Case preparation management

Notification cessing

study

the

of the local data base system of the availability image information to optimize it for display

image, select field of view, correct image tables for video display), and performance monitor

of a study, prepro(reformat file, resize

orientation, of periodic

and optimize lookup quality control of the

Case

selection

Selection of cases for a given subpopulation of patients, selection of cases from the image management network data base, or automatic selection of the next case on the basis of a radiologist’s work list

Case

presentation

Manipulation

of patient

of the radiologist about the selected in a number Case

Provision to the

to the

of records for medicolegal purposes, referring physician, and for analysis

referring

physician.

The

of workstation

logical

operations

can

be classified into six major categories (Table). The focus of this article is the examination of the functional factors crucial for the successful conversion to i-adiologic gray-scale display systems, user

starting

with

architecture interface

and cognitive

obtaining and images

background and viewing

processes information the images

of modes

Presentation of a completed consultation port) to the referring physician

of diagnosis

decomposition

ware

including reports,

of (a) image processing tools to aid in handling images and formatting them onto a video monitor (cut, paste, resize, etc) and to make signifIcant features (soft tissue, bony structures) easier to visualize, (b) calibrated mensuration facilities (distance, area, volume, local density values) to follow the progression of an object (eg, nodule) or to compare the size or density of such an object in the general normal population, and (c) a display in which requisite comparison images can be seen in proximity to the current study

Documentation of diagnosis

agnosis

that the perceptual

Provision

interpretation

Presentation

data such

are optimized, case, previous

a description

and

design

software.

of hard-

requirements

Workstation

for presenta-

conveying the diagnostic results of future studies of the patient

(ie, a comprehensive

pends on the target elements of a digital

yet concise

re-

application. The essential display system are a host

computer, which serves as the system platform; a video display controller; a frame buffer contaming copious amounts of random access memory (RAM); lookup tables (LUTs); a video buffer board consisting of fast video RAM; the video monitor; an interactive device, such as a mouse, trackball, or light pen; and perhaps a

tion function requirements are then examined, including case preparation, case selection, case presentation, and case interpretation, as well as the documentation and presentation of the di-

specialized image processing hardware board with a digital signal (1) or graphics (2) processing chip (Fig 1).

agnosis.

play

A multilayered

software

architecture

for an adaptive, intelligent user interface is proposed: a hardware interface layer, an object-onented layer, and a knowledge-based layer. Lastly, layer

three are

features

presentation, nostic

of this

discussed:

expert

decision

knowledge-based

context-dependent

system

making,

image

assistance

and

in diag-

computer-assisted

diagnosis.

U HARDWARE Image

display

as a video

display

computer bus supercomputer.

May

1994

ARCHITECTURE systems board

can

example

into

of an ultrahigh-resolution

dis-

of the authors (B.K.S., SJ.D.) were at the University of California, Los Angeles. It was a two-monitor, ultrahigh-resolution system (Fig 2) consisting of four main components: (a) a Sun SPARCstation 470 host platform (Sun Microsystems, Mountain View, Calif), (b) a gray-scale display system (described later), (c) a 2.6-Gbyte high-speed disk array (Storage Concepts, Irvine, Calif), and (ci) a very high-speed network interface (3-5) system

(UltraNetwork

be as uncomplicated

plugged

or as complex The requisite

One

was

developed

while

Technologies,

San

Stewart

et at

two

Jose,

Calif)

(6).

a micro-

as a graphics complexity

minide-

U

RadioGraphics

U

659

Frame Buffer

Video

Buffer (VRAM)

Figure

1.

Diagram

depicts

(RAM)

the

architecture for a generic digital gray-scale display workstation. The size of the arrows represents

the relative terconnection. RAM.

bandwidth VRAM

=

of the invideo

Figure 2. Diagram shows an ultrahigh-resolution gray-scale display workstation (2,048 x 2,560 x 1 2 bit) devel-

oped at the University of California, Los Angeles. This workstation was integrated from a Sun SPARCstation 4/470 and off-the-shelf VMEbus components. DVMA = direct virtual memory access, ESDI = enhanced small device interface, SCSI = small computer systems interface.

This

display system used two 2,048 x 2,560(200 dots per inch) resolution, 72-Hz, progressive-scan, gray-scale monitors (MegaScan Technologies, Hopkinton, Mass). These monitors ran at a 500-MHz video bandwidth and had a measured luminance of 70 foot-lamberts. Both landscape and portrait monitors were used. The monitors were placed side by side, as this configuration mimics the conventional light box or film alternator best. The display controller consisted of two VMEbus (Versa Module European bus) boards: (a) a 4,096 x 4,096 x 12-bit dynamic RAM-based frame buffer and (b) a video buffer board with two 2,048 x 2,560 x 8-bit pixel

platform

This chitecture

(CPU) for quick simultaneously:

transferred frame

handling of multiple processes user input, communication

from

buffer

the

disk

at a sustained

array

directly

into

the

rate of 8 Mbytes!

sec. This system used the X window system, with XView and the OpenLook window manager (Sun Microsystems). In contrast to the ultrahigh-resolution workstations, creasingly vironments.

50 megapixels

image

second.

a scalable processor arprocessing unit

central

servers, data base programs, and window managers. The multiple bus architecture allowed for fast input-output throughput. Images were

video RAM arrays. These two boards were connected through a proprietary frame copy bus, which transfers pixel data at a nominal rate of per

used

(SPARC)

teleradiology

standard being

personal computers are inused as image workstations for

and in less demanding High-resolution,

clinical

en-

high-performance

workstations typically contain video hardware lookup tables that can directly convert extended contrast (12- and 16bit) radiologic images into gray levels for the monitor (Fig 3). RAM and

660

U

Imaging

& Therapeutic

Technology

Volume

14

Number

3

RAM

Video

16-Bit Grayscale

Graphic Hardware

(12-16_bits/pixel)

Display

System

Hardware

LUT

RAM

Grayscale Video Signal

Disk Storage

t

DAC

16 Bit Grayscale

Image

System

RAM

Window and Level Controls

RGB Graphic

Figure 3. Diagram of a l&bit image workstation. The video RAM stores the current 16-bit image. The hardware lookup table (LU7) and digital-to-analog converter (DAC) changes the 16-bit image into an 8bit video signal. The window and level controls regulate the 16-bit to 8-bit conversion.

8-Bit RGB Graphic

Hardware

Figure 5. Diagram depicts how 16-bit images are displayed on 8-bit RGB hardware. The 16-bit image is loaded into system RAM. A software lookup table (LU7) then converts the 16-bit image into an 8-bit gray-scale image. The 8-bit image is loaded into the video RAM and then displayed, as in Figure 4. A software window or level control interactively regulates the 16- to 8-bit image conversion.

I

diologic

images.

computers, special sonal

graphic

8 BIt Image

Color

System

RAM

Palette

Figure 4. Diagram of 8-bit RGB graphic hardware. The video RAM stores an image with 8 bits per pixel. The hardware lookup table (LU7) and digital-to-analog converters (DACs) change the 8-bit pixel into an

RGB color

signal.

The color

palette

regulates

which

256 colors are available. For a gray-scale image, the video signals for the red, green, and blue channels are of equal intensity.

Personal computers, on the other hand, usually contain RGB (red-green-blue) graphic hardware that is not designed to accommodate 12(4,096

digital

levels)

discrete

discrete digital

levels)

gray-scale

bit

systems

use

RGB

or

a color

(65,536

16-bit images.

palette

Eight-

to match

the image value to a specific RGB color value (Fig 4) (7). This method is known as an indexed color model (8). RGB systems with 16-, 24-, and 32-bit

display

mon.

In these

rectly

mapped

capabilities

systems,

increasingly

pixel

RGB

color

value.

none

of the

RGB

graphic

directly

display

16-bit

May

1994

extended

corn-

value

However, can

to an

are

each

options that

such

them are

display

computer

(however,

Grayscale

To display

three

hardware can

display

hardware

on personal

available: for 16-bit

is expensive,

(a) the

buy

perimages usually

(b) perform a onetime conversion of the 16-bit image data to 8-bit (256 discrete digital levels) gray-scale data and then adjust the brightness and contrast of the image by manipulating the color palette (palette animation), or (c) use a software lookup table to interactively convert the 16-bit image data to 8-bit gray-scale values with different window width and window level parameters (Fig 5) (9). With the second method, image appearance can be adjusted in real time, but some image features may not be visible because of the lack of access to the full contrast range of the image. The third method allows “windowing and leveling” through the full contrast range costing

more

of the

image,

than

the

but

there

computer),

is a slight

delay

after

each adjustment. In one program developed for the Macintosh personal computer, the window width and window level controls could be interactively adjusted for a 16-bit CT image about five times per second on a Macintosh Quadra 700 unit (Apple Computer, Cupertino, Calif) (9). This level of performance allows personal computers to perform adequately as low-end image display workstations.

is disystems

contrast

ra-

Stewart

et al

U

RadioGraphics

U

661

U SOFTWARE REQUIREMENTS In designing user interfaces for display stations, the goal is to create an intuitive tool that rapidly becomes transparent to the user, allowing performance of complex tasks with a minimal

or high-powered an inexpensive

amount

of technical

training.

Most

current

larly,

in-

teractive computer systems rely on underlying graphical tools used for display, often referred to as graphical user interfaces (GUI5) (10). Although there are some hybrids, GUIs consist of three major components: a windowing system, an imaging model, and an application program interface (API). Most window-based GUIs pro-

vide

a rich environment

the display and graphics

designed

and manipulation in windows

the

from

a microcompu-

ter can be displayed

on a network

tion.

The

system

dow

function

X window

into

interacts server,

with which

display.

The

tween

user

two

display

separates

distinct

parts.

acts

programs

as an

(client

the

intermediary

be-

applications)

run-

ning on remote computer systems on the network. The client applications or computations communicate across the network with the X

server by means of calls to the low-level (1 2) library of C programming language

to facilitate

of images, text, that may overlap.

win-

The user the X

a display system running is the program controlling X server

sta-

the

Xlib

subrou-

oped

be very tedious,

by the Massachusetts

Institute

of Tech-

nology (Cambridge) and Digital Equipment Corporation (Maynard, Mass) (Athena project), which runs on both UNIX and non-UNIX plat-

should output

forms

the programming tion becomes

in various

instances

(eg,

Motif,

Microsoft explored representaand and per-

sonal

Macintosh

computer

environments.

Finder

and

Microsoft

briefly

described.

The

Windows

systems

X window computing

for networked

system

system is excelenvironments.

was designed

by

splitting

the

job

as a dis-

of drawing

sive

application

be run

on a supercomputer

Technology

panels.

Given

such

tool

of a typical window a matter (although not

kits,

applicaa trivial

fmement of the user interface. Analogous to the open systems interconnection (051) seven-layer model for network communication (13), in which the various networking functions are split into distinct functional layers, the User Interface Reference Model (14)

sysand

the

windowing

layers:

the

task

physical

(window

transport

into

four

logically

(hardware)

or graphics)

dis-

layer,

layer,

the

the com-

ponent (tool kit) layer, and the policy (systemuser interface) layer. Because multiple applications can run simultaneously on a server, rules must

exist

for

arbitrating

conflicting

demands

for input. Unlike most windowing systems, X windows defmes no arbitration or management rules. Rather, it has a special client application

named

the window

manager

that manages

the

positions and sizes of the main windows of applications on the display of the server. In theory, the window manager is simply another client application, but, by convention, it is given

& Therapeutic

text

of assembling components from the tool At the interactive tool level, the windowing system provides interactive GUI programs to ease in creating the prototype and re-

tinct

results on the local server cornthe user sits. A computer-intencan

and

inputsuch

one)

splits

maintaining windows into two parts, with use of the increasingly familiar client-server model. In the client-server model, a computing task is split into (a) the application calculations run on a remote client computer and (b) the display of

the calculation puter at which

system

kit library.

are

tributed, network-transparent, device-independent, multitasking windowing and graphics tern (1 1). In X windows, device independence is achieved

in

so the windowing

provide a tool kit at the standard layer of user interface components,

as menus

Open-

Look); (b) Macintosh Finder; and (c) Windows. The X window system is detail herein because this system is tive of the other GUIs, is nonproprietary, can be run in the UNIX, Macintosh,

The X window

Imaging

obtained

on Simi-

. Window System Concepts Windowing systems with GUIs were developed to give users a means of viewing multiple applications simultaneously and of manipulating both the windows and their contents in a userfriendly manner. Three of the major windowing systems are: (a) the X window system devel-

lent

U

results

and displayed platform.

tines. A windowing system should provide facilities at three levels: the system call level, the standard input-output level, and the interactive tool level. At the system call level, clients (application programs) need the ability to create and lay out windows, to draw in them, and to obtain input events from them. Programming every application at the system call level would

x Windows.-The

662

workstation microcomputer

special

responsibility

Volume

to mediate

14

corn-

Number

3

peting display the

demands for the physical (eg, screen space, color

keyboard).

Applications

the window dow manager ager;

manager hints)

however, to honor

window

managers

OpenLook

window these

Finder

for

and the

only

the

leaves

the

display

roll

(winman-

are

several

example,

the

and

the

imaging

to the

GUI

API,

memory

and

GUI,

(Graphics (Graphic

in

its own

Programming

cx-

a complete

windowing

Interface),

imaging

Interface),

U PRESENTATION FUNCTIONS The logical decomposition of workstation presentation functions is classified into six major case

case

preparation,

presentation,

mentation

case

case

selection,

interpretation,

of diagnosis,

and

the

docu-

presentation

of

the diagnosis (Table). These functions provide for the interactive selection, positioning, and sequencing of image data on the display screen, including (a) patient selection from the image archive data base, (b) selection for presentation on

the

ite

image

monitor, display,

Three

major

stack, lows

tile,

(c)

multimodality and

modes

and

cine

(ci)

The

of images

composdisplay.

of presentation modes.

presentation

and

consultation

are the

stack

mode

sequentially,

alone

at a time, with windows overlapping. The tile mode allows presentation of images side by side. The cine mode provides dynamic sequential

viewing

contiguous

of either images,

spatially

or temporally

or of multispectral

(eg,

Ti-

weighted, T2-weighted, and proton density MR) images of the same imaging plane (15), in either stack or tile mode. Stepping through the cine stack is typically done by means of either clicking a mouse button or rotation of a dial or trackball. In some instances, this stepping can occur automatically, at a rate that the user can adjust.

The

loop

through

May

1994

cine

the

stack

stack,

can

either

proceeding

beyond

the

or stop

the

the

image.

wealth

generating

if an attempt

beyond first

last

of different

images

and

image

imaging

the

and

pictorial

index.

modali-

possibly

In the

is

or to

large two

addi-

important:

survey

mode,

images are displayed at a lower resolution than that at which they were acquired to allow more images to be displayed simultaneously. For cxample, a chest radiologist may wish to see the latest eight plain radiographs for a patient on a system with only two monitors. In the survey mode on a 2,048 x 2,048 video monitor, the eight 2,048 x 2,048 chest images would be disas four

each size.

images

on

each

monitor,

with

image displayed at a 1 ,024 x 1 ,024 matrix The pictorial index is an index of all image

modality

examinations

available

for

a patient

and

API.

categories:

ties

image,

conveyed in the form of minified versions of the actual diagnostic images. A typical size for a pictorial index image is 1 28 x 1 28 pixels; however, the chosen size also depends on the matrix size, physical dimensions, and layout design of the video monitor.

these

specffied

is also

with

of an

model

Finder;

back

With

played

pro-

functions

imaging

and

Device

GUIs

Quickdraw

the

Windows

proprietary

of API (win-

Macintosh

Toolbox,

integrated

Microsoft

model

and

and

(usually

extant

all of the

primitives,

system

choice The

speciGUI

vendor

other

system,

are tightly

actly.

the user,

combines

its read-only

pieces

to the

coupled.

windowing

graphics

of the

model

tightly

prietary

portion

first

to proceed

survey

Winsystem

and

manager)

more

Microsoft

to the

made

number of images for a given patient, tional viewing modes are especially

universal

X window

windowing

postscript)

dow

and

to give

is not

There

manager

dows.-Whereas

are

last

manager.

Macintosh fies

required

manager

hints.

available,

window

window

are

of a and

certain information that help the window

the

obliged

resources resources,

. Case Preparation Display optimization involves formatting both the spatial and gray-level data and video monitor quality control. Spatial optimization depends on the workstation host, storage devices, image processor, and monitor. It includes standardizing byte or pixel ordering, transferring data onto appropriate storage devices, and resizing the field of view by means of subsamping, interpolation, or cropping. Gray-level optimization is especially important for displaying images that have greater than 8-bit density resolution (the dynamic range of CRT [cathode ray tube] displays). For example, a 1 2-bit CT image typically contains CT numbers in the range of 200-1,800. Thus, a blind 12- to 8-bit conversion will result in a displayed image with suboptimal contrast. Window and level operations must then be performed manually or through a selies

of predetermined

lookup

tables

that

step

through the brightness and contrast ranges of the video monitor. Predetermination of optimal lookup tables is desirable for bone, soft tissue, lung, etc. Optimal default lookup tables can be computed on the basis of a histogram of the image, the image type (eg, anatomy, modality), and an estimation of the signal-to-noise ratio.

continuously

from

the

Stewart

et al

U

RadioGraphics

U

663

Video odically

monitor checking

luminous

quality control includes perithe analog response and the

output

of the

video

monitor.

The

ana-

log response is checked with a video test pattern, with the most widely used pattern being that of the Society of Motion Picture Test Engineers (SMPTE). This pattern is employed to test both spatial and contrast resolution by using ruled areas of different spatial frequency and orientation and contiguous patches of various gray

levels,

respectively.

used

to check

the

monitor. This cause monitor function contrast

A luminance

brightness

response

check is especially luminance output

of time. resolution

(a)

a text

list

is

of the

important decreases

A weekly check and luminance

. Case Selection Image sets may be selected means:

meter

beas a

of spatial and is sufficient.

by one

index

or

(b) an image archive query, which initiates a global search of all images available for the selected patient within the PACS network. This operation uses the PACS data base, which returns

table

records

describing

each

PACS

image

matrix.

monitors,

(two

different

quarter

each

im-

concentrates

screening on

for comparison. than one image

The the

require

time optical

a much

required disk

longer

to transfer

archive

involves

time

to re-

images not

only

the transfer rate once the optical disk is placed in the disk controller unit by a robotic arm (250-500 kbytes/sec), but also the access Iatency if the disk is in the jukebox but not in the controller (8- 1 0 seconds). Images are received by the workstation in the background, so that the radiologist may review another case if there are many images to be transferred or if they reside on the optical archive. On receipt of these images by the workstation, the local workstation data base is immediately updated and a message window informs the user.

& Therapeutic

Technology

that

most

ages tion

obtained of contrast

and

level

These images may span set and may include im-

before and after material, with

settings,

with

the administravarious window

different

pulse

se-

quences or modalities, and at different times. Images that have previously been diagnosed and annotated are by default presented in the most compact, concise mode on the monitor. most

graphic

relevant

images

together

overlays

for

with

must

archive

the radiolo-

show the pathologic condition. Typically, only a select few images are vital for making a radiologic diagnosis. The current practice in a film-based system is to shuffle the films and line them up on the alternator according to the

has been

optical

sets

clearly

important

the

or one-

images

diately,

on

two

image

phase,

Case Presentation Once a patient case

residing

span

im-

of a monitor

selected

posed

images

may

different

age set, a network message is sent from the workstation to a server executing on the appropriate archive controller. Images residing on the magnetic disk cache are sent almost immewhereas

is

im-

sets per monitor), (four

per monitor). After the initial gist

set

one-half

image

the

For

image

of a monitor

displayed

network.

the

displayed quality. This

for cross-sectional

monitor,

only

the

mode

An

one

are

from

Imaging

display

study

from

maximizes

ages (eg, MR and CT images), for which the age dimensions are smaller than the monitor

Thus,

trieve.

U

display

ies

disk

664

the default

file and its location within the optical and magnetic PACS archives. From the results of this archive query, the user may select specific studto retrieve

mode

of images simultaneously compromising image

need more

of two

or a pictorial

The tile presentation number without

any

(pointers

and

the

corn-

text).

.

decisions

the presentation primary overlay, display sues

area

made

several

concerning

of patient

case information. is how to arrange the image, information on the available

concern and text of the

(a)

include

selected,

be

monitor.

how

Crucial

many

design

image

Of

is-

sets or com-

parison images are necessary to be displayed multaneously to diagnose the current patient study and (b) how many monitors are necessary,

since

two

some cases, split-screen

The text cludes

patient

ology

services

monitors

even with operations.

may

the

information

insufficient

ability

in

to perform

to be presented

demographics,

request

be

si-

case

(ie, requisition)

in-

history,

radi-

informa-

tion, previous radiologic reports, listing of available studies, radiologic technique information, image processing and analysis values, user instructions, and more. This information is cx-

Volume

14

Number

3

tracted

from

the

radiology

information

(RIS), so the establishment face is extremely important. dude

what

able how

type

of information

to the user, to edit the

should

be

are obtained

ports

or text

the

patient

be

on the

the

text

keyboard.

re-

pa-

should to be

by double-clicking

desired

editing may

category

dictation The

information information

windows

field. be

and

Alterna-

used

to Se-

to edit.

as cross-sectional

modalities

ing and CT begin will likely become

to produce

more

Image

Processing

there

are a plethora

egated

found

is mis-

Quantification

25%). the size

refers

position

of image

enhancement measurements

catheter

has decreased to manipulate

windows,

perform

operations, and make of the image data.

image

on

(lesions,

tinum, dude preset

nodules,

bone

fractures,

medias-

soft tissue). The basic tools required inintensity transformation tables (automatic windows, manual independent window

and

level

age

enlargement

pan).

control,

and and

Adjustment

image

reversal)

translation

of the

and

(zoom

intensity

im-

and

transforma-

undoubtedly be the most quently used image processing operation. probability, it will be performed on each tion

will

tables

played

image.

It therefore

simple

to use.

Furthermore,

must

be

performed

In addition,

lays

(text

tion

capabilities

necessary and

digitized

must

the

in real the

annotations are

function plain

ability

be

important.

for computed radiography.

Rotation

1994

radiologists

and are not

three

image

types

of clini-

processing

func-

subtraction,

compres-

data

volumetric

displays,

extraction. Functions.-Image

to extracting

some

(eg, size of tumor, density

other

tumor

of a particular

cardiac

of knowledge

volume

bone

ejection

analysis

sort

change,

region,

fraction,

mass

spleen

vol-

size)

from

heart

presented image data. The basic analysis tools should include calibrated spatial measurements in one, two, and three dimensions; calibrated density measurements (eg, Hounsfield units); and calibrated temporal measurements (eg, arterial flow). These measurements should be compared with similar measurements for normal and abnormal populations as well as with measurements for the same patient obtained at specific intervals over the course of treatment. Other typical interactive analysis functions indude length, surface, and volume mensuration; analysis;

and

texture

analysis.

of Diagnosis diagnostic

process

is not corn-

rotais a

diologic

over-

are

seven possible combinations including 90#{176} rotations, clockwise and counterclockwise about the z axis; 180#{176} rotations about the z and y axes; 180#{176} rotation about the y axis with 90#{176} ro-

May

func-

stations (eg, or unsharp

that most

enhancement,

. Documentation The radiologic

radiography There

processing

plete until the referring physician receives an unambiguous, accurate, concise, and accountable report from the radiology department. Dctails as to how to document the results of a ra-

graphic

and

contour

dis-

extremely

pointers)

edge

and

histogram

time.

and

sion,

image

freIn all

operations

to add

clinically.

workstations

the

Other

include

ume,

physical

Interactive Manipulation Functions.-Interactive manipulation operations should enhance the visibility of anatomic and pathologic features

important

found

to the analysis

with

and

imag-

voxels,

Functions.-Although

we have

masking),

typically

Swan-Ganz

as MR

either do not have the patience for them or do not value their utility. These functions are rel-

tions

the

generation images,

isotropic

tions available for use in display adaptive histogram equalization

. Case Interpretation Once the case information is initially displayed, the radiologist must interpret the radiologic images (eg, this radiographic pattern is consistent atelectasis,

and

such

of image

cal workstations.

placed, the tumor volume The radiologist may want

counterclockwise;

re-

demographics,

accomplished

pull-down

lect

the

clockwise

180#{176} rotation about the z axis. The of oblique views and three-dimensional

avail-

Text

transcribed

with

and requisition Selecting the

cursor

lively,

from

editing

can

be

displayed.

entered

tient history, be consistent. edited

should

simultaneously

for

tations

where to place these data, data, and how much data

ports method

system

of a PACS-RIS interCrucial issues in-

study

properly,

however,

(ie, so that

the

report is clear, concise, and useful to the referring physician) have only recently been formally provided (16).

Stewart

et al

U

Ra4ioGrapbics

U

665

Expert

Rule-based

of multilayered

Image

Speci

Classification

User Specific

Understanding

de-

Object-oriented Layer

of an

intelligent

PACS. The programming, data representation, and results become more abstract as one moves from the hardware interface layer

Tools that documentation recting

the

are

I 2.5kx2kx8

X-Windows

I

Window

I

Manager

Xlib

to the object-oriented layer and finally to the knowledge-based layer.

believed to be necessary include graphic pointers

viewer’s

attention

to crucial

dictation

mode

perform

normal

processing, multaneously.

allows

screen

. Presentation Both the radiologist

can benefit report

greatly

of a given

views

the

tained

some time the diagnosis

know

made.

the

user

workstation

images

and

the

from case.

When

the

a case

that

from

the

patient’s

The

referring

terpretation

physician

si-

radiologist

re-

Bus

dent,

not

or medical

has

had

the

usage.

underproblem.

the expert

friendly on an individual scale); no more time to use than the current process (high throughput); (c) is caeffectively handling both simple and tasks; (ci) is cooperative in that it voland organizes information; (e) is con-

in-

(ie,

(a)

an image

(ie, user (b) takes film-based pable of complex

sensitive

that

requires

system

text

and

PACS

the radiologist

is adaptive

the images

to the

user

and information

presented are context dependent); and (J) uses object-oriented design and expert-system, rulebased models for control. The multilayered architecture for the software of such an adaptive, inteffigent

PACS

interface

is given

layer,

the

6: the object-oriented in Figure

the knowledge-based data

more abstract the layers.

Technology

who

on workstation

display

ming,

& Therapeutic

student

of instruction

Ultimately,

and

Imaging

to the referring

require as few user operaOne should not assume that

workstation fellow, resi-

ware

U

of a diagnosis

the referring physician is an expert user. The physician may be a new

unteers

oh-

immediately requires

of a study.

Copy

U IISTELUGENT

(image

physician

clinical

Hardware Interface Layer

Buffer

benefit

ago, it is advantageous to of the study as previously history

Frame

Presentation

structured were

Frame

I

physician should tions as possible.

display)

referring

a concise,

.j

for

of Diagnosis

He or she can then

stand

image

II

12-bit]

and

operations

formatting,

Video

for diregions

to dictate

I4kx4kx

bit

Buffers

:

Notifier

of interest; text pop-up windows for summarizing problems, conditions, or concise impressions; and digitized voice report playback for more elaborate descriptions. The digital voice

666

Knowledge-based Layer

Processing Analysis Understanding

worksta-

lion software adaptive,

Assisted

Diagnosis

Perception

Task

Figure 6. Diagram picts the architecture

Computer-

System

1

Radiological Procedures

layer.

representation,

and

as one moves

Volume

hardlayer,

The programresults

upward

14

become through

Number

3

Properties: ID String

Image Date Time

ID Number

Patient Anatomical

Description

Patient

Orientation

Technique Display

x

Factors Parameters

Dimension

Y Dimension

z Computed

(bit depth) ID ID

Dimension

Radiological

Tomography

Optical

Device Archive

8.

7.

Figures 7, 8. (7) Diagram shows an example of the class hierarchy for a radiologic image. Image is the super class for the original, raw, derived, region of interest, and report classes. A raw image refers to unprocessed data acquired by an imaging modality before backprojection or transformation. A derived image refers to an image produced through image processing of one or more images of a single modality or through combining information from multiple modalities, as in composite imaging. Digitized film, computed tomography, and computed radiography are subclasses of the class x-ray. (8) Diagram shows the typical properties of a radiologic object: in this case, the image object from Figure 7. ID = identification.

Hardware

Interface

interface

layer

consists

mainly

window

software

the

of the

area,

coordinates

user

associated

device

drivers.

hardware

layer

should

be provided system

hardware

the

GUI.

and The

monitor

interactions,

ages

display

system

X window

manages

with of the tines.

hardware

Layer.-The

drives

through

The window

X

display

and man-

by the

manufacturer of caffing

rou-

Object-oriented Layer.-The object layer is an object-oriented (17) model for the organization, structure, and hierarchy of radiologic ohjects, which may be images, reports, or patient folders (analogous to the patient film jacket).

There

is a one-to-one

real-world This allows

correspondence

between

entities and objects in the system. direct representation of the items requiring management. Object-oriented systems describe relevant entities as strictly encapsulated units (objects) that communicate through message passing. Objects may be cornposed of other objects, called composite ohjects. Each object has attributes and methods. Attribute values represent the state of the ohject that is accessed or changed through messaging. Each set of images, for example, can be distinguished by attributes, making it easier to access by means of simple or complex queries.

May

1994

per classes

(also

super

encapsulated

class

be thought

can

interaction

software

as a library

Objects are structured into class hierarchies in which instances of objects receive the properties of their class, as well as those of any su-

referred

to as inheritance properties).

of as a collection

of

A class

of objects

sharing a common structure and set of behaviors. Figure 7 provides an example of the class hierarchy for some objects of radiologic interest. An example of the properties of a radiologic object (image) is given in Figure 8. Because the storage, retrieval, and delivery of images, graphics, text, and eventually hypermedia with digitized sound and full-motion video may be required, the functional abstraction provided by an object-oriented model is appropriate. (Hypermedia is the creation and representation of links between discrete datum,

which

clips,

text

Two

can be graphics, documents,

and

images, audio

video

segments.)

motivations for using object-oriit supports the natural pathways of human reasoning and classifIcation and that the developed software can have a high rate of reusability. Object-oriented de-

ented

sign

lows

major

design

is good

the

are that

for

designer

data

structuring

because

to attach

a number

Stewart

et al

it al-

of meth-

U

RadioGraphics

U

667

ods

to each

tion

space

class

or prototype,

in an

ily be

partitioned

into

various

levels

of abstraction.

design

with

used

for

of user-guidance

that

support

interaction,

and

icons.

A logical

but

one

that

However, provides

only

create the

object

end

user

interaction

user.

user

To

interface

a higher-level

for a

layer

will

bilities:

multimedia

object

management,

new

support

genera-

inteffigent (18),

capa-

high-level

rule-based

data

processing,

reading

proce-

users,

and

corn-

user

Higher-level

interface

abstraction

self-modifying

of this

kind

Image/icon

versity

(22).

expert

system

for self-

interfaces.

of adaptive

system

In this

is selfregulat-

is required

An cx-

user

interface

developed

system,

(Icon)

is

at Yale

Uni-

a knowledge-based

critiques

proposed

diag-

nostic hypotheses. The choice of alternate hypotheses is a function of the spread of compet-

which

however,

based

on

ologic

features.

are weighted.

make

pattern

diagnostic

Radiolodecisions

recognition

and

Therefore,

subtle

in addition

morph-

to the

image displayed for diagnostic purposes, the expert system selects, retrieves, and displays reference images (Image) in support of the case

radiologic

fmdings

diagnosis,

procedures

fairly

rules

requirements

center dependent), cific information dependent, with gists,

etc.

The

(radiology

universal),

formation

clinicians,

dependent

about

task-specific

(department

physicists,

of the

cess.

This

and

at which

technologists).

intelligence

display 24

CT

an easy

2,048 only

display

radiologist’s

is not

two-monitor

x 2,048

for

display

whereas of these

scanning

as,

the films

(or

pro-

example,

a

system

(or can

96 CT sections).

Somehow, the images must be organized, with the most relevant brought to the foreground. Relevant images and information on this costly and limited display system must be organized for

easy

comprehension

contextual goal

basis.

is to formulate

text-dependent

and

One

means rules

models

based

presented

on

of achieving on

of diagnostic

explicit

require-

a

this con-

analysis standing

level

images.

disease

category,

of an inteffigent

The output

at the

middle

at the

have been Chicago (23-25). CAD

user

of such

highest

level,

and

level.

inter-

an inter-

developed However,

image

Many

highly ad hoc which uses

pirically

rules

determined the

the

accuracy

false-positive

under-

efforts

in

at the University of progress has been

slow because of the gorithm development,

proved,

Technology

suspected

face is the alerting of the radiologist to potential lesion or abnormality sites or providing measurements of specific areas or patterns in the image. This output requires the interface to use image processing at the lowest level, image

although

& Therapeutic

the

face is that of computer-assisted diagnosis (CAD). CAD refers to a diagnosis made by a radiologist who has taken into consideration the results of an automated computer analysis of ra-

diologic

can

digitized films film alternator

for

and (c/) morphologic variations of the fmding. These image sets are provided to assist in the radiologist’s interpretation of the abnormalities, correct classification of the fmdings, and cornprehensive understanding of the significance of the findings.

The highest

can be of image under-

visual task,

14 x 1 7-inch

two

sections), eight

in-

or medical

added to PACS is in the presentation data. This requires a comprehensive standing

knowl-

and rules about user-sperequirements (individual user the users including radiolo-

The first level

Imaging

in-

being critiqued. The following image sets, on the basis of the diagnostic hypothesis, are displayed: (a) proved examples of the suspected diagnosis, (b) differential diagnoses that could cause the observed fmding, (c) spectrum of

and

U

radiologic

user information

edge-based layer, the domain of expert systems and model defmition and codifIcation, provides this intelligence. This is the level at which rules are defined from observed models: rules about

computer-assisted

668

varying

of cases.

gists,

Layer.-The require new,

a cooperative

to the

of workstation

ing diagnoses,

atop

layer.

Knowledge-based tion of PACS

plexity

the

interface

platform.

intelligent

requires

spectrum

ample

opera-

to the

object-oriented

a dynamic,

workstation

of different

dures,

mediating,

is an cx-

system

creating adapts

The cooperative

fields

tile

that

ing.

are those

as menu

supports

a static

is

(19-21),

needs

operation.

objects

is transparent

the

terface

eas-

systems

system

such that

ments

can

represent

display

frame-buffer

of an object

tion,

that

and

An example

ample

informa-

Object-oriented

to image

guidance

the

system

entities

regards

user

and

object-oriented

and

of CAD

rate

Volume

nature of alsets of em-

techniques. programs

remains

14

Also, has

im-

high,

Number

3

which tolerate.

is something that radiologists In the future, CAD programs

routinely

applied

applications structured,

to all image

most yet

likely

will

generalized

may

types,

but

require image

14.

not

could

be 15.

wider

prototype

a more analysis

para-

16.

digm. 17.

U REFERENCES 1 Saffari B. Putting .

2.

3.

4.

Byte

1989;

14:

259-272. Mills KS, Wong GK, Kim Y. A high-performance floating-point image computing workstation for medical applications. Proc SPIE 1990; 1232:246-256. Stewart BK, Lou SL, Wong WK, Huang HK. An ultra-fast network for radiological image communication. AJR 1991; 156:835-839.

18.

Stewart BK, Lou SL, Wong A, Chan KK, Huang HK. Performance characteristics of an ultrafast

20.

network for PACS. Proc SPIE 1991; 1446:141153. Stewart BK. Three tiered network architecture for PACS clusters. In: Huang HK, Ratib 0, Bakker AR, Witte G, eds. Picture archiving and communication systems in medicine. NATO ASI Series F: Computer and System Sciences, vol 74. Berlin, Germany: Springer-Verlag, 1990; 113-118.

5.

6.

Cho PS, Ho BKT. ware architecture.

7.

8.

9.

10.

11

DSPs to work.

.

12. 13.

May

Display

workstation:

19.

Kopans porting.

(abstr).

2 1.

22.

23.

25.

1992;

185(P):416.

mammography reAm 1992; 30:257-

261. Cox BJ. Object oriented programming: an evolutionary approach. Reading, Mass: AddisonWesley, 1987. Fox EA. Advances in interactive digital multimedia systems. Computer 1991; 24:9-21. Wendler Th. Cooperative human-machine interfaces for medical image workstations: a seenario. In: Lemke FlU, Rhodes ML, Jaffe CC, Felix R, eds. CAR ‘89: computer assisted radiology. Berlin, Germany: Springer-Verlag, 1989; 775779. Grewer R, MOnnich KJ, Schmidt J, Svensson H, Wendler Th. Object-oriented design for mcdi-

Verlag, Wendler

In: Lemke

R, eds.

HU, Rhodes

ML, as-

CAR ‘89: computer

Berlin,

1989; 780-784. Th, Wein B.

Germany:

Springer-

A rule-based

model

of di-

agnostic information requirements for the design of adaptive image workstations. In: Lemke H1J, Rhodes ML,Jaffe CC, Felix R, eds. CAR ‘91: computer assisted radiology. Berlin, Germany: Springer-Verlag, 1991; 629-635. Swell HA, Fisher PR, Cohn Al, Mifier PL, Mutalik PG. Expert system-controlled image display. Radiology 1989; 172:487-493. Giger ML, Ahn N, Doi K, MacMahon

puterized

24.

Radiology

DB. Standardized Radiol Clin North

cal workstations. Jaffe CC, Felix sisted radiology.

hard-

In: Huang HK, Ratib 0, Bakker AR, Wifle G, eds. Picture archiving and communication systems in medicine. NATO AS! Series F: Computer and System Sciences, vol 74. Berlin, Germany: Springer-Verlag, 1990; 5 1-56. Gillespy T III, Rowberg AH. Displaying radiologic images on personal computers. J Digital Imaging 1993; 6:151-163. Apple Computer. Graphics overview. In: Inside Macintosh. Vol 7. Reading, Mass: Addison-Wesley, 1991; 16-16, 16-17. Gillespy T III. Optimized algorithms for displaying 16-bit gray scale images on 8-bit computer graphic systems. J Digital Imaging 1993; 6:25-29. Hayes F, Baran N. A guide to GUIs. Byte 1989; 14: 250-257. Jones 0. Introduction to the X window system. Englewood Cliffs, NJ: Prentice-Hall, 1988. Wood P. Introduction to Xlib programming. UNIX World 1989; 6:121-127. Tanenbaum AS. Computer networks. 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1989; 14-27.

1994

Lee E. Window of opportunity. UNIX Rev 1988; 6:46-61. Chang PJ. MR multiband viewing workstation

detection

of pulmonary

H.

Corn-

nodules

in digi-

tal chest images: use of morphological filters in reducing false-positive detections. Med Phys 1990; 17:861-865. Katsuragawa S, Doi K, Nakamori N, MacMahon H. Image feature analysis and computer-aided diagnosis in digital radiography: effect of digital parameters on the accuracy of computerized analysis of interstitial disease in digital chest radiographs. Med Phys 1990; 17:72-78. MacMahon H, Doi K, Chan HP, Giger ML, Katsuragawa S, Nakamori N. Computer-aided diagnosis in chest radiology. J Thorac Imaging 1990; 5:67-76.

Stewart

et a!

U

RadioGrapbics

U

669