University of Otago Interactive Visualisation Tools for ... - OUR Archive

University of Otago Te Whare Wananga O Otago

Dunedin, New Zealand

Interactive Visualisation Tools for Analysing NIR Data Hayden Munro Kevin Novins George L. Benwell Alistair Mowat

The Information Science Discussion Paper Series Number 96/13 July 1996 ISSN 1172-455X

University of Otago Department of Information Science The Department of Information Science is one of six departments that make up the Division of Commerce at the University of Otago. The department offers courses of study leading to a major in Information Science within the BCom, BA and BSc degrees. In addition to undergraduate teaching, the department is also strongly involved in postgraduate programmes leading to the MBA, MCom and PhD degrees. Research projects in software engineering and software development, information engineering and database, artificial intelligence/expert systems, geographic information systems, advanced information systems management and data communications are particularly well supported at present. Discussion Paper Series Editors Every paper appearing in this Series has undergone editorial review within the Department of Information Science. Current members of the Editorial Board are: Mr Martin Anderson Dr Nikola Kasabov Dr Martin Purvis Dr Hank Wolfe

Dr George Benwell Dr Geoff Kennedy Professor Philip Sallis

The views expressed in this paper are not necessarily the same as those held by members of the editorial board. The accuracy of the information presented in this paper is the sole responsibility of the authors. Copyright Copyright remains with the authors. Permission to copy for research or teaching purposes is granted on the condition that the authors and the Series are given due acknowledgment. Reproduction in any form for purposes other than research or teaching is forbidden unless prior written permission has been obtained from the authors. Correspondence This paper represents work to date and may not necessarily form the basis for the authors’ final conclusions relating to this topic. It is likely, however, that the paper will appear in some form in a journal or in conference proceedings in the near future. The authors would be pleased to receive correspondence in connection with any of the issues raised in this paper. Please write to the authors at the address provided at the foot of the first page. Any other correspondence concerning the Series should be sent to: DPS Co-ordinator Department of Information Science University of Otago P O Box 56 Dunedin NEW ZEALAND Fax: +64 3 479 8311 email: [email protected]

Visualisation

Interactive Ilayden

*I

Munro

Kevin

for

Tools

Novinsl

Analysing NIR

GeorgeBenwell*

.luly 18,

Data Mowati

Alistair

1996

Abstract This

describes

paper

measured

using

dimensional

a

These

of fruit.

acteristics

tool

being developed characteristics, such

non~destruc:tive ofthe

nature

NIR

Near

Infrared

data

introduces

to

allow

acid

sugar, Reilectance

device

visualise

to

and

the

moisture

visualisation to

necessary

design

scientific

1

be

can

The

four

problems.

The

two

dimensional

the dataset, a fully understand flexible visualisation tools. provides

user

Polhemus interaction, interactive N1R,spectrosPolhemusFa,s"I‘rakTM,interaction,interactivegraphics, Fa,s"I‘rakTM, K ‹5yW()]‘(lSZ

interfaces, visualisation,

char~

ripening

content,

(NIR) analysis techniques.

interesting

some

only provides two dimensions, making it In order to help the methods for representing the data. graphical display system is created with an interface that display

users

as

graphics,

visualisation.

Introduction

can be described in two ways; as a tool for discovering and understandto the It is used to present information communicating and teaching tools user in visual forms that appeal to his/her intuitive understanding. Thus, visualisation of knowledge from often very complex datasets. facilitate the extraction to visualise data about the ’I‘his paper describes a tool that is being developed to allow users olffruit. ’lfliese such as acid and moisture content characteristics characteristics, ripening sugar, Near Infrared Reflectance can be measured using non-destructive (NIR) analysis techniques [6, 8, QI. By cornparingthe information gained by NIR analysis with the physical ripeness of the which NIR readings correspond to a good product. Once fruit it will be possible to determine fruit this has been done NIR analysis will be able to be used alone to measure quality. ’l‘he information gained using NIR analysis could then be used to tailor the fruit growing and market acceptance. It may also be possible to measure ripening storage processes to maximize for prediction of fruit quality before the fruit has reached maturity. characteristics nature of the NIR data introduces some The multidimensional interesting visualisation problems. The data is four dimensional, whereas the display device only provides two dilt is therefore to discover two dimensional methods for representing the mensions. necessary data the users of the do not know what are Also, application they looking for in the to create an to assist data, so it is important application with a high degree ol interaction ln order to tackle these problems this implementation their exploration. provides a suite of interactive visualisation tools rather than a In addition the user will single display mode. to modify the display for his/her own be given the power This should provide the purposes. maximum iiexibility of the computer to assist the user s exploration of the dataset. The work described here was undertaken as a joint research project by Otago University

Scientific

ing; and

visualisation

as

a

tool

for

and Hort-l-Research. Department

oflnforznation

’Department of

Computer

iHort~{-Research,Ruakura

of

Science, University Science,

University

Agricultural

Centre,

of

Otago, P.O.Box Otago, P.O.Box

Ruakura

Road,

1

NZ.

56, Dunedin,

56, Dunedin, Private

NZ.

Bag 3l23,

([email protected]

nr)

([hayden|novins}@cs.otago.ae nz) Hamilton,

NZ.

(amowat@ho| t.cr|

117

2

Description

The of

data

for

light

subsurface

this

project

discrete

at

of

the

of Data

provided by I-lort-4-Research.

was

in

wavelengths fruit.

Collection

the

infrared

near

The

spatial coordinates Gala apples and kiwifruit.

data

It consists

of these

locations

of reflected

several

at

spectrum,

also

were

intensities

locations

the

on

The

provided.

was collected from The spatial coordinates were obtained using a Polhemus FasTraldevice device Eachfruit Each fruit placed inside a cylinder with the longer axis perpendicular to the cylinder s straight edge, shown in Figure 1. A longitudinal line of nine holes ha.d been drilled at 15 intervals

was as

around the holes

eighth

degree

the cyIinder s to

measure

of the

circumference. a

fruit.

position

The

fruit

The

stylus

the fruit

on

then

of the

surface.

rotated

tracker

pushed through These point locations correspond was

each to

of one

/L5

degrees to collect the next set of points sampled. ln total 72 points, distributed across eight points of latitude and nine points of longitude, were selected for spectral analysis. Because the spatial points were collected at angular intervals, a polar coordinate transform and

so

used

was

until

on,

to

After

the

the

whole

surface

the coordinates

reconstruct

Fas’Ira.kTM device

Rellectance

(NIR) probe

was

at

the

subsurface

at

at

1100

fruit s

was

fruit

had

inserted each

had

been

in SD. been into

of the

removed

from

the

the 72

guide holes points. The

guide holes,

to record

the diffuse

a

Near NIR

Infrared

spectrum

diffuse

NIH spectrum was sampled shows the sampled spectrum amount of energy that different measured There is an wavelength error associated with each measurement because for each of the 1100 wavelengths sampled the a Qnm photosensor collected light over rather than at one discrete range, wavelength on the A file was Created spectrum. of the 1100 sampled wavelengths for containing the intensities each point.

wavelengths

in the

range 507.66nm is reflected at each

],026.23nrn.

to

Holes

¢

¢

"

"

-

1: Cross

Visualisation

3 The

data

the

fruit s

maintain

described subsurface

their

for Polhemus

and

NIR Probe

5 Plcisiic

Figure

The

section:

cylinder

selecting spatial points

on

a

fruit.

Methods

in the

previous section

be considered

can

four

dimensional:

where

wavelengths were niczasured must be described spatial relationships; and the spectral wavelength inforrnation

the

points

in 3D in order

becomes

a

on

to

fourth

dimension.

Since

the

data

is four

dimensional

and

the

display

is

only

two

dimensional, no single simultaneously. Also, thc user cannot predetcrniine which view Fortunately the computer s flexibility allows a suite of visualisation tools presenting different data to be conceptual views ofthe supplied. By using the tools interactively and in concert the user can discover features in the visualisation

method

will

be able

to

display

all aspects will be best.

data.

2

of the

dataset

The

tools

divided

categories. The first is the spectrum intensity at all I1‘1Cil.Sl1I‘ ‹(_lwavelengths for a single point on the fruit surface. The second category contains spatial views. Spatial views 3D fruit and display intensity values for a single wavelength at all surface points. geometry These views are described in detail in the following subsections. The software environment is OpenGL* working with Tcl/Tk2. OpenGL is a standard the degraphics library designed for real-time applications [10]. It is designed to encourage The programs are written velopment of portable programs. using the (J language to call and in~ procedures from OpenGL. The OpenGL library makes it particularly easy to create teract with the polygonal models used in the programs. Tcl/Tk was also chosen for its flexibility. Tk provides a library of tools that are used for creating application interfaces; and Tcl is an interpreter that builds the progra1n s interface from command scripts [1], 16]. Using scripts rather than library calls allows the interface to be modified while the application is running. The software tools described here are available for In any platforms; for the irnplementations described a Silicon Graphics Indigo was used. view

3. 1 The

The

A11

the top

edge

of the

tool

spectrum

the

in

view

spectrum

given wavelength values. The primary any

illustrates

the

interface

allows each

create

an

reason

effect

of

the

user

averaged

The

3.2

may

is

two

the

at

Figure

2. The

wavelengths for a single point on the fruit in the intensity data can be seen along

noise

intensities.

2: The

be modilied

spect1~u1n view. what

by

we

call

smashing.

the

srnush

adjust how

to

many

seven

Spatial

3.2.1 The

their

spatial relationships. be displayed. Model

3D SD model

wavelength. lOpenGI_, is 2

l

cl/Tk

a

stands

for

lt is necessary

to introduce

aside but and a

from

the

intensity provides

slider

displays 3D fruit geometry coloured according data a 3D model can Using the spatial dataa3Dmodelcanbecreatedbyplottingthe

Tool

3

Figure

values.

The

combined

to

at an

to select

view:

spectrum

only

one

wave~

opportunity the

to

wavelength

View

view

t1 aclema.rk

for

intensity

Views

trait that sets them following three views have a common spatial view shows multiple points on the fruit surface, between points, length. This enables comparison of intensities

is to

value

value.

each

that

smushcd

adjacent intensity neighbouring intensity values are average

The

observe

The

of that wavelengths intensity and neighbouring average for srnushing is to reduce noise in the intensity data.

applying to

all

intensity

Figure The

into

displays

View

view displays example is shown

surface.

are

view

spectrum

Spectrum spectrum

provided

currently

category.

of Silicon Comxnand

Graphics,

lnc.

Language/Toolkit. 3

to

the intensity

be created

at

a

single

by plotting

the

Figure points in

a

3D model

virtual

into

3: The

3D space,

trackball

The

angle, giving

the

of

Plotting the points This

intensities.

as

4: The

3D

spatial points.

in 3D is not

to find a way to show the spectral sufficient; it is necessary at a single wavelength as colour by displaying the intensities By mapping the largest intensity to white and the smallest to black, these and

be achieved

can

information.

the intermediate colour

seven

for

Figure

data.

smus/L, averaging every

shown in Figure 4. A perspective projection transforms the The user can interact with the scene a virtual display using mouse manipulates the viewing position so the model can be seen from any impression of a three dimensional object.

2D picture

a

effect

scaleis

values constructed

can

be shown

using

as

levels

similar

method

of gray and

on

the

fruit

model.

A green

to

red

provided as a menu option. A slider can be used to select the individual wavelength to display. The points chosen for each fruit represent a. rather sparse sample. ln order for the fruit to be displayed as a skin the intensity information is interpolated between points. The method used to interpolate this colour information is known as Gouraud Gouraud shading shading is a popular technique because it is simple and can be computed using graphics hardware. It is useful

extend

these points represented

a

to

form

a

skin.

skin

for two reasons, model Firstly the shape ofthe secondly the point colour is interpolated across an area, rnaking it easier to see. The Skin is made from polygonal surfaces constructed using extracted from the latitudes and longitudes of each point. adjacency information If the model is shown on a 24-bit Representing intensity with grayscale has a drawback. to only 256 grey values. This means that it is only possible monitor, there is a restriction to distinguish between 256 intensity values in the dataset, and By using the red, green, blue channels independently it is possible to increase this number by a factor of six. Ware is a minor problem when compared with systematic suggested that the lack of colour resolution errors that can arise from interpretive effects ofthe human visual system, such as simultaneous contrast. With this in mind it is inorc important to reduce these perceptual effects than is clearer

to

if it

is

a

And

4

5: The

Figure it

is

to

reduce

of

quantisation

approximation

to

the

visual

the

data.

displayed is made

spectrum

model

-view.

in

accordance

available

as

a

colour

with

this

map

for

philosophy an wavelength

the

data.

The

3.2.2 The at

3D tool any

lt is

time

possible picture. This

Map and is useful the to

for

back

viewing part

unwrap

is shown

Field

Height

the fruit

of the

the model

in the

Views

fruit

in is

view

view.

so

a

virtual

environment

occluded, leaving that

An

the

full

the

surface

but

data

it is restrictive

only partially

of the fruit

is shown

since

visible. in

one

example map view is shown in Figure 6. The data collection method provides a. natural In latitude-longitude coordinate system for the data. principle, this partitioning allows us to use any of the techniques developed by cartographers for mapping the earth. In order to flatten the fruit surface our current tool simply plots as the y~~axisin what is known longitude as the a:»axis and latitude as an equirectangular projection [14]. However all 2D maps of curved objects will contain distortions, and while of the equirectangular plot is high at the {ruit s accuracy equator it decreases rapidly as the poles are approached. The map view reduces visual complexity and allows us to use the 3D graphics hardware for another In the lieightfeld view intensities are purpose. interpreted as altitudes, and the data is displayed as a 3D relief map. This artificial terrain helps to clarify the relative distance between intensity values, which is difficult to determine when only colour information is used. The height field can be viewed from trackball. any The angle by manipulating a virtual of the wavelength slider. heights also respond interactively to movement lt is important to incorporate same the surface stationary reference for this view; otherwise as if it is appears floating in space. The stationary reference is created by vertical lines below the height lield in Figiire 7. Also a new slider is introduced for the height held view. It controls the height of the maximum value that is in the dataset, acting as a scaling constant for the height of the height field. This is provided so the user has the flexibility to exaggerate the disparity between similar intensity values that are displayed. it is important to note that applying heights to the intensity values in the model view would not be benericial. The 3D object is too complex to act as an adequate reference for the to distinguish between bumps that changing amplitudes. It would be diilicult fruit represent map

5

Figure

Figure

6: The

7: The

map

view.

height field

6

view

and

geometry

View

4

that

bumps

represent

data.

intensity

Integration

The

and map views described in the previous section are spectrum combined to create a single a.ppl_ication. These views present different viewing concepts so they occupy separate windows. To support exploration using multiple windows, the parameters from each view are linked. in section 5. lntegratiou of the model, height field and volume views is described Each point shown on the map view can be selected in by clicking or dragging the mouse the display window. The closest is highlighted and the spectrum data point to the mouse is dragged between relating to that point is used to create a. spectrum view. As the mouse points the spectrum display changesinteractively. The cohesion between the views provides a fast and effective have been created. way to view the data using the tools that Moving the wavelength slider in the map view automatically updates the spectrum view so that the same wavelength is shown consistently in both views. It is important to maintain this single environment to avoid confusing the user.

Conclusion

5 The

tools

created

considered work

and provide and

promising

section the

have

been

data

in

a

Future level

great

the

of

will

project

fulfilled.

Work

Four

flexibility continue

interactive

for until

methods enhance

these

A

tools

view

a

aims

visualisation

Each is variety of different ways. of visualising the dataset, and each is designed to

present

the

exploring the

views

successful be very

dataset.

They

presented in this have

been

are

future

created

to

presenting different flexible. Proposed ways to at

follow.

is

proposed called the volume view. It would represent the existing data as a superimposed one in front of another, like a deck of cards. Each 2Dlmap would correspond to one sampled. wavelength. The stack of maps would effectively create a third It would still only be possible to see one at a time, but wavelength dimension. map the volume created by the stack would give a stronger sense of the current position in the NIR, Spectrum data. [l2, 171. [Integration of the map and spectrum views greatly simplified navigation within the dataset. The views will continue to be integrated until all views are collected into the one application. The 2D map view, the height field view and the volume view all present the same information but in slightly modified format. These will be combined to exist in the same window frame. A menu will be provided to switch between the views. The model and spectrum views each present different windows of their own. To support viewing concepts so they will occupy simultaneous exploration using multiple views, the parameters from all views will be linked. stack

new

of 2D

The

maps,

view

would

be

useful

if it

was modified to act as a toolbox for manipexisting smush slider could be combined with a slider and bounding lines. The bounding lines could select the range of intensities the user is interested in investigating. This would provide a tool for controlling the levels of quantisation in the display. The bounding values would update the colour values used in the spatial displays interactively. The wavelength slider could be presented in ’the spectrum view so that all the tools for manipulating the views are in the same location. It is currently possible to click in the map view to select a point for displaying in the lf the views were spectrum view. integrated it would be useful to be able to select points in each of the spatial displays, and have each of the displays update interactively. This would require rotating the object in the model view for the highlighted point to appear at the front . The fruit model used in the model view uses a sparse a sample of points to reconstruct fruit shape. It would be possible to use Magnetic Resonance Image (MRI) data taken from a A more accurate single fruit sample to represent an ideal fruit shape reconstruction could be created using this information, providing a display model that looks realistic. lt. is possible to measure a fruit at different times in its life cycle. By collecting this time could be added to the visualisation dependent data, a fifth dimension problem. lt would be interesting to investigate this data to gain some of ripening insight into the evolution

spectrum

ulating the wavelength

characteristics

contents

in the

of the

other

more

views.

The

fruit.

7

The

equirectangular

is desirable. tion

projection

distorts

of different

map

map

Implementation

the fruit

geometry to produce

will

projections

greater degree than

a

with

maps

less distor-

[14].

When to colour values it mapping wavelength intensities being compared. If comparison is made between fruit then the therefore be computed using the maximum and and should the ability to distinguish different However, this may inhibit a particular wants to examine fruit s intensities, if the user

the

on

the user s

minimum

and

maximum

on

immediate

values

for

that

the interface

goal,

fruit

should

alone.

to consider is important what is colour map should be consistent,

minimum the

on

from

all

fruits.

particular

a

colour

should

map the choice of

As

fruit;

be based

strategy depends approaches.

both

supports

values

intensities

References The

[1] http://wwwcivm.mc.duke.edu/default.html. [2]

T.

A.

M.

DeFanti,

D.

Brown,

and

Center

for In Vivo

B. H. McCormick.

Microscopy.

Visualisation,

ln Gregory M. Nielson, Bruce opportunities. engineering J. Rosenblum, editors, Visualisation in Scientific Computing, Lawrence IEEE Computer Society Press, 1990.

Y3_J.

D.

A.

Foley,

and Practice.

ny

Graphics Gems, chapter 8, and

Illumination

Color

in.

M.

J.

Murakami,

Imagery. Springer-Verlag,

Nondestructive

(NIR)

1990.

determination

transrnittance.

February

of sugar 1993.

in scientific

llimoto, and K. Itoh. September 1993.

near

Analysis of apple quality by

1988.

content

Computer graphics: Special issue on visualisation 1987. Report 21(6), ACM SIGGRAPH, November

J.

M.

Himoto,

reflectance

near-infrared

10’ J. Neider, T. Davis, and _

Generated

Press,

com-

infrared

re-

spectroscopy.

9_ M. Murakami,

,111 J. K

Academic

462-463.

pages

Computer

S. Kuwano, T. Fujiwara, and M. Iwamoto. mandarin in satsuma using near infrared

flectance

32»~47.

pages

Dam, S. K. Feiner, and J. F. Hughes. Computer Graphics, Principles 2nd edition, 1990. VV(-zsley,

van

7_ B. H. McCormick. puting. Technical

8_

and

Shriver,

Addison

,4_ A. S. Glassner. R.. Hall.

scien-

expanding

research

and

tific

Ousterhout.

Tcl

_l2_‘ L. J. R.ose1|blum.

Natsuga,

spectroscopy. M. Woo.

and

June

ltoh.

Addison

Visualisation

of

J. P. Curtis.

A fast

and

Technical

report,

The

of

Analysis

OpenGL Programming

the T/c Ybolkil.

and

K.

pumpkin

quality by

1992.

Guide.

Wesley,

Addison

VV(-rsley,1993.

1994.

experimental data at the naval research laboratory. ln J. Rosenblum, editors, Visuali.salio1z Gregory M. Nielson, Bruce Shriver, and Lawrence in Scientific Computing, pages 245---253. IEEE Computer Society Press, 1990.

‘].3_

S. Smith

G.

in three

New

and

dimensions.

Zealand

P. Snyder.

\:15_C.

Ware.

Flattening

Color

sequences

Computer Graphics B. Welch.

_17_ G. Wyvill,

l’llUl,ll0(.l

Horticulture

and

of

measuring

Food

Research

tree

structure

Institute

of

Ltd, 1995.

[144 J.

116

effective

Practical 1996.

and

the Earth.

for univariate

Applications,

programming Informal

of

University maps:

Chicago Press,

Tlieory, experiments,

8(5):41-49, September

in tcl and tk.

Cornmunication.

8

1993.

Draft

and

1988.

Manuscript,

1995.

principles. IEEE