A Unified Monitoring Framework for Distributed ... - IEEE Xplore

7 downloads 6215 Views 1MB Size Report
spanning multiple enterprise information systems distributed in a heterogeneous IT ... links between the monitoring tool and the components to be monitored.
A Unified Monitoring Framework for Distributed Information System Management Shiping Chen, Surya Nepal and Suraj Pandey Information Engineering Laboratory, CSIRO ICT Centre Crn. Vimiera and Pembroke Roads, Marsfield NSW 2122, Australia {Firstname.Lastname}@csiro.au

Abstract -

With the increasing complexity and scale of business processes, and the underlying information systems, there is a demand for monitoring the complicated business processes spanning multiple enterprise information systems distributed in a heterogeneous IT environment. This paper presents a unified monitoring framework for such distributed information system management. This framework utilizes Web service and messaging queue technologies to collect log data for business process visualization and monitoring and decouples the tight links between the monitoring tool and the components to be monitored. XML is used to define a universal data format for different types of information to be monitored. A prototype tool is implemented and presented to demonstrate the feasibility and usability of the proposed framework, and experiments are conducted to evaluate the performance overhead introduced by the monitoring.

considerable

technologies

[1][2][3][4]

and

distributed

monitoring,

several

open

issues

still

remain.

Monitoring is still difficult and/or expensive for various business and system activities running on heterogeneous IT infrastructures (hardware and software) without a unified interface to these business process and software components. In this paper, we propose a unified monitoring framework for complex distributed information systems. This framework applies messaging queue technology to loosely couple the monitoring system and the components to be monitored. XML is used to define a universal message format for various data to be collected. A prototype proof-of-concept is implemented and presented in this paper to demonstrate the feasibility and usability of the proposed framework. The prototype shows that our framework is able to effectively visualize and efficiently

Keywords Distributed Systems, Business Processes, Distributed System Monitorring, Visualization and Debugging -

I.

Although

specifications [11][12][13][14] have been developed to enable

monitor complex distributed information systems built and running on different technologies and different platforms and extendable

INTRODUCTION

for

emerging

technologies

and

ever-charging

business requirements.

IT environments are becoming increasingly complicated. With the increasing complexity and scale of distributed systems, it is critical and in high demand to monitor the activities, performance

and

resources

usages

in

heterogeneous

distributed information systems [1].

II.

MONITORING DISTRIBUTED INFORMATION SYSTEMS

A. Motiviations Monitoring is an effective approach for understanding and

A typically distributed system consists of a number of components (EJB, COM+, web servers, databases) and

people

managing

information

distributed systems.

systems,

especially

complicated

They can provide values to software

that perform a set of specific business functionalities. The

development across the whole software development life cycle

components are deployed into different sub-systems, or called

(SDLC) in the following aspects:

services, according to their roles and security policies. These services usually run as an independent process on different server machines. While these services and people are able to provide particular functionalities by themselves, they more likely work in collaboration to complete a complex business operation

by

interacting

each

other

synchronously

or

asynchronously [16]. Sometimes, an enterprise system has to contact

external

services

for

some

specific

business

requirements, such as suppliers and banking. This makes it difficult and laborious to validate system design/deployment, understand system behaviors and detect the performance bottleneck/failure-root as a whole system. Furthermore, these internal and external business components and systems can be

Design visualization: It is hard to design a complex distributed information system. It is even harder to prove and present your design to others, especially to people with less technical background. In this case, visualization is usually an effective way to help others to understand your design concepts and principles. For example, a monitoring tool can be used to show how the key business components are deployed in a real distributed system and how they interact with each other to complete prototypes

on

different

operating

systems.

This

adds

business can

be

operation

also

used

dynamically. to

validate

a

Preliminary design

by

incorporating into the tool, making your design feasible and convincing [5].

implemented in completely different technologies and/or even run

a

another

heterogeneous dimension to this problem.

- 259 -

Distributed systems debugging/testing: Debugging and testing

processes require that a monitoring tool should have an

distributed systems is an extremely challenging task. The

agile infrastructure and universal data structure to adapt

challenge

the

is

resulted

technology aspects.

from

both

technology

and

non­

Since a lot of activities can occur

concurrently in different components, which may be built on different technologies and be geographically distributed in different

places,

all

these

make

it

laborious

and



and

exceptions

in

an

turning

and

quality

assurance

(QA):

A

monitoring tool can also be used to check the availability of distributed

business

components,

find

out

performance

at

design

or

to

build

distributed systems,

they

introduce

result from operating systems (Unix and Windows), (TCP/IP,

HTTP,

SOAP). All of these make unified

visualization and monitoring extremely difficult if not

integrated environment by using technologies [6][7].

Performance

available

middleware (J2EE and .NET) and networking protocols

distributed activities, validate business logics and analyze the activities

not

Heterogeneities choices

environment without any helps from tools. A good monitoring

these

are

heterogeneities into the systems. The heterogeneities can

tool can help developers to collect information about these between

which

As modern technologies provide us with a number of

time­

consuming to know what is exactly happening in such an

causality

changes,

deployment times.

impossible. Therefore, a layer of abstraction is required to hide the heterogeneities from different technologies. •

Incompatibility

bottleneck for performance tuning, and detect various failures

Although a number of specifications exist for system

on real time in a distributed environment. By mean of expert

monitoring and management, such as [11][12][13], there

knowledge bases and models, a monitoring tool can also help

are

to ensure Quality of Services (QoS) of a distributed system by

specifications making them difficult to directly use them.

considerable

overlaps

between

Additionally,

developers. Furthermore, most existing tools have 'black­ products

researchers and vendors have invested considerable efforts in development for

of

monitoring

various

processor

approaches

and

performance

[4],

network

utilization

[11].

A

number

of

to

[1]

or

technologies

[2]

resulting

in

being

incompatible between each other. •

Cost-effectiveness Due to the fact that there is no complete solution to

memory hierarchies [5], parallel computing applications [7], and

confusion

box' infrastructures/data format and support only specific

Having recognized the advantages of system monitoring tools, and

considerable

these

automatically response, diagnose and even predicate system

technologies

adds

gaps

failovers to occur [1][2].

research

it

and

addressing

computer

the

above

[11][12][13][14].

To

support

efforts

efficient

technologies (J2EE and .NET) provide their own management

III.

infrastructures and instrumentation, JMX for J2EE [13] and WMI for .NET [4], based on these specifications. Several tools making best use of these infrastructures and instrumentations [1][2].

While the above technologies and tools enable us to before, we are restricted to monitor an arbitrary component in an arbitrary distributed system at reasonable efforts and costs

monitoring

that

many

AN UNIFIED MONITORRTING FRAMEWORK

A. Overall software architecture In paper, we propose a unified monitoring framework to

explore the possibility of monitoring various heterogeneous

architecture of our proposed framework. The monitoring framework consists of three layers, which we describe as follows: •

due to:

Measurement

Layer

represents

all

business

components/nodes to be monitored, which can be built in

different technologies and running on different operating

Complex and ever-changing business processes The modern technologies, such as EAI and Web Services, enable the applications to work more closely than before. This capacity brings a distributed application close to a set of complex business processes, which requires multiple distributed components (services) to work coordinately [16]. Usually an integrated control/monitoring is no available

in

such

a

complex

distributed

environment; thus making it difficult to monitor and manage such complicated business processes as a whole system.

system

architecture and interface. Figure 1 illustrates the general

monitor and understand computer systems better than ever

longer

on

components in a distributed system with an open monitoring

B. Challenges



costs

different

customers are reluctant to pay.

distributed monitoring, the two major distributed middleware

have been developed to enable distributed monitoring by

and

monitoring

to

components in their systems. This may result in higher

as

for

have

standardize the definitions of the information for specific such

tools

customers

buy/develop

areas,

multiple

issues,

information specifications were developed in an attempt to

Additionally,

the

ever-changing

business

- 260 -

systems. Sensor agents are deployed on each node to measure the application server processes' internal states as well as the external running environment resources, by making the best use of the native system monitoring infrastructure and instrumentations, such as JMX and WMI. We can also embed the application sensors into applications to monitor the component activities and interactions at business level if required. All measured data are formatted in a standard XML message and sent to the above messaging layer asynchronously.

:;;

>. '" ..J '" c:

'c, '" II) II) ., �

Monitoring Data Respository

Figure I. A unified monitoring framework for distributed information systems •

Messaging Layer

the

relatively independent and autonomous systems, which

monitored data as a XML message from measurement

makes it easy to deploy and configure each of the systems

layer to monitoring layer. The messaging layer can be

separately; (b) minimizing the effect of monitoring on the

implemented

performance

with

is

responsible

off-the-shelf

for

Web

delivering

services

and



monitored

components;

(c)

naturally

IBM MQSeries, which support multiple programming

analyze the causality between these events that may occur

languages and operating systems. It can also be deployed

in a complex distributed environment; (d) allowing the

using multiple messaging systems and across multiple

monitored systems and the monitoring working in different

networking protocols according to the requirements for

pace. For example, when the monitored components are

performance and system management.

running

Monitoring Layer

on

visualization,

provides a monitoring console to

the to

fly,

we

achieve

can the

perform

best

step-by-step

trade-off

between

flexibility and real-time monitoring.

display real-time monitoring information from different

viewpoints and optional monitored data repository to store



the interested data for further analysis and/or playing­

A unified and Extendable intet/ace The interface between monitoring tools and monitored

back. The monitoring console and the data repository

components are a unified XML message. This makes it

should be technology-independent. This means that they

possible to define a universal data format to incorporate

can be implemented in Java or C/C++/C# running on

multiple information specifications. The universal data

different operating systems as long as they can access the

format is also extendable to meet specific monitoring

messaging queues in some ways.

requirements at different levels.

B. Features of our monitoringframework



Comparing to existing monitoring approaches/tools, our framework has the following features: •

of

classitying and ordering events, which makes it easy to

messaging queue technologies/products, such as JMS and

Queue-based data collections Our monitoring framework uses queues to provide the data collection infrastructure (as the messaging layer). This design has the following advantages: (a) decoupling the monitoring tool and the monitored components as two

- 261 -

An open solution Since three layers in our monitoring framework are technology/vendor-independent, our framework provides an open solution to monitoring distributed systems. This means: (a) each layer can be implemented in different technologies provided by different vendors; (b) the monitoring consoles can consume the data measured from different sensors developed by the third-party and vice verse; (c) multiple information standards and specifications can be incorporated into our framework for a complete monitoring solution.

IV.

PROTOTYPING



A. The monitored information system

Event description All normal activities and exception with their status data are modeled as an

RuralOnline is a distributed e-procurement application as a

event is described as a message in XML. Table II is an

real-life trading activities for Australian rural brokers. The

example event for an invoking activity.

core business of a rural broker is to buy commodities from farmers and sell them back agricultural products, such as

TABLE II.

feedstock, machinery, fertilizer and seeds. The prototyped



buying and selling commodities and equipments between its



members and integrates them with Web services provided by

to call Database DB1

business partners such as banks and machinery suppliers.



RuralOnline consists of two



components - Trading and Banking. The trading component



deployed in .NET accepts various requests from the Web, including registration,

querying,

buying and selling.



For



buying and selling requests, the trading component needs to



access the Web services provided by the banking components, deployed in J2EE Web Services, via SOAP. The architecture of

XML MESSAGES FOR EVENTS



application provides typical e-business functionalities such as

The current implementation of

event, representing something (what) (space) at a particular time. The

occurs in somewhere

test-bed for our distributed system research, which mimics the

RuralOnline is shown in Figure 1.



Messaging Layer We use a Web service (ASP.NET) as a universal interface to all monitored components and a message queue

Trading

BlInks

SOAP

(MSMQ) is also used as its backend component for storing and organizing the monitored event messages in

VI Q)

.�

XML as shown in Table II. It can also be implemented



using [15] when the corresponding specification and

• •



middleware technology become mature.

� �==:::: :1::; Suppile�s



Monitoring Console The monitoring console is implemented in MFC. Firstly, the console loads a configuration file of the monitored

Figure I

-

RuralOnline system architecture

systems as listed in Table I so that the console can display the system architecture as shown in Figure 2. Then the

B. A Monitoring/Visualization Tool

monitoring console can monitor the information for

As a part of our test-bed, we prototyped a monitoring!

activities, performance and runtime resources on each

visualization tool based on the unified monitoring framework

component in the system by reading the messages from

proposed in this paper. The prototype system was built on Windows

platform.

The

key

technologies

used

in

the queue. The messages are used for different monitoring

our

views according to their types. This monitoring console

implementation are described below: •

supports multiple viewpoints and paces (on-the-tly, step­

System description

by-step, etc) for monitoring.

A set of XML schemas is defined to describe the

V.

configuration of the system to be monitored, including the components in the systems and the details for each component.

For

example,

a

simplified

configuration file is listed as shown in Table I. T ABLE I.

XML FOR SYSTEM CONFIGURATION

Trading AppServer

system

DEMONSTRATION AND PERFORMANCE ANALYSIS

A. Demonstration: monitoring and visualization To visualize business-level interesting events are to occur. responsible

of

dynamically

The monitoring code is

composing

xml-based

event

messages as shown in Table I and sending the messages to the messaging queue via the Web service gateway when the events occur. The events are classified into the following catalogues

trading.gif

applications:

50

1)

lOO

and

can

be

further

extended

for

specific

Invoking other components - before/after calling DCOM, EJB, Web Services, etc.



logics occurred in each

component, we have to instrument monitoring code at where

2)

Performance/resource measurement - such as throughput, latency, CPU utility etc.



- 262 -

CSIRO

I

Dlstnbuted Systems MOnltonng and Visualisation Tool

FII.

Edit

View

';1i;{.J.@ffift!ifflf¥ffftJ"Iffl1j;.I,j,i.,;,I·"iMi#Nffl

J

Help

Action

I

Fie

E..

View

Action

Help

CI

CSIRO

Rrr.:l'On'ine

.&

R�alOnline

lTr�ding

!

:·B