Recurrent Neural Networks in Systems Identification Chris M ... - lapis

Recurrent

Neural

Networks

Chris

M.

Department

in

Jubien,

Nikitas

of Electrical

Abstract

--A

that

asymptotically

works

are

training

procedure

weIghts procedure as well Is a gradient as the relaxation method tlvatlon

functions

obtained bllity

used

responses

en. Such a boat

so as the

was

error

between

Victoria.

of neural The

to Identify

the

for

the training

used

on collected

a class

Is presented.

A method

throughout

a network

based

3055.

net-

expected

procedure the

rudder/heading

dynamic

Dimopoulos

Computer

BC.

training

Engineering.

V8W

3P6

neural

CANADA

network.

neuromirne monotomcally

and

antees

sta-

itive

entries

Is also

glv-

easy

way

behavior

of

data.

and

that

0)

side

the

neural

It must

main

class

shown

of

so-called

positive

and

n W ~t

contaIn

diagonal

a neural

network

the

essentially condiuon

IS that

of the

whether

connection

to

are ~

behavior

on one to check

the

belongs

which

".on-decr~sl.ng.

asymptouc

stance.

f(

functions

that

assuring

Identification

of Victoria,

which constantsadapts and the the Interconnection. slopes of the ac-

Is miniMized.

Is maintained

for

stable

Box

J.

and

University PO

Systems

all

[I].

network

guar-

of Its pos-

This

gives

an

For

in-

is stable.

in Figure

I is stable

provided

wei g

hts

in

submatrices

Wand

Ware 23

I. This area

paper

of

is a summary

identification

works.

The

Iy

models

neural are

network

One neural

with

is the

single

Fortunately.

known

to

works

be

to this

to

neural

provide

in

the net-

a way that

selected

of

can

the

area

ture

of

ing

with

of

since

identification

a controller

most

of

In

(I).

0

=

-TO+

the

[I]

which

is that their

real of

networks This done

neural

.sented IS the state

a controller

or

exists

class on

of

which neural

W

-:

~W

I

the

W

12

is

per-

asymptotic described

stability by

the

is ensured

differential

Fig. 1. Sample

for equa-

(I)

divided

into

k classes.

is the onal

network matrix

Network

PARAMETER

This and

section

other

here

discusses

parameters

described

of

ADJUSTMENT NE1WORKS

IN STABLE

a method

adjusting

the

that

stable

neural

in Section

is to define

some

2. The

a criterion

for

networks general and

then

classical back propagation [5]. However. ral networks described in section 2. have

J

(2) neural

are

approach adjust

NEURAL weights

that

in

the

is used

the parameters

since certain

the stable restrictions

neuon

the polarity of the connection of classes. a straightforward dient adjustment is not possible. A solution for this is also

grapre-

here.

network.

j

...W

A.

Gradient

of Cost

The general class of neural

1k

6

=

-TO

Function

equation networks

+ Wf(O)

for calculating of interest here + b

is

the

behavior

of

the (4)

...Wkk

connectivity of

Neural

in a direction that will decrease this cost. In this sense the technique is similar to linear recursive adaptive methods [4] and to

and

(3) Wn

start-

net-

identification

-: Wkl

By

is ensured.

networks.

...°l!J

of

be stable.

stability

in fea-

stasta-

BACKGROUND

that are

are N neurons

02

it must

important

they

l 01

(I).

most

34

systems.

+b

[ 0 I 0 2 ...o

-r -L

by

The

useful

as a nonlinear

in particular.

property

work

dynamic

Wf(O)

there

is that

as defined

34

is extremely

~

sense =

control.

be

Uon 6

result

that

setting

with

neural stable.

and

of

dealing

important

a class

shown

networks

and

or model

a model

III. It has been

This

a trained

properties

as complex

fully;

When

II.

~eural

inhibitory).

as a potential-

implement.

or identification

to analyze

here.

class

using

systems.

a sysfem

asymptotically

is used

tains

using

be assured.

model.

done

systems

are

these easy

in a control

not

bility

is to

work

(i.e..

controllers.

complex

can

work nonlinear

modeling

too

bility

recent

networks

and

non-positive

systems

this

complex

is fast

problem

often

of

in real

network

of

Neural

way

desirable

are

of

some

nonlinear

purpose

in controllers.

effective

of

of

main

establishing used

INTRODUC11ON

neural

matrix.

relaxation

T

=

constants.

diag

('t.)

I. b is the

is the

diag-

using the same notation criterion for measuring

input

to the

functIon

introduced in section 2. One possible the performance is the quadratic cost

J ( e)

=

1/2 =

1/2

where A is us~

( 0 -0 TA

.e

d) TA ( 0 -0

e

parameters B.

.(~)

0 d 15 the desIred

state of the neural

to eli;lnina~

the C?st ~y

from

not crucial. A 15 a diagonal output neurons and zero's adaptive adjusted

d)

network.

neurons

whose

Matnx s~te

is

methods [2,4], parameters e in the neural network along the negative gradient of this coSt, i.e.,

are

de aJ --ai = -11 ae

(6)

The chain rule for differentiation is used to allow for the calculation of this gradient for parameters associated with neuron j: aJ

-aJ

the I.

.

the derivative

f

actIvatIon

.

o

this derivative i

neuron

J.

is simply

given

of the cost with

If

In a manner

J

analogous

to traditional

k that have neuronj

this equation (7) may now

adjusting

Here,

the notation

an

of the previous

eq:~-

section,

(8) or (9) as appropriate. Parameters

by

stants

back propagation

'ti (analogous

of the

To calculate which

to represent

Ati'

defines

)

L

wkf(Oj)

{(0)

t

=

~ if I + e-