Systems. Identification. Chris. M. Jubien,. Nikitas. J. Dimopoulos. Department of Electrical and Computer. Engineering. University of Victoria,. PO Box 3055.
Recurrent
Neural
Networks
Chris
M.
Department
in
Jubien,
Nikitas
of Electrical
Abstract
--A
that
asymptotically
works
are
training
procedure
weIghts procedure as well Is a gradient as the relaxation method tlvatlon
functions
obtained bllity
used
responses
en. Such a boat
so as the
was
error
between
Victoria.
of neural The
to Identify
the
for
the training
used
on collected
a class
Is presented.
A method
throughout
a network
based
3055.
net-
expected
procedure the
rudder/heading
dynamic
Dimopoulos
Computer
BC.
training
Engineering.
V8W
3P6
neural
CANADA
network.
neuromirne monotomcally
and
antees
sta-
itive
entries
Is also
glv-
easy
way
behavior
of
data.
and
that
0)
side
the
neural
It must
main
class
shown
of
so-called
positive
and
n W ~t
contaIn
diagonal
a neural
network
the
essentially condiuon
IS that
of the
whether
connection
to
are ~
behavior
on one to check
the
belongs
which
".on-decr~sl.ng.
asymptouc
stance.
f(
functions
that
assuring
Identification
of Victoria,
which constantsadapts and the the Interconnection. slopes of the ac-
Is miniMized.
Is maintained
for
stable
Box
J.
and
University PO
Systems
all
[I].
network
guar-
of Its pos-
This
gives
an
For
in-
is stable.
in Figure
I is stable
provided
wei g
hts
in
submatrices
Wand
Ware 23
I. This area
paper
of
is a summary
identification
works.
The
Iy
models
neural are
network
One neural
with
is the
single
Fortunately.
known
to
works
be
to this
to
neural
provide
in
the net-
a way that
selected
of
can
the
area
ture
of
ing
with
of
since
identification
a controller
most
of
In
(I).
0
=
-TO+
the
[I]
which
is that their
real of
networks This done
neural
.sented IS the state
a controller
or
exists
class on
of
which neural
W
-:
~W
I
the
W
12
is
per-
asymptotic described
stability by
the
is ensured
differential
Fig. 1. Sample
for equa-
(I)
divided
into
k classes.
is the onal
network matrix
Network
PARAMETER
This and
section
other
here
discusses
parameters
described
of
ADJUSTMENT NE1WORKS
IN STABLE
a method
adjusting
the
that
stable
neural
in Section
is to define
some
2. The
a criterion
for
networks general and
then
classical back propagation [5]. However. ral networks described in section 2. have
J
(2) neural
are
approach adjust
NEURAL weights
that
in
the
is used
the parameters
since certain
the stable restrictions
neuon
the polarity of the connection of classes. a straightforward dient adjustment is not possible. A solution for this is also
grapre-
here.
network.
j
...W
A.
Gradient
of Cost
The general class of neural
1k
6
=
-TO
Function
equation networks
+ Wf(O)
for calculating of interest here + b
is
the
behavior
of
the (4)
...Wkk
connectivity of
Neural
in a direction that will decrease this cost. In this sense the technique is similar to linear recursive adaptive methods [4] and to
and
(3) Wn
start-
net-
identification
-: Wkl
By
is ensured.
networks.
...°l!J
of
be stable.
stability
in fea-
stasta-
BACKGROUND
that are
are N neurons
02
it must
important
they
l 01
(I).
most
34
systems.
+b
[ 0 I 0 2 ...o
-r -L
by
The
useful
as a nonlinear
in particular.
property
work
dynamic
Wf(O)
there
is that
as defined
34
is extremely
~
sense =
control.
be
Uon 6
result
that
setting
with
neural stable.
and
of
dealing
important
a class
shown
networks
and
or model
a model
III. It has been
This
a trained
properties
as complex
fully;
When
II.
~eural
inhibitory).
as a potential-
implement.
or identification
to analyze
here.
class
using
systems.
a sysfem
asymptotically
is used
tains
using
be assured.
model.
done
systems
are
these easy
in a control
not
bility
is to
work
(i.e..
controllers.
complex
can
work nonlinear
modeling
too
bility
recent
networks
and
non-positive
systems
this
complex
is fast
problem
often
of
in real
network
of
Neural
way
desirable
are
of
some
nonlinear
purpose
in controllers.
effective
of
of
main
establishing used
INTRODUC11ON
neural
matrix.
relaxation
T
=
constants.
diag
('t.)
I. b is the
is the
diag-
using the same notation criterion for measuring
input
to the
functIon
introduced in section 2. One possible the performance is the quadratic cost
J ( e)
=
1/2 =
1/2
where A is us~
( 0 -0 TA
.e
d) TA ( 0 -0
e
parameters B.
.(~)
0 d 15 the desIred
state of the neural
to eli;lnina~
the C?st ~y
from
not crucial. A 15 a diagonal output neurons and zero's adaptive adjusted
d)
network.
neurons
whose
Matnx s~te
is
methods [2,4], parameters e in the neural network along the negative gradient of this coSt, i.e.,
are
de aJ --ai = -11 ae
(6)
The chain rule for differentiation is used to allow for the calculation of this gradient for parameters associated with neuron j: aJ
