IGOR AIZENBERG AND CLAUDIO MORAGA, “THE GENETIC CODE AS A FUNCTION OF MULTIPLE-VALUED LOGIC OVER THE FIELD OF COMPLEX NUMBERS AND ITS LEARNING USING MULTILAYER NEURAL NETWORK BASED ON MULTI-VALUED NEURONS”, JOURNAL OF MULTIPLE-VALUED LOGIC AND SOFT COMPUTING, NO 4-6, NOVEMBER 2007, PP. 605-618.
The Genetic Code as a Function of MultipleValued Logic Over the Field of Complex Numbers and its Learning using Multilayer Neural Network Based on Multi-Valued Neurons* Igor Aizenberg1 and Claudio Moraga2 1-Texas A&M University-Texarkana, Department of Computer and Information Sciences
2600 N. Robison Rd., Texarkana 75505 TX USA 2- University of Dortmund, Department of Computer Science-1 (Germany) and European Centre for Soft Computing (ECSC)
33600 Mieres, Asturias, Spain E-mail for correspondence:
[email protected] Received
Abstract. It is shown in this paper that a model of multiplevalued logic over the field of complex numbers is the most appropriate for the representation of the genetic code as a multiple-valued function. The genetic code is considered as a partially defined multiple-valued function of three variables. The genetic code is the four-letter nucleic acid code, and it is translated into a 20-letter amino acid code from proteins (each of 20 amino acids is coded by the triplet of four nucleic acids). Thus, it is possible to consider the genetic code as a partially defined multiple-valued function of a 20-valued logic. Consideration of the genetic code within the proposed mathematical model makes it possible to learn the code using a multilayer neural network based on multi-valued neurons (MLMVN). MLMVN is a neural network with traditional feedforward architecture, but with a highly efficient derivativefree learning algorithm and higher functionality than the one of the traditional feedforward neural networks and a variety of kernel-based networks. It is shown that the genetic code multiple-valued function can be easily trained by a significantly
* Based on "The Genetic Code as a Multiple-Valued Function and its Implementation Using Multilayer Neural Network based on Multi-Valued Neurons ", by Igor Aizenberg and Claudio Moraga which appeared in The Proceedings of 37th International Symposium on MultipleValued Logic (ISMVL-2007), © 2007 IEEE
IGOR AIZENBERG AND CLAUDIO MORAGA, JOURNAL OF MULTIPLE-VALUED LOGIC AND SOFT COMPUTING, NO 4-6, NOVEMBER 2007, PP. 605-618
smaller MLMVN in comparison with a classical feedforward neural network. 1. INTRODUCTION It is well-known that genome sequences do contain information about protein structure. This sequence is a sequence of twenty amino acids in various combinations. Each of twenty amino acids is represented in the sequence by a codon (triplet) of three nucleic acids. Since there are exactly four different nucleic acids, there exist 43=64 different combinations of them “by three”. However, some of these combinations code the same amino acids. So there are exactly 20 amino acids that form a genome. There is also the 21st amino acid, which marks the end of the genomic subsequence. But it does not participate in the genome. Thus, the genetic code can be considered as the mapping between the four-letter alphabet of the nucleic acids (DNA) and the 20-letter alphabet of the amino acids (proteins) [1]. In other words, this mapping can be considered as a discrete function of three variables: G j = f ( x1 , x2 , x3 ) , (1) where
G j , j = 1, ..., 20
is
the
amino
acid
and
xi ∈ { A, G , C , T } , i = 1, 2, 3 (A, G, C, T are the four nucleic acids Adenine,
Guanine, Cytosine and Thymine, respectively). Since this function can take exactly 20 different discrete values and depends on three variables that can take exactly 4 different discrete values, it would be natural to consider it as a partially defined multiple-valued function of a 20-valued logic. In this paper, we will consider how this function can be learned by a neural network. This problem is not new. It has been discovered in [1],[2]. However, in those earlier publications a standard feedforward neural network [3] based on the neurons with sigmoid activation function [4],[5] has been used. A minimal standard feedforward network that could learn the genetic code is 3Æ12Æ2Æ20 (an input layer with three inputs, two hidden layers containing 12 and 2 neurons, respectively, and one output layer containing 20 neurons) [2]. We will consider here a different solution that can, on the one hand, significantly reduce the number of neurons in a network and, on the other hand, will be based on the representation of the genetic code as a multiplevalued function. We will learn a genetic code by a multilayer neural network based on multi-valued neurons (MLMVN). This network has been introduced in [6] and then investigated and developed further in [7]. The MLMVN network consists of multi-valued neurons (MVN). This neuron has been introduced in [8]. It implements those mappings that are
THE GENETIC CODE AS A FUNCTION OF MULTIPLE-VALUED LOGIC OVER THE FIELD OF COMPLEX NUMBERS AND ITS LEARNING USING MULTILAYER NEURAL NETWORK BASED ON MULTI-VALUED NEURONS
described by the multiple-valued threshold functions over the field of complex numbers [9]. The basic ideas of multiple-valued threshold logic over the field of complex numbers have been formulated in 1971 by N. Aizenberg et al. [9] and then developed in [10], [11]. The crucial idea behind multiple-valued threshold logic over the field of complex numbers is that the values of k-valued logic are kth roots of unity. MVN is a neuron with complex-valued weights and an activation function, defined as a function of the argument of a weighted sum. This activation function was proposed in [9]. A comprehensive observation of the discrete-valued MVN, its properties and learning is presented in [11]. A continuous-valued MVN and its learning are considered in [6], [7]. The most important properties of MVN are: the complex-valued weights, inputs and output coded by the kth roots of unity (a discrete-valued MVN) or lying on the unit circle (a continuous-valued MVN), and the activation function, which maps the complex plane into the unit circle. It is important that MVN learning is reduced to the movement along the unit circle. The MVN learning algorithm is based on a simple linear error correction rule and it does not require differentiability of the activation function. The most impressive and important application of MVN is its use as a basic neuron of the MLMVN [6], [7]. MLMVN outperforms a classical multilayer feedforward network and different kernel-based networks in the terms of learning speed, network complexity, and classification/prediction rate tested for such popular benchmarks problems as the parity n, the two spirals, the sonar, and the Mackey-Glass time series prediction [6], [7]. These properties of MLMVN show that it is more flexible and adapts faster in comparison with other solutions. In this paper, we apply MLMVN to learn the genetic code, which is considered as a multiple-valued function. The basic ideas behind multiple-valued threshold logic over the field of complex numbers, the MVN, the MLMVN and its backpropagation learning algorithm are described in Section 2. The representation of the genetic code as a multiple-valued function over the field of complex numbers is considered in Section 3. Finally, the simulation aspects and results are presented in Section 4. 2. MULTILAYER NEURAL NETWORK BASED ON MULTIVALUED NEURONS Multiple-Valued Threshold Logic over the Field of Complex Numbers and Multi-Valued Neuron Multiple-valued threshold logic over the field of complex numbers is based on the following idea proposed in [9]. Let M be an arbitrary additive
IGOR AIZENBERG AND CLAUDIO MORAGA, JOURNAL OF MULTIPLE-VALUED LOGIC AND SOFT COMPUTING, NO 4-6, NOVEMBER 2007, PP. 605-618
group
and
its
cardinality
is
not
lower
Ak = {a0 , a1 ,..., ak −1} , Ak ∈ M be a structural alphabet. Definition
[9]-[11].
1
Let
us
than
call
f ( x1 ,..., xn ) | f : A → Ak of n variables (where A
a
n k
n k
k.
Let
function is the nth
Cartesian power of Ak ) a function of k-valued logic over group M.
C as a group M k −1 roots of unity Ε k = {ε , ε , ε , ..., ε } , where
Let us take the field of complex of complex numbers th
and a set of k
0
2
ε = exp(i 2π /k ) (i is an imaginary unity) is a primitive kth root of unity (see FIGURE 1) as a set
Ak . Hence, any function of n variables
f ( x1 ,..., xn ) | f : Ε → Ε k is a function of k-valued logic over the field n k
of complex numbers according to Definition 1. Let K = {0,1,..., k − 1} be a set of values of a regular k-valued logic. Evidently, it is very easy to build one-to-one correspondence between set K and a set of kth roots of unity
Ε k = {ε 0 , ε , ε 2 ,..., ε k −1} . Thus, the regular k-valued logic becomes the
one over the field of complex numbers and any multiple-valued function of k-valued logic becomes a multiple-valued function over the field of complex numbers [11]. Evidently, the function values and its arguments in this case are the kth roots of unity: ε j = exp(i 2π j/k ) , j = 0 ,..., k - 1 , where i is an imaginary unity. Let us consider the following function (or k-valued predicate) proposed in [9]: P( z) = exp(i2π j/k ), if 2π j/k ≤ arg z < 2π ( j +1) /k , (2) where arg z is the argument of the complex number z. Function (2) divides a complex plane onto k equal sectors and maps the whole complex plane into a subset of points belonging to the unit circle (see FIGURE 1). This is a set of kth roots of unity. Definition 2 [9]-[11]. A k-valued function f ( x1 , ..., xn ) over the field of complex numbers is called a k-valued threshold function over the field of complex numbers if the following condition holds for all x1 , ..., xn from the domain of this function:
f ( x1 , ..., xn ) = P ( w0 + w1 x1 + ... + wn xn ) , where W = ( w0 ,w1 , ...,wn ) is a weighting vector, P is function (2).
(3)
THE GENETIC CODE AS A FUNCTION OF MULTIPLE-VALUED LOGIC OVER THE FIELD OF COMPLEX NUMBERS AND ITS LEARNING USING MULTILAYER NEURAL NETWORK BASED ON MULTI-VALUED NEURONS
i
ε2
1
ε 0 k-1 k-1
j-1
ε
z
k-2
j
j+1
FIGURE 1 Geometrical interpretation of k-valued logic over the field of complex numbers and the MVN activation function
i
q
ε
q
s
ε
s
FIGURE 2. Geometrical interpretation of the MVN learning rule
MVN has been introduced in [8] as a neural element with activation function (2) which implements those mappings described by multiplevalued threshold functions over the field of complex numbers. MVN has been deeply considered in [11], where its theory, basic properties, and learning are comprehensively observed. A single discrete-valued MVN performs a mapping between n inputs and a single output according to (3). This mapping is described by a multiple-valued (k-valued) function of n variables f ( x1 , ..., xn ) with n+1 complex-valued weights as parameters. Evidently, the MVN’s inputs and output are kth roots of unity. The MVN learning is reduced to the movement along the unit circle. This movement is derivative-free. Since the learning process is reduced to movement along the unit circle, any direction along the circle always leads
IGOR AIZENBERG AND CLAUDIO MORAGA, JOURNAL OF MULTIPLE-VALUED LOGIC AND SOFT COMPUTING, NO 4-6, NOVEMBER 2007, PP. 605-618
to the target. The shortest way of this movement is completely determined by an error that is a difference between the desired and actual outputs. Let ε q be a desired output of the neuron (see FIGURE 2) and ε s = P ( z ) be an actual output of the neuron. The most efficient MVN learning algorithm is based on the error correction learning rule [11]: C Wr+1 = Wr + r (ε q - ε s ) X , (4) (n+1) where X is an input vector, n is a number of neuron’s inputs, X is a vector with the components complex conjugated to the components of vector X, r is the number of iteration, Wr is a current weighting vector, Wr +1 is a weighting vector after correction, Cr is a learning rate. The convergence of the learning process based on the rule (4) is proven in [11]. The rule (4) ensures such a correction of the weights that the weighted sum is moving from the sector s to the sector q (see FIGURE 2). The direction of this movement is completely determined by the error δ = ε q − ε s . The correction of the weights according to (4) changes the weighted sum exactly on the value δ. Indeed, let
z = w0 + w1 x1 + ... + wn xn
be a current weighted sum. Let us correct
the weights according to the rule (4) (we take C=1):
~ =w + w 0 0
δ ~ = w + δ x ; ... ; w ~ =w + δ x . ; w 1 1 1 n n n (n + 1) (n + 1) (n + 1)
The weighted sum after the correction is obtained as follows:
z% = w% 0 + w% 1 x1 + ... + w% n xn = = ( w0 + = w0 +
δ (n + 1)
δ
) + ( w1 +
+ w1 x1 +
δ (n + 1)
δ
x1 ) x1 + ... + ( wn +
+ ... + wn xn +
( n + 1) ( n + 1) = w0 + w1 x1 + ... + wn xn + δ = z + δ .
δ ( n + 1)
δ (n + 1)
xn ) x n =
=
The latter considerations show the importance of the factor 1/(n+1) in the learning rule (4). This factor shares the error δ uniformly among the neuron's weights. The activation function (2) is discrete. It has been recently proposed in [6], [7], to modify the function (2) in order to generalize it for the continuous case in the following way. If k → ∞ in (2) then the angle value of the sector (see FIGURE 1) tends to zero. Hence, the function (2) is transformed in this case as follows:
THE GENETIC CODE AS A FUNCTION OF MULTIPLE-VALUED LOGIC OVER THE FIELD OF COMPLEX NUMBERS AND ITS LEARNING USING MULTILAYER NEURAL NETWORK BASED ON MULTI-VALUED NEURONS
P( z ) = exp(i (arg z )) = eiArg z =
z , |z|
(5)
where Arg z is a main value of the argument of the complex number z and |z| is its modulo. The function (5) maps the complex plane into a whole unit circle, while the function (2) maps a complex plane just into a discrete subset of the points belonging to the unit circle. Thus, the activation function (5) determines a continuous-valued MVN. The learning rule (4) is modified for the continuous-valued case in the following way [6], [7]: Wr+1 = Wr + = Wr +
Cr (ε q - eiArg z ) X = ( n +1)
Cr ⎛ q z ⎞ ⎜ε ⎟ X. ( n +1) ⎝ | z |⎠
It is worth to consider the following modification of (4) and (6): Cr % Wr+1 = Wr + δ X, (n+1)
(6)
(7)
z z =T − using a normalization by where δ% is obtained from δ = ε q − |z| |z| the factor 1 | z | :
δ% =
1 1 ⎛ z ⎞ δ= ⎜T − ⎟, |z| | z |⎝ | z |⎠
(8)
Learning according to the rule (7)-(8) makes it possible to squeeze a space for the possible values of the weighted sum. Thus this space can be reduced to the respectively narrow ring, which includes the unit circle inside, by using (7), (8) instead of (6). This approach can be useful in order to exclude a situation when small changes either in the weights or the inputs lead to a significant change of z. On the other hand, 1/ z in (8) and in (7), respectively, can be treated as a variable part of the learning rate. MVN has at least three very important advantages in comparison with other neurons: 1) its functionality is higher than the one for other neurons; 2) its training is simpler; 3) it implements those mappings that are be described by multiple-valued threshold functions. MVN-based Multilayer Feedforward Neural Network
A traditional multilayer feedforward neural network (MLF, it is also often referred to as a "multilayer perceptron" - MLP) and the backpropagation learning algorithm for it are well known. A multilayer
IGOR AIZENBERG AND CLAUDIO MORAGA, JOURNAL OF MULTIPLE-VALUED LOGIC AND SOFT COMPUTING, NO 4-6, NOVEMBER 2007, PP. 605-618
architecture of the network with a feedforward dataflow through nodes that requires full connection between consecutive layers and an idea of a backpropagation learning algorithm was proposed in [3] by D. E. Rumelhart and J. L. McClelland. It is well known fact that MLF is based traditionally on the neurons with a sigmoid activation function [4], [5]. MLF learning is based on the algorithm of error backpropagation. The error is being sequentially distributed form the "rightmost" layers to the "leftmost" ones. A crucial point of the backpropagation is that the error of each neuron of the network is proportional to the derivative of the activation function [3]-[5]. A multilayer feedforward neural network based on multi-valued neurons (MLMVN) has been recently proposed in [6], [7]. This network has at least two principal advantages in comparison with an MLF: derivative-free learning and higher functionality, i.e. an MLMVN with the smaller number of neurons outperforms an MLF with the larger number of neurons [6], [7]. As it is mentioned above for the single neuron, the differentiability of the MVN activation function is not required for its learning. Since the MVN learning is reduced to the movement along the unit circle, the correction of weights is completely determined by the neuron's error and the learning process is derivative-free. The same property holds not only for a single MVN, but for any MVN-based network. The MLMVN is a multilayer neural network with standard feedforward architecture, where the outputs of neurons from the preceding layer are connected with the corresponding inputs of neurons from the following layer. The network contains one input layer, m-1 hidden layers and one output layer. Let us use here the following notations. Let Tkm be a desired output of the sth neuron from the mth (output) layer; Ysm be an actual output of the kth neuron from the mth (output) layer. Then the global error of the network taken from the kth neuron of the mth (output) layer is calculated as follows: * δ sm = Tsm − Ysm . (9) The MLMVN learning algorithm is derived from the same considerations as the learning algorithms (4), (6) and (7) for the single MVN that have been considered above. The learning algorithm has to minimize (down to zero, if it is necessary) the global error of the network expressed in terms of the mean square error (MSE). The square error functional for the rth pattern X r = ( x1r , ..., xnr ) is as follows: * 2 Er = ∑ (δ sm ) (W ) , (10) s
where δ
* sm
is a global error of the sth neuron of the mth (output) layer, Er is
THE GENETIC CODE AS A FUNCTION OF MULTIPLE-VALUED LOGIC OVER THE FIELD OF COMPLEX NUMBERS AND ITS LEARNING USING MULTILAYER NEURAL NETWORK BASED ON MULTI-VALUED NEURONS
a square error of the network for the rth pattern, and W denotes all the weighting vectors of all the neurons of the network. It is fundamental that the error depends not only on the weights of the neurons from the output layer but on all neurons of the network. The mean square error functional for the network is defined as follows:
Ε=
1 N
N
∑E r =1
r
,
(11)
where E is a mean square error of the whole network and N is a total number of patterns in the training set. * The backpropagation of the global errors δ sm through the network is used (from the mth (output) layer to the m-1st one, from the m-1st one to the m-2nd one, …, from the 2nd one to the 1st one) in order to express the error of * each neuron δ sj , j = 1,..., m by means of the global errors δ sm of the entire network. Following the backpropagation learning algorithm for the MLMVN proposed in [6], [7], the errors of all the neurons from MLMVN are determined by the global errors of the network (9). The MLMVN learning is based on the minimization of the error functional (11). Let us use the following notations. Let wikj be the weight corresponding to the ith input of the sjth neuron (sth neuron of the jth level), Y ij be the actual output of the ith neuron from the jth layer (j=1,…,m), and N j be the number of the neurons in the jth layer. It means that the neurons from the j+1st layer have exactly N j inputs. Let x1 ,..., xn be the network inputs. The backpropagation learning algorithm for MLMVN was presented in [7]. The global errors of the entire network are determined by (9). We have to * distinguish the global error of the network δ sm from the local errors δ sm of the particular output layer neurons. It is important to remember that the global error of the network consists not only of the output neurons errors, but of the local errors of the output neurons and hidden neurons. It means that in order to obtain the local errors for all neurons, the global error must be shared among these neurons. This is similar to sharing of the error among the inputs of the single neuron using a factor 1/(n+1) in (4), (6) and (7) (see above). Hence, the local errors are represented in the following way. The errors of the mth (output) layer neurons are: 1 δ sm = δ s*m , (12) tm where sm specifies the sth neuron of the mth layer; tm = Nm −1 + 1 , i.e. the
IGOR AIZENBERG AND CLAUDIO MORAGA, JOURNAL OF MULTIPLE-VALUED LOGIC AND SOFT COMPUTING, NO 4-6, NOVEMBER 2007, PP. 605-618
number of all neurons in the previous layer (layer m-1 which the error is backpropagated to) incremented by 1. The errors of the hidden layers neurons are computed as follows:
δ sj =
1 tj
N j +1
∑δ i =1
ij +1
( wsij +1 ) −1 ,
(13)
where sj specifies the sth neuron of the jth layer (j=1,…,m-1); t j = N j −1 + 1, j = 2,..., m, t1 = 1 is the number of all neurons on the layer j-1 (the previous layer j which error is backpropagated to) incremented by 1. Thus, the equations (12), (13) determine the error backpropagation for MLMVN. The weights for all neurons of the network are corrected after calculation of the errors. In order to do this, we can use either the learning rule (4) or (6) depending on the discrete- (2) or continuous-valued (5) model. Hence, the following correction rules are used for the weights following [6], [7]: Csm w% isj = wism + δ smY%im −1 , i = 1,..., n, ( N m + 1) (14) Csm δ sm , w% 0sm = w0sm + ( N m + 1) th for the neurons from the m (output) layer (kth neuron of the mth layer), w% isj = wisj + w% 0sj = w0sj +
Csj ( N j + 1) | zsj | Csj ( N j + 1) | zsj |
δ sjY%i j −1 , i = 1,..., n, (15)
δ sj ,
for the neurons from the 2nd till m-1st layer (kth neuron of the jth layer (j=2, …, m-1), and Cs1 w% is1 = wis1 + δ s1 xi , i = 1,..., n, (n + 1) | zs1 | (16) Cs1 s1 s1 w% 0 = w0 + δ s1 , (n + 1) | zs1 | for the neurons of the 1st hidden layer, where Csj is a constant part of the learning rate. The factor 1/ z sj is a variable and self-adaptive part of the learning rate that was introduced in the modified learning rule (7), (8). It is used in (15) and (16) that determine the learning process for the hidden neurons. However, it is absent in (14) that determine the learning process for the output neurons because for the output neurons the errors calculated according to (9) are known, while for all the hidden neurons the errors are
THE GENETIC CODE AS A FUNCTION OF MULTIPLE-VALUED LOGIC OVER THE FIELD OF COMPLEX NUMBERS AND ITS LEARNING USING MULTILAYER NEURAL NETWORK BASED ON MULTI-VALUED NEURONS
obtained according to the heuristic rule. It should be mentioned that it is possible to set Csj = 1 in (15), (16), and then learning rate contains only a variable part. The learning rate in this form is used in simulations in Section 4. In general, the learning process should continue until the following condition is satisfied: 1 N 1 N * E = ∑∑ (δ smr ) 2 (W ) = ∑ Er ≤ λ , (17) N r =1 k N r =1 where λ determines the precision of learning. In particular, in the case * when λ = 0 the equation (17) is transformed to ∀r , ∀s δ sm r = 0. 3. LEARNING OF THE GENETIC CODE USING MLMVN The Genetic Code as a Multiple-Valued Function Over the Field of Complex Numbers
As it was mentioned above, the genetic code can be considered as the mapping between the four-letter alphabet of the nucleic acids (DNA) and the 20-letter alphabet of the amino acids (proteins). In other words, the genetic code can be considered as a partially defined multiple-valued function of 20-valued logic, which is defined by (1). Now we have to build a one-to-one correspondence among the 20 amino acids and the 20th roots of unity and among the 4 nucleic acids and the 4th roots of unity, respectively. It should be mentioned that a set of the 4th roots of unity is a subset of a set of the 20th roots of unity. As it is well known [12] there are two complementary pairs among 4 nucleic acids (A-T and CG, which means that in two spirals DNA A always bonds only with T and C always bonds only with G). So it would be natural to locate A, T and C, G in the unit circle in such a way that the complementary nucleic acids will be quite the contrary to each other. Without loss of generality, let us locate them starting from the real 1 with a step π / 2 as follows: A, G, T, C (see FIGURE 3). So A, T and, respectively, G, C are exactly quite the contrary to each other. Table 1 contains all 20 amino acids and their codons. It would be natural and reasonable to distribute the 20 amino acids along the unit circle in such a way that each value of function (1) will be placed as close as it is possible to the corresponding values of the function arguments
x1 , x2 , x3
IGOR AIZENBERG AND CLAUDIO MORAGA, JOURNAL OF MULTIPLE-VALUED LOGIC AND SOFT COMPUTING, NO 4-6, NOVEMBER 2007, PP. 605-618
W
C
G G
R
E
M T
I F L V S
A
P T C
D N K A Y Q
H
FIGURE 3 Correspondence among the amino acids and the 20th roots of unity and among the nuclei acids and 4th roots of unity (four nucleic acids A, G, T and C are shown in bold).
For example, if f ( x1 , x2 , x3 ) = K , so this is amino acid Lysine, then f ( A, A, A) = K (see the second row from the bottom in Table 1). Since the nucleic acid A is located at the “real 1” then it is natural to locate the amino acid K at the same point (see FIGURE 3). The same approach leads us to the one-to-one correspondence among the 20 amino acids and a set of the 20th roots of unity that is shown in FIGURE 3. On the other hand, it is possible to distribute the amino acids among the 20th roots of unity randomly (but preserving the distribution of the nucleic acids). However, it is shown by the simulation results (see below) that the first approach leads to much better results. MLMVN Structure
As it was mentioned above, a minimal standard feedforward network that could learn the genetic code is 3Æ12Æ2Æ20 (one input later with three inputs, two hidden layers containing 12 and 2 neurons, respectively, and one output layer containing 20 neurons) [2]. We would like to use the simplest possible MLMVN to learn the genetic code. First of all, we will use a single output neuron instead of the twenty ones. This will be the MVN with the discrete activation function (2), k=20, where the amino acids are distributed either as it is shown in FIGURE 3 or randomly. Instead of two hidden layers in the standard network, we will use a single hidden layer with a minimal possible amount of neurons, for which the convergence of the learning
THE GENETIC CODE AS A FUNCTION OF MULTIPLE-VALUED LOGIC OVER THE FIELD OF COMPLEX NUMBERS AND ITS LEARNING USING MULTILAYER NEURAL NETWORK BASED ON MULTI-VALUED NEURONS
process can be reached in a reasonable time. Thus, a structure of the MLMVN will be 3ÆnÆ1, where n is the number of neurons in a single hidden layer, 3 is the number of inputs. Hidden neurons will have three discrete four-valued inputs and continuous-valued output, while the output neuron will have n continuous-valued inputs and discrete twenty-valued output. Of course, the genetic code must be trained completely, without errors. Thus, λ = 0 in (17).
1
Table 1. Amino acids, their single-letter codes and their Corresponding DNA Codons1. Amino Acid Code DNA codons Isoleucine I ATT, ATC, ATA Leucine L CTT, CTC, CTA, CTG, TTA, TTG Valine V GTT, GTC, GTA, GTG Phenylalanine F TTT, TTC Methionine M ATG Cysteine C TGT, TGC Alanine A GCT, GCC, GCA, GCG Glycine G GGT, GGC, GGA, GGG Proline P CCT, CCC, CCA, CCG Threonine T ACT, ACC, ACA, ACG Serine S TCT, TCC, TCA, TCG, AGT, AGC Tyrosine Y TAT, TAC Tryptophan W TGG Glutamine Q CAA, CAG Asparagine N AAT, AAC Histidine H CAT, CAC Glutamic acid E GAA, GAG Aspartic acid D GAT, GAC Lysine K AAA, AAG Arginine R CGT, CGC, CGA, CGG, AGA, AGG
- The codons TAA, TAG and TGA code the 21st amino acid that does not participate in formation of the genetic code. So we do not need to take into account this amino acid and the corresponding codons.
4. SIMULATIONS
The experimental results obtained with the Pentium IV 3.8 GHz CPU using a software simulator of the MLMVN developed in Delphi 5.0 are summarized in Table 2.
IGOR AIZENBERG AND CLAUDIO MORAGA, JOURNAL OF MULTIPLE-VALUED LOGIC AND SOFT COMPUTING, NO 4-6, NOVEMBER 2007, PP. 605-618
The best results are obtained with the distribution of the amino acids shown in FIGURE 3. The genetic code can be easily learned using the MLMVN 3Æ4Æ1 (4 neurons in a single hidden layer and a single output neuron). The networks 3Æ5Æ1 and 3Æ6Æ1 can be trained a bit faster, but anyway training of all these networks takes just a few seconds. If the amino acids are distributed randomly, training requires more epochs, more time and the bigger network, respectively. The minimal MLMVN, which can be trained in the reasonable time for the randomly distributed amino acids is 3Æ7Æ1. Anyway, each of these MLMVNs is significantly smaller than a traditional feedforward network 3Æ12Æ2Æ20 that was used to learn the genetic code in [1], [2] and considered as the best solution developed so far. It is interesting that if the nucleic acids A, G, T, C would be distributed without taking into account the complementary pairs A-T and C-G then much larger network than those used in the paper will be needed to learn the genetic code. This distinguishes importance of consideration the genetic code as a multiple-valued function definitely over the field of complex numbers as it is done in our work. Table 2. Simulation Results Distribution of MLMVN amino acids (like structure in FIGURE 3 or random) FIGURE 3 3Æ4Æ1 FIGURE 3 3Æ5Æ1 FIGURE 3 3Æ6Æ1 Random 3Æ7Æ1
Number of training epochs (median of 5 independent runs) 9427 5262 1366 61802
5. CONCLUSIONS
In this paper, we proposed a new view on the genetic code. It is considered as a partially defined multiple-valued function (a function in 20valued logic defined on the three 4-valued variables). It has been shown that the multiple-valued logic over the field of complex numbers is a very good mathematical model for representation of the genetic code. A problem of learning the genetic code using a neural network has been considered. It was shown that the multilayer neural network based on the multi-valued neurons (MLMVN) is the best choice for solving this problem.
THE GENETIC CODE AS A FUNCTION OF MULTIPLE-VALUED LOGIC OVER THE FIELD OF COMPLEX NUMBERS AND ITS LEARNING USING MULTILAYER NEURAL NETWORK BASED ON MULTI-VALUED NEURONS
REFERENCES [1] B. Rost, “Neural Networks Predict protein Structure: Hype or Hit”, In Artificial Intelligence and Heuristic Methods in Bioinformatics, (P. Frasconi and RonShamir – Eds.), IOS Press, Amsterdam, 2003, pp. 34-50. [2] N. Tolstrup, J. Tofgard, J. Engelbrecht and S. Burnak, “Neural Network Model of the Genetic Code is Strongly Correlated to the GES Scale of Amino Acid Transfer Free Energies”, Journal of Molecular Biology, vol. 243, 1994, pp. 816-820. [3] D. E. Rumelhart and J. L. McClelland, Parallel distributed processing: explorations in the microstructure of cognition. MIT Press, Cambridge, 1986. [4] M.M. Gupta, L. Jin, and N. Homma "Static and Dynamic Neural Networks", John Wiley & Sons, Hoboken/New Jersey, 2003. [5] S. Haykin Neural Networks: A Comprehensive Foundation (2nd Edn.), Prentice Hall, 1998. [6] I. Aizenberg, C. Moraga, and D. Paliy, "A Feedforward Neural Network based on MultiValued Neurons", In Computational Intelligence, Theory and Applications. Advances in Soft Computing, XIV, (B. Reusch - Ed.), Springer, Berlin, Heidelberg, New York, 2005, pp. 599612. [7] I. Aizenberg and C. Moraga "Multilayer Feedforward Neural Network Based on MultiValued Neurons (MLMVN) and a Backpropagation Learning Algorithm" Soft Computing, Vol. 11, No 2, February, 2007, pp. 169-183.. [8] N. N. Aizenberg and I. N. Aizenberg, "CNN Based on Multi-Valued Neuron as a Model of Associative Memory for Gray-Scale Images", Proceedings of the Second IEEE Int. Workshop on Cellular Neural Networks and their Applications, Technical University Munich, Germany October 14-16, 1992, pp.36-41. [9] N. N. Aizenberg, Yu. L. Ivaskiv and D. A. Pospelov, "About one generalization of the threshold function" Doklady Akademii Nauk SSSR (The Reports of the Academy of Sciences of the USSR), vol. 196, No 6, 1971, pp. 1287-1290 (in Russian). [10] N. N. Aizenberg and Yu. L. Ivaskiv, Multiple-Valued Threshold Logic, Naukova Dumka Publisher House, Kiev, 1977 (in Russian). [11] I. Aizenberg, N. Aizenberg and J. Vandewalle, Multi-valued and universal binary neurons: theory, learning, applications, Kluwer Academic Publishers, Boston/Dordrecht/London, 2000. [12] M. Singer and P. Berg, Genes & Genomes, University Science Books, Mill Valley, California, 1991.
