Associative Memory (BAM) network. Their approach, however, does not deal with inference using multiple patterns. Studies have been done on pattern and.
An Inference Method Using Multiple Patterns and Modi cation of Pattern Space Ichiro Takeuchi, Takeshi Furuhashi Nagoya University, Furo-cho, Chikusa-ku, Nagoya, 464-01, Japan This paper deals with inference using multiple patterns. Inference among vague information, e.g. patterns, has caused \explosion of vagueness". This paper proposes a new mechanism to suppress the increase of vagueness. This paper also proposes a modi cation method for pattern space based on a new concept of vagueness in patterns. Simulations are done to demonstrate the potentiality of the proposed inference method and the pattern space modi cation method.
Abstract.
1
Introduction
Patterns have been symbolized for recognition. The symbolization reduces information in patterns. We need to nd novel symbolizing method to minimize the reduction of important contents. However, it has been very dicult to nd out distinctive features from patterns for the symbolization. Inference using multiple patterns has a high potential for intelligent information processing. This inference method directly deals with patterns without reducing their contents. Since patterns with gray code contain more vagueness, the inference among this type of patterns enlarges vagueness, and sometimes infers nothing. This results in \explosion of vagueness". Fuzzy inference is an interesting approach for handling vagueness. M. Hagiwara 1 proposed a Multi-directional Associative Memory (MAM). MAM realizes association among patterns. However, MAM treats patterns with binary code. These patterns contain vagueness little. S. Yamamoto, T. Yamaguchi and T. Takagi 2 proposed a method for fuzzy knowledge representation using associative memory. Takagi et al. 3 proposed a mechanism called coordinator to avoid explosion of vagueness in Fuzzy Associative Inference System. They introduced inhibitory connections in layers of Bidirectional Associative Memory (BAM) network. Their approach, however, does not deal with inference using multiple patterns. Studies have been done on pattern and symbol processing 4 5 . The authors have also made attempts to show the possibility of new processing by patterns and symbols 6 7 . These studies also do not treat inference using multiple patterns. This paper proposes a new mechanism to suppress the increase of vagueness for the multiple pattern inference. This inference directly uses multiple patterns without reducing the contained information. This paper also proposes a modi cation method for pattern space based on a new concept: vagueness in patterns. The portions of axes in pattern space can be expanded or contracted using the new concept of vagueness. Simulations show the e ectiveness of the proposed system. [ ]
[ ]
[ ]
[ ][ ]
[ ][ ]
2 2.1
Basic Framework Patterns and inference method
Patterns are, for example, input images from CCD camera, input signals from sensors, and in a more abstract sense, sets of data in nonlinear feature space. Patterns contain various vague information. The conventional inference methods symbolize patterns and use the symbols for inference. Fig.1 shows an example of this conventional inference process. Fuzzy inference is categorized into this type of inference. The problem of this inference method is how to select distinctive features from the patterns for the symbolization. However, it is very dicult to select features without missing important contents in patterns. Fig.2 illustrates the inference which directly uses the patterns directly. This inference method has a high potential for handling vague contents. Pattern
Pattern
rn te Pat
ol mb Sy
te Pat
Fig. 1. conventional inference
Pa tte rn
Pa tte rn
Sy mb ol
rn
Symbol
Fig. 2.
Inference
using
multiple
pat-
terns
Then how are the patterns represented and how is the inference with multiple patterns realized? Fig.3 shows an example of pattern in a feature space. This pattern is a distribution which represents a kind of feature of an object. The vertical axis means, for example, frequency of occurrence. As shown in Fig.4, patterns are quantized and represented by the vectors whose elements are the quantized values. This paper studies cases where objects have many attributes represented by patterns. Fig.5 is an example of a concept \apple" consisting of three attributes, color, shape, and taste. Each attribute is represented by a pattern as shown in Fig.6. The basic architecture of the multiple pattern inference is Bidirectional Associative Memory(BAM) 8 . BAM has the architecture shown in Fig.7. It consists of two layers. Pairs of vectors are memorized on the matrix of connection weights. [ ]
Feature
Fig. 3.
Feature
A pattern
Fig. 4.
Quantized pattern
Color
Apple Shape
Fig. 5.
Taste
Concept and attributes
yellow
Fig. 6.
Color
red
Color pattern for apple
When a new vector is inputted into BAM, a memorized vector near to the inputted one is recalled through an association process between the two layers. The detail of BAM is described in the next section. Two attributes represented by patterns can be put into this network. Fig.8 shows multi-directional associative memory architecture. Multiple layers are connected with each other for incorporating multiple attributes. Increase of memory capacity has been the most important topic for BAM. BAM requires that memorized vectors are orthogonalized. But pairs of vectors(pairs of patterns) which have week orthogonality, in other words strong correlations, also deserve to consider. Closely correlated vectors cause \explosion of vagueness".
2.2 "Explosion of vagueness" in multiple pattern inference As mentioned above, patterns have vagueness and the inference from a pattern to another enlarges vagueness. Inference using multiple patterns makes the vagueness and derives nothing. Explosion of vagueness is one of the important problems of fuzzy inference. Symbolization of patterns is a solution for the explosion of vagueness. This means to reduce the vagueness completely by taking risks
Associative Memory Matrix
Fig. 7.
BAM
Fig. 8.
MAM for multi attributes
of missing important contents. Another approach is to introduce inhibitory connections to suppress vagueness 3 . This approach in [3], however does not treat inference using multiple patterns. This paper presents a mechanism to suppress vagueness in the process of interactions between patterns. This mechanism is well described in section3. [ ]
2.3 Modi cation of pattern space The coordinates in multi-dimensional pattern space are often nonlinear. It is dicult to infer an appropriate set of patterns in the nonlinear space. This paper proposes a way to modify the pattern space. The portions of axes in pattern space are expanded or contracted utilizing a new concept of vagueness. The details of this modi cation of pattern space are given in section 4.
3
Proposed inference method using multiple patterns
3.1 Memorization and Recollection by MAM The network structure discussed in this section is the MAM in Fig.8. Patterns are inputed into the layers, and are memorized in the connection weights between the layers. The memorized pattern ca for attribute a of concept c is U dimensional vector. The elements of this vector are normalized so that the sum of all the elements is 1.0. The matrix a1 a2 and a2 a1 between attributes a1 and a2 are given by C 1 T T (1) a2 a1 = a1 a2 a1 a2 = C ca1 ca2 c=1 where C is the number of concepts, T means transpose of vector/matrix. Input vector to an attribute is denoted by a . When inputs are given to the network, inference with the initial internal state vector a (0) starts. The U dimensional vector a (0) is expressed as (2) a (0) = a :
M
M
x
mp
M X mp mp
M
i
x
i
x
M
xa(t + 1) is given by
xa(t + 1) = xothera(t) + xa(t)
(3)
where is the coecient for putting weights on the in uence from other attributes, other a (t) means the information from other attributes given by the following eq.
x
A X
(4) M a axa (t) where A is the number of attributes. xother a (t) and xa (t +1) are also normalized.
xothera(t) =
a =1 a 6=a 0
0
0
0
In this paper, the recollection degree for concept c is de ned as follows:
rec(t) =
A X U X a=1 u=1
min fmpcau ; xau (t)g
(5)
mp
where mpcau and xau are u-th elements of the memorized pattern vector ca and the internal state vector a (t) respectively, min is the minimum operator. rec(t) is also normalized. The set of rec (t) for all concepts c correspond to the output of this network at t.
x
3.2
Two mechanisms for avoiding explosion of vagueness
This paper presents two mechanisms for avoiding \explosion of vagueness". One is to use reciprocal of variance of patterns ea (t) to suppress the in uence from patterns with larger vagueness. The other is to introduce suppression parameter d(t) which diminishes smaller elements of patterns for suppression of vagueness. Eq.(3) is modi ed by using these values as
xa (t + 1) = 0
A X a=1 a6=a
0
M aa ea(t) fxa(t)gd t + xa (t) ( )
0
(6)
0
where ea (t) is expressed as
2 92 301 8 U X > > > > 66 U > vxau (t) > > > =7 7 < X 1 7 6 v =1 xau (t) 0 ea (t) = 6 7 U > 7 64 U u=1 > X > > > > > > v ;5 :
(7)
v=1
This value is normalized so that the sum of ea (t) for all attributes is 1.0. ea (t) means the reciprocal of variance of a (t). a (t) is the internal state vector for attribute a at t. The variance is considered as the indicator of \vagueness", it follows that ea (t) means the reciprocal of vagueness of internal state a (t). When vagueness is \high", in other word, the variance of a (t) is large, the output from
x x
x
x
this pattern to other layer is made small. d(t) in (6) is to enlarge the di erence between larger values and smaller values of elements of xa (t). d(t) is given by
d(t) = t+
(8)
where and are constants. Since inference through multiple patterns exaggerates \vagueness", d(t) is made larger as multiple inference goes on to suppress the vagueness.
Memorized patterns (mean, standard deviation) attributes color shape taste size touch apple (90, 5) (35, 5) (55, 10) (55, 20) (70, 5) banana (20, 5) (85, 5) (30, 5) (40, 10) (15, 5) concepts pear (70, 10) (65, 15) (35, 15) (85, 5) (85, 5) orange (40, 15) (25, 5) (70, 5) (10, 10) (45, 15) Table 1.
1 "apple" "banana" "pear" "orange"
0.8 0.6 0.4 0.2
Recollection degree
Recollection degree
1
"apple" "banana" "pear" "orange"
0.8 0.6 0.4 0.2 0
0 0
5 10 15 20 25 30 35 40 45 50 Repetiton number of inference
Fig. 9.
Explosion of vagueness
0
5 10 15 20 25 30 35 40 45 50 Repetiton number of inference
Fig. 10.
E ect of ea (t) and d(t)
Fig.9 shows an example of recollection degrees de ned by eq.(5). This gure shows the case by eq.(3) where the two mechanisms are not used. The coecient was set at 10.0. Four concepts, \apple", \banana", \pear", and \orange", having ve attributes, \color", \shape", \taste", \size", and \touch", are memorized in the MAM. The MAM has ve layers for the ve attributes. The patterns are given by normal distributions in this paper. Each layer consists of 100 units, the mean and the standard deviations of patterns representing the attributes are
given in Table.1. In this case, as the inference is repeated, all the recollection degrees came to the same value. This is the \explosion of vagueness". The e ectiveness of the above two mechanisms is shown in Fig.10, where Eq.(6) was used. The coecients were = 10:0, = 0:10, = 2:0. \Apple" was recollected. In this case, the proposed two mechanisms worked well to suppress the explosion of vagueness.
4 Modi cation of pattern space One more concept \lemon", whose attributes are similar to those of \orange", was memorized in the network as shown in table.2. In this case, the recollection degrees became indistinctive as shown in Fig.11. The coecient were = 10:0, = 0:10, = 2:0.
Table 2.
New memorized pattern (mean, standard deviation) attributes color
shape
taste
size
touch
concepts lemon (30, 15) (15, 5) (80, 5) (0, 10) (35, 15)
For distinguishing similar patterns, a modi cation method of pattern space is presented in this section. This method is to expand or contract the portions of axes of pattern space without losing the contents of memorized patterns, in other words, without changing the shapes of the memorized patterns. The total lengths of expanded/contracted axes are always kept constant by the normalization. Because the memorized patterns have normal distributions in this paper, expansion/contraction and normalization processes can be replaced by the changes of standard deviations of patterns. After the modi cation of an axis, the u-th element mpu is given by
mpu = 0
p1
2sd
2 ( )2 3 1 u 0 mean 5 exp 40 sd 2
()
0
(9)
where is the ratio of expansion/contraction. The standard deviation is multiplied by 1 . So the control of enables to change the pattern space by keeping vagueness of patterns. is de ned as a (t) which is a function of the number of repetition of inference t. In this paper a (t) is automatically determined by \vagueness" of patterns. The vagueness of pattern is de ned as information content given by
ica (t) = 0
C X c=1
reca(t)lnreca(t)
(10)
where reca (t) is expressed as
reca(t) =
X U
u=1
min fmpcau ; xau (t)g
(11)
and normalized so that the sum of all concepts is 1.0. From eq.(5), it is clear that reca (t) is the recollection degree of each attribute for each concept. The parameter a (t) in (9) is controlled by the degree of vagueness ica (t) as
a (t) = f (ica (t)):
(12)
f (t) = ica (t)
(13)
In this paper, f is decided as
where and are constants. Fig.12 is the result when was 2.0, was 1.0 ( = 10:0, = 0:10, = 2:0). By comparing with Fig.11, it can be known that lemon and orange were distinguished.
1 "apple" "banana" "pear" "orange" "lemon"
0.8 0.6 0.4 0.2 0
"apple" "banana" "pear" "orange" "lemon"
0.8 0.6 0.4 0.2 0
0
5 10 15 20 25 30 35 40 45 50 Repetition number of inference
Fig. 11.
5
Recollection degree
Recollection degree
1
rec (t)
for 5 concepts
0
5 10 15 20 25 30 35 40 45 50 Repetition number ot inference
Fig. 12.
E ect of a (t)
Conclusion
This paper studied the inference using multiple patterns. Two mechanisms to suppress the explosion of vagueness were proposed. A method for modi cation of pattern space was also proposed. A new concept of vagueness was introduced. The portions of axes in pattern space could be expanded or contracted based on this new degree of vagueness. Simulation results showed that the proposed two mechanisms worked well to suppress the explosion of vagueness and similar patterns could be distinguished by the modi cation of pattern space. In the future, we will study the theoretical aspect of information contents in the symbolization process.
References 1. M. Hagiwara, \Multidirectional Associative Memory", Porc. IEEE Joint Conf. on Neural Networks, Washington, D.C, January 15-19, 1990. New York: IEEE, Vol I, pp. 3-6, (1990) 2. S. Yamamoto, T. Yamaguchi, T. Takagi, \Fuzzy Associative Inference System and Its Features", IEEE Int. Conf. on System Engineering, pp. 155-158, (1992) 3. T. Takagi, A. Imura, H. Ushida and T. Yamaguchi, \Inductive learning of conceptual fuzzy sets", Proc. 2nd Int. Conf. on Fuzzy Logic and Neural Networks IIZUKA-92, pp. 375-380, (1992) 4. T. Omori, N. Yamanashi, \PATON: A Model of Concept Representation and Operation in Brain", Proc. of Int'l Conf. on Neural Network'94, pp. 2227-2232, (1994) 5. Ron. Sun, \Integrating Rules and Connectionism for Robust Commonsense Reasoning", Wiley-interscience, (1994) 6. Y. Hattori, T. Furuhashi, Y. Uchikawa, \A Proposal of Multi-Module Network for Association of Patterns and Symbols", the Journal of Robotics and Mechatronics Vol.8, No.4, pp. 345-350, (1996) 7. T. Furuhashi, Y. Hattori, Y. Uchikawa, \A New computational Framework for Vague Patterns and Symbols", Proc, of 1996 Bienmal Conf. of the North Americal Fuzzy Infrormation Processing Society (NAFIPS'96), pp. 170-174 (1996) 8. B. Kosko, \Adaptive Bidirectional Associative Memories", Applied Optics, Vol.26, No.23, pp. 4947-4960, (1987)
