Proceedings of the 9th International Conference on Neural Information Processing (ICONIP'OZ) , Vol. 2
Lip0 Wang, Jagath C. Rajapakse, Kmihiko Fukushima, Soo-Young Lee, and Xin Yao (Editors)
TRAINING RBF NEURAL NETWORKS ON UNBALANCED DATA Xiuju Fu *, Lip0 Wang* *, Kok Seng Chua *, Feng Chu** *Institute of High Performance Computing Singapore Science Park IT Singapore 117528 Emai1:
[email protected]
** School of Electrical and Electronic Engineering Nanyang Technological University Singapore 639798 Email: elpwang@ ntu.edu. sg
http://www.ntu.edu.sg/home/elpwang ABSTRACT This Paper presents a new algorithmfor the construction and training of an RBF neural network with unbalanced data. In applications, minority classes with much fewer samples are The learning process Of a Often present in data ral network usually is biased towards classes with majority populations. Our focused On improving the cation accuracy of minority classes while maintaining the overall classification performance. 1. INTRODUCTION
In neural network training, if some classes have much fewer samples compared with the other classes, the neural network system may respond wrongly for the minority classes because the overwhelming samples in the majority classes dominate the adjustment procedure in training. Berardi and Zhang [3] proposed a cost-sensitive neural networks, by introducing different costs associated with making errors in different classes. When they calculated the sum of squared errors for the multiplayer perceptron (MLP) neural network, they multiplied each term by a classdependent factor (cost). This idea has a much earlier origin in machine learning, that is, the loss matrix [4] which deals with different risks (costs) associated with making errors in different classes. For example, in classification of medical images, these class-dependent risk factors will need to be selected from practical experiences. By assigning larger costs to minority classes, Berardi and Zhang [3] were able to improve the classificationaccuracies for minority classes. In this work, as in earlier discussions on risk matrices, cost factors are selected in an ad hoc manner. In this work, we
propose a method that determines these cost factors automa6cally such that all classes, i.e., minority and majority classes, are roughly equally important. We demonstrate this method in the of the RBF classifier. For unbalanced data sets, two methods had been prethe samples of minority sented in [ I l l . In the first classes were duplicated to increase their effects on ing neural networks. In the second method, the so-called snowball method proposed in [181 for multi-font character recognition was used to improve the accuracy of the minority class [ 1I]. In the snowball method, neural networks were first trained with the samples in the minority class, which favors the minority populations. Next, samples of majority classes were added gradually while training the neural network dynamically. The two methods were used in the MLP neural network, ART (Adaptive Resonance Theory) neural network and RBF neural network. However, it was found that the two methods mentioned above had no effect on the MLP and RBF neural networks. As an important neural network, the RBF (radial basis function) neural network has been applied widely in various applications, such as function approximation, noisy interpolation, density estimation, optimal classification tasks and data mining tasks [2][10] [14][15][16]. It is desirable to explore a training method of RBF neural networks handling .unbalanceddata. The popularity of the RBF neural network is due to its fast training and its global approximation capability with local responses. Like other neural networks, constructing an RBF neural network with a compact architecture but robust for unbalanced data is a challenging task. Here we propose an algorithm by increasing the contribution of minority samples on the MSE (mean squared error) function. The MSE function is a powerful objective
1016
function in neural network training. When learning a data set with a minority class, the weights of a neural network will be dominated by the majority classes. For example, when using the back propagation method to train weights, the weights are updated according to the variation in the error function of the neural network [ 111. It is clear lhat the weights obtained finally will reflect the nature of the majority classes but not much of the minority class. Thus, lit motivates us to increase the magnitudes of weighted par-
