The development of a weighted evolving fuzzy ... - Semantic Scholar

Expert Systems with Applications Expert Systems with Applications 32 (2007) 86–96 www.elsevier.com/locate/eswa

The development of a weighted evolving fuzzy neural network for PCB sales forecasting Pei-Chann Chang *, Yen-Wen Wang, Chen-Hao Liu Department of Industrial Engineering and Management, Yuan-Ze University, 135 Yuan-Dong Rd., Taoyuan 32026, Taiwan, ROC

Abstract This research develops a weighted evolving fuzzy neural network for PCB sales forecasting and it includes four major steps: first of all, collecting 15 factors among macroeconomic data, downstream production demand and total industrial production outputs and then using the Grey Relation Analysis (GRA) to select a combination of key factors which have the greatest influence on PCB sales. Secondly, taking the time serial factor into consideration, the Winter’s Exponential Smoothing method is applied to predict the tendency of PCB sales and the influences of seasonal effects. Thirdly, applying historical data to proceed the training of Weighted Evolving Fuzzy Neural Network (WEFuNN) and then forecasts the future PCB sales by the WEFuNN. Finally, compare three kinds of performance measurements for each model. The experimental results reveal that the MAPE for WEFuNN model is 2.11% which is the best compared to others. In summary, the WEFuNN model can be applied practically as a sales forecasting tool in the PCB industry. 2005 Elsevier Ltd. All rights reserved. Keywords: Sales forecasting; Weighted evolving fuzzy neural network; Grey relation analysis; Printed circuit board sales

1. Introduction Printed circuit board industry brings a significant economic contribution to Taiwan every year. In order to increase the profits, manufacturers constantly expand production capacity; however, the rapid changes of market demands lead to disequilibrium between supply and demand. Therefore, inventory stacking and material lacking problems are still prevalent. Accordingly, through an accurate sales forecasting model, it can offer a tendency of future demand which would further facilitate the preparation of material flows, product scheduling and so on. Consequently, the cost of inventory and the loss of shutdown will be reduced. Printed Circuit Board (PCB) undertakes a task of connecting various kinds of electronic components. Therefore, the growth of sales in the electronic industry has a close relationship with the PCB industry. The electronic industry *

Corresponding author. Tel.: +886 3 4352654; fax: +886 3 4559378. E-mail address: [email protected] (P.-C. Chang).

0957-4174/$ - see front matter 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2005.11.021

has the power to drive the expansion of PCB industry. Due to the cost pressure, the replacement of production location and the slow transfer of production technique, the value of PCB is getting less and less and thus causes a negative effect on the profits of PCB industry. As the result of profit decrease, many small-size PCB manufacturers either move their production plants to Mainland China or close their business. On the other hand, large-size manufacturers also confront with these challenges. In order to cope with the severe competition environment in this market, manufacturers need to rest on some reliable information for decision making and therefore ‘‘sales forecasting’’ is viewed as a key factor. Reasonable and reliable prediction could offer the quantity of order demand in advance so as to facilitate the preparation of materials flows, capacity planning and then reduce the cycle time. This is regarded as the most efficient tool for manufacturers to control their business resources and to increase their competition and then to strengthen their survival ability in this highly competitive and changeable environment.

P.-C. Chang et al. / Expert Systems with Applications 32 (2007) 86–96

The field of Artificial Intelligence (AI), such as Neural Network (NN), Fuzzy Theory, Expert System (ES), Genetic Algorithms (GA), and Rule Induction and so on has been rapidly developed in recent years. Each algorithm has their own strengths but there are still some limitations of them. In order to reduce these limitations, the hybrid algorithm was developed from combining two or three different A.I. approaches and then draws on the strength of each to offset the weakness of the others. Thus, the main ideal of this research is to develop a better sales forecasting model by integrating neural network and fuzzy theory for PCB industry. This research focuses on the sales forecasting of printed circuit board, moreover, with the purpose of improving the forecasting accuracy. This study modifies the Evolving Fuzzy Neural Network Framework (EFuNN framework) proposed by Kasabov (1998) and adopts a weighted factor to calculate the value of similarity of different rules. 2. Literature review Soft computing algorithm which combined fuzzy theory with neural network has found a variety of applications in various fields ranging from industrial environment control system, process parameters, semi-conductor machine capacity forecasting, business environment forecasting, financial analysis, stock index fluctuation forecasting, consumer loan, medical diagnosis and electricity demand forecasting. The study by Lin and Lee (1991) is the earliest study to combine the fuzzy theory with neural network. They proposed a hybrid model which combines the idea of fuzzy logic controller, neural network structure and learning abilities into an integrated neural-network-based fuzzy logic control and decision system. Subsequently, several researchers also investigate some related studies with regard to the application of this combined approach and then continually developed several kinds of approaches (Arifovic & Ramazan, 2001; Chang & Hsieh, 2003; Chang & Lai, 2005; Chang, Wang & Liu, 2005; Chang, Wang & Tsai, 2005; Elalfi, Haque, & Elalami, 2004; Kim & Han, 2000; Kuo & Chen, 2004; Liang, 1999; Liu, Liu, Wang, & Niu, 2004 Montana & Davis, 1989; Srinivasan, 1998; Yao, 1999; Sexton & Gupta, 2000). The commonly used methods are as follows: 1. Fuzzy Adaptive Learning Control Systems (FALCON) (Zhung, 1999). 2. Fuzzy Back-Propagation Network (FBPN). 3. Adaptive Neuro-Fuzzy Inference Systems (ANFIS). 4. Fuzzy Hyper Rectangular Composite Neural Networks (FHRCNNs). 5. Fuzzy Neural Network (FuNN). Some of the previous studies concerning the application of fuzzy neural network in the forecast aspect will be presented as follows.

87

Kasabov, Kim, and Watts (1997) modified fuzzy neural network and then proposed the method of FuNN/2. Under the framework of FuNN, the author employed the function of Genetic Algorithms which has the ability to search quickly in large spaces and then offset the insufficiency of neural network’s parameters setting. The EFuNN combined unsupervised learning and supervised learning was proposed by Kasabov (1998). The first stage: in order to achieve the objective of connecting fuzzy rules, Kasabov employed unsupervised learning to adjust the connection weights of fuzzy input and fuzzy rule. The second stage: the supervised learning of BPN was used to adjust the connection weights of fuzzy rule and fuzzy output so as to achieve the goal of inner rule of incoming vectors and outgoing vectors. Abraham and Baikunth (2001) proposed a method of Fuzzy Neural Network that employed hybrid supervised and unsupervised learning to forecast short-term electricity demand in the State of Victoria. The authors regarded ‘‘the maximum and minimum temperatures of the day’’, ‘‘previous day’s electricity demand’’, ‘‘season’’ and ‘‘the day of week’’ as input factors and used EFuNN to develop a forecasting model of electricity demand. The EFuNN forecasting technique is also compared with Conjugate Gradient Algorithm (CGA), Back-propagation Network (BPN) and Box-Jenkins. Abraham, Baikunth, and Mahanti (2001) Abraham, Steinberg, and Philip (2001) utilized Fuzzy Neural to forecast the long-term rainfall in Kerala state. The rainfall was affected by global warming, season, storms and butterfly effect and so on. Under the BPN framework, the authors made explicit values to be fuzzy and then proceeded with fuzzy rule aggregation, clustering and extraction. Moreover, employed a process of training via BPN algorithms and adjusted the weights of concealment to output through each time of training. Abraham, Baikunth, et al. (2001) and Abraham, Steinberg, et al. (2001) applied Fuzzy Neural Network to analyze the strategic decision-making of stock market. Firstly, the researchers chose six stock indices from the listing companies in the Nasdaq Stock Market and used conjugate gradient algorithm to forecast these stocks. Kasabov (2001) investigated the short-term forecasting of the Gross Domestic Product (GDP) for totally 15 countries which are EU countries, USA and so on. The related factors considered in this study are Consumer Product Index (CPI), Interest Rate (IR), Unemployment Rate and GDP per capita. The results of this study revealed that the MSE of the 15 countries performed well after fuzzy rule re-aggregation, rule adaptation and rule extraction. It also reveled that the results of EFuNN forecasting are highly accurate in various kinds of environments. Soft computing forecasting tool such as fuzzy neural network can solve the problems with the better degree of accuracy and with the smaller computational times. In the past, the clustering of fuzzy rules in fuzzy neural network usually has pre-designated number of clusters. Few

88


of the previous study employed the method of adaptive cluster movement; hence, there are still lots of opportunities to explore in this aspect. Accordingly, apply this proposed method to PCB industry will be a worthy study. This study is mainly divided into four major steps: first of all, collecting 15 factors among the macroeconomic data, the downstream production demand and the total industrial production outputs and then using the Grey Relation Analysis (GRA) to select a combination of key factors which have the greatest influence on PCB sales. Secondly, taking the time serial factor into consideration, the Winter’s Exponential Smoothing method is applied to predict the tendency of PCB sales and the influences of seasonal effects. Thirdly, applying historical data to proceed the training of Weighted Evolving Fuzzy Neural Network (WEFuNN) and then forecasts the future PCB sales by the WEFuNN. Finally, compare three kinds of performance measurements for each model. The most commonly used methods in the existing literature will be compared. They are EFuNN, Traditional Back-Propagation Network (BPN), Multiple Regression Analysis (MRA) and Genetic Algorithms Neural Network (GANN) (Chang, Wang & Tsai, 2005).

monthly demand data. Among them, the first 48 items of real demand data are used for training and the rest of them are used for testing the forecasting model. In addition, there are totally 15 items of related production index selected from the above-mentioned four categories. These data are collected from ‘‘Yearbook of Industrial Production Statistics’’ (1999–2002), department of statistics, Ministry of Economic Affairs; ‘‘Statistical yearbook of the Republic of China’’ (1999–2003) and ‘‘Indices of Consumer Price’’ (1999–2003), from Directorate General of Budget Accounting and Statistics Executive Yuan, ROC. 3.2. Evaluating indices A good forecasting method needs to take the following information has to be taken into consideration in a good forecasting model. They are: (1) cost, (2) degree of accurate, (3) available data, (4) time periods, (5) the nature of product and service and (6) reflecting the impact and reducing irregular variable factors (Chang, 2002). Hence, take the degree of accuracy and cost have been taken into account in the evaluating indices of this study. The forecasting methods of evaluating indices are presented as follows:

3. Problem definition and description 1. MAPE (Mean Absolute Percentage Error) 3.1. Factors collecting Generally, there are lots of factors affecting the demand of PCB. They are the global business cycle, the structure of PCB downstream demand, macroeconomic environment and variations in industrial production pattern. It is also affected by some uncertain factors. Hence, the PCB related factors are divided into four categories in this study: time serial domain, macroeconomic domain, downstream production demand domain and industrial production domain. According to Chang, Wang and Tsai (2005), there are several sub-factors under these domains and the details of these factors are shown as follows: • Time Serial domain: Winter’s Exponential Smoothing. • Macroeconomic domain: Gross National Product (GNP), Unemployment Rate, Consumer Price Index (CPI), Index of Import Trade and Index of Export Trade. • Downstream production demand domain: Monitor, Notebook PC (NB), Desktop PC (DT) and Motherboard (MB). • Industrial production domain: Manufacturing Production Index, PCB sales value, Semi-conductor Production Index, Manufacturing Sales Index and Manufacturing Production value Index. This study mainly forecasts the monthly actual demand of PCB and the data are from a PCB company in Taoyuan, Taiwan from 1999/1 to 2003/12 totally 60 real

MAPE ¼

n 1X jF t At j n t¼1 At

ð1Þ

where Ft is the expected value for period t, At is the actual value for period t, n is the total number of periods. The smaller the value of MAPE, the better the forecasting ability will be. Generally, the Signification of MPAE is presented in Table 1. 2. MAD (Mean Absolute Deviation) MAD ¼

n 1X jF t At j n t¼1

ð2Þ

3. RMSE (Squared Root of Mean Squared Error) sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Pn 2 t¼1 ðF t At Þ RMSE ¼ n1

ð3Þ

MAPE is used in this study to evaluate the performance of various kinds of forecasting models. A better forecasting model can be found with smaller MAPE value.

Table 1 The signification of MAPE MAPE

Signification

50%

Excellent forecasting ability Good forecasting ability Reasonable forecasting ability Bad forecasting ability


4. Development of WEFuNN The WEFuNN will be used to forecast the demand of PCB product and it is a five-layer network (as shown in Fig. 1) where nodes and connections are created/connected as data examples are presented. 4.1. Feed-forward learning phase For each training case, there are four characteristics to be processed by using the following steps: Step 1: Data Fuzzification Triangular membership function is used to transfer the input characteristics to the fuzzy membership ~x (Fig. 2 and Eq. (4)) function l

OUTPUTS Output layer W 4n

Fuzzy output layer output

W 3 m,n A1i +1

Rule base layer input

W 2 j,k,m A1i

Fuzzy input layer

89

8 0; x > > > < xa ; a 6 x < b ba ~x ¼ cx l > ; b6x cb > > : 0; xPc

In this research, the dynamic weight (W1j) for character j between the 1st layer and the 2nd layer is added to the process. It is expected that better forecasting result can be derived because the triangular membership function will become a non-isosceles one. For the first data, MFinput j;k ð1Þ is its input fuzzy membership function for the jth characters and the kth region. MFoutput ð1Þ is its output fuzzy membership n function for the nth fuzzy region. Step 2: Initial Network Setting In the beginning, there is no fuzzy rule in the network. Thus, the first rule should be built by the first data input. The connection weights between the 2nd layer and the 3rd layer are W 2input j;k;m ; between output the 3rd layer and the 4th layer are W 3m;n ; between the 4th layer and the output layer are W4n, where m is the rule number. Step 3: Network Constructing In order to construct the fuzzy rule and weights of the network, all of the training data must be processed by the following sub-steps: Step 3.1. Similarity Computing Instead of Kasabov’s fuzzy distance function, weighted Euclidean distance Di,m is used to generate the distance between the ith case and the mth rule, which have been determined as Di;m ¼

sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi h i2ffi XX input W 2input MF ðiÞ R ðmÞ j;k j;k;m j;k j

k

W1 j

ð5Þ Input layer

INPUT i

Fig. 1. Architecture of WEFuNN.

μ~x

In this weighted distance function, the direct distance between case i and rule m is computed to represent the difference of them. The weights are further taken into account in this study in an effort to express the degree of importance between each factor. Moreover, to present the relation between distance and similarity, an exponential transfer function is used to transfer the distance to the similarity. Which, Ai;m ¼ expðDi;m Þ

1

0

a

b

c

Fig. 2. The triangular membership function.

ð4Þ

x

ð6Þ

Comparing with the linear transfer function, much larger similarity in the close distance and much smaller similarity in the large distance from this function will be obtained. Step 3.2. Rule Determining Find the most similar rule of case i, where, A1i = max(Ai,m)If A1i > S, here S is the

90


similarity threshold, it presents that the case has to be merged in this rule. Thus, go to Step 3.3. Otherwise, create a new fuzzy rule and compute its connection output weights, W 2input j;k;m and W 3m;n , then go to Step 4. Step 3.3. Output Computing In this sub-step, saturating linear transfer function will be used to transfer the fuzzy membership function of case i to the fuzzy forecast output, A2i;m ¼ Satlin W 3output A1i ð7Þ m;n The Saturating linear transfer function is given in Fig. 3. Step 3.4. Error Computing Compute the error between fuzzy forecast of the case i and its actual fuzzy demand Ai, Erri ¼ jA2i;m Ai j

ð8Þ

If Erri < Ethr retain this forecasting result, where Ethr is the error threshold. Otherwise, create a new fuzzy rule and compute its connection weights, W 2input and j;k;m W 3output , then go to Step 4. m;n Step 3.5. Defuzzification Each fuzzy forecast output has been defuzzified to the real forecast output, where Oi ¼ W 4n A2i;m

ð9Þ

Step 3.6. Weights Updating For the occurrence of each merging process, the connection weights W 2input j;k;m and W 3output should be updated as follows: m;n h i dist ¼ MFinput ðiÞ R ðmÞ ð10Þ j;k j;k old Rj;k ðmÞnew ¼ Rj;k ðmÞold þ a1 dist

ð11Þ

MFoutput ðiÞnew ¼ MFoutput ðiÞold n n þ a2 ðErrÞ ðA1i Þ ð12Þ

Output +1

-1

0

+1

Input

-1

Fig. 3. Saturating linear transfer function.

Original rule scope Rule scope after update

data4

data1

data2

data3

Fig. 4. The diagram of rule scope before/after update.

Output

r1

r2

r3 r

r2

Input over time Fig. 5. The diagram of fuzzy rule generating.

where a1 is the learning rate of Rj,k(m) and a2 is the learning rate of MFoutput ðiÞ. The n diagram of rule scope before/after update is shown in Fig. 4. Step 4: Network generating For each training data, go to Step 3 to generate our network which includes the connection weights and fuzzy rules. After network feed-forward training phase, we can get the fuzzy rules of these training data. And these fuzzy rules will be used to forecast in recall forecast phase. The original concept of fuzzy rule after generating will be similar as Fig. 5. 4.2. Recall forecast phase In general, the most similar rule can be retrieved from the rule base for each testing data according to the similarity Di,m. In addition, the fuzzy output from this similar rule will be treated as the fuzzy forecast of this testing data. After defuzzification, the forecast demand can be divided. However, the concept of K-NN (k-nearest-neighbor method, Gordon, 1997) is utilized to find the forecast in this research. The most similar k rules of each testing data will be considered, the distances between the k-nearest rules and the testing data present the ratios in summarizing the forecasts of k rules. Thus, we can get the combined result from many


different rules. To sum up, we expect that we can get more accurate forecasting results.

5. Experiment results and analysis In this section, using WEFuNN model proposed in Section 4 to investigate the PCB sales forecasting and the most appropriate setup of parameters for this model. In addition, evaluating the forecasting results of WEFuNN, EFuNN, BPN, MRA and GANN, it is expected to find out the strengths and weakness of these models. This study proposed the most suitable sales forecasting model for PCB industry through the comparisons of various kinds of models and the experimental analysis. The commercial NN and language software used by WEFuNN and GANN and BPN are Matlab 6.5, Borland C++ Builder 5.0 and Neural Work Professional II Plus respectively. Moreover, grey relational analysis, Winter’s exponential smoothing and multiple regression analysis employ Excel built-in function. The experimental environment is in Pentium IV 2.4G, 256RAM. According to Chang and Lai (2005), Chang, Wang and Liu (2005) and Chang, Wang and Tsai (2005), the factors selected in this study have a maximum value of GRG in each domain and this represents that each factor has close relation to the monthly demand of PCB. However, the maximum values of GRG in above three domains are Index of Export Trade, Liquid Crystal Element and PCB Production value respectively. In the macroeconomic domain, the one has the highest GRG is the index of export trade. One possible reason is that more than 80% of the PCB produced by the Golden Circuit Company in Taiwan is for export. Accordingly, comparing with other factors, it is obvious that the demand of PCB highly rely on the value of export trade. In the downstream demand domain, liquid crystal element has the highest value of GRG. One possible reason is that the monitors for desktop PC and notebook PC are mainly liquid crystal monitors. Therefore, liquid crystal element is the most representative one in the downstream demand. In the industrial production domain, the one has highest GRG is the PCB sales value. One possible reason is that the change of the PCB sales value is the most representative trend for the demand of PCB industry. In addition, comparing with other industrial factors, it does not include the noise which is caused by other industries and it can also reflect the demand of the company directly. Therefore, PCB sales value is more useful since it takes other industrial indices into consideration. In order to take the effects of seasonality and trend into consideration, Winter’s exponential smoothing is used to preliminarily forecast the time serial of PCB production. There are three parameters in Winter’s exponential smoothing and via the method of trail-and-error can get the best parameter combination.

91

5.1. Parameters design for WEFuNN In order to get a better result, WEFuNN needs to decide the relevant parameters before forecasting. Hence, this study is divided into two stages to set up the parameters. The first stage employed the Taguchi method proposed by Dr. Taguchi to explore the parameter with the most significant influence. The second stage collected the best parameter combination from the Taguchi method. At the first stage, in order to use the less number of experiments to get a better result, this study employed the orthogonal array and then selected the best control value of each factor. There are 13 factors chosen as setting parameters of WEFuNN. The details of factors and levels are presented in Table 2 (where X1, X2, X3, and X4 represented as index of export trade, liquid crystal element, PCB production value and forecasting value of Winter’s exponential smoothing respectively). The objective function of this test is the smaller the better. It means that the error between the forecasting value and the actual value is the smaller the better. Hence, it needs to use a formula with this feature when calculating Signal-to-Noise (S/N) ratio. The formula is defined as ! n 1 X 2 S=N ¼ 10 log y ð13Þ n i¼1 i where n is the total number of experiment, yi is the result of the ith test, yi 2 (0, 1), "i = 1, 2, . . . , n. The orthogonal array used in this research is L27 (313). The results of each combination in WEFuNN are the same; hence, this experiment is only preceded once. Then compute the S/N ratio of each factor in each level and record all results into one table (as shown in Table 3). The higher the Signal-to-Noise (S/N) ratio the better the parameter combination will be. Therefore, the best parameter combination is to collect the biggest value in each domain from the table above. The best parameter combination can be found as A2-B3-C1-D3-E2-F1-G1-H1-J3K3-L1-M3-N2 and shown in Table 4. In order to make the best parameter combination more precise, this study further ranked the Delta statistic of 13 Table 2 Factors and their test levels in Taguchi experimental design Code

Factors

A B C D E F G H J L K M N

The no. of fuzzy region The no. of fuzzy region The no. of fuzzy region The no. of fuzzy region The no. of fuzzy region Weight of X1 Weight of X2 Weight of X3 Weight of X4 Threshold of error Threshold of similarity Learning rate of W1 Learning rate of W2

in in in in in

X1 X2 X3 X4 Y1

Level 1

Level 2

Level 3

8 3 3 8 3 0.25 0.20 0.15 0.25 0.1 0.7 0.1 0.1

9 4 4 9 4 0.275 0.25 0.20 0.3 0.2 0.8 0.5 0.5

10 5 5 10 5 0.3 0.3 0.25 0.35 0.3 0.9 0.9 0.9

92


Table 3 S/N ratio Level

A

B

C

D

E

F

G

H

J

K

L

M

N

1 2 3 Delta Rank

20.11 23.12 21.19 3.01 2

20.98 20.58 22.85 2.27 6

22.26 21.01 21.15 1.24 9

19.97 22.06 22.39 2.42 5

21.31 22.31 20.80 1.52 8

22.67 20.06 21.69 2.61 3

22.76 21.36 20.29 2.47 4

22.03 21.57 20.82 1.21 10

19.04 21.47 23.90 4.87 1

21.32 21.31 21.79 0.48 13

22.26 21.43 20.73 1.53 7

21.18 21.22 22.02 0.84 11

21.38 21.85 21.19 0.65 12

Weight

Code

Factors

Parameter

0.25 0.20 0.15 0.35

K L M N

Threshold of error Threshold of similarity Learning rate of W1 Learning rate of W2

0.3 0.7 0.9 0.5

Table 4 The best parameter combination of WEFuNN Code

Factors

A B C D E

The The The The The

no. no. no. no. no.

of of of of of

fuzzy fuzzy fuzzy fuzzy fuzzy

region region region region region

in in in in in

X1 X2 X3 X4 Y1

No.

Code

Factors

9 5 3 9 4

F G H J

Weight Weight Weight Weight

of of of of

X1 X2 X3 X4

Table 5 Detail setting in the 2nd Taguchi experiment Factors

Level 1

Level 2

Level 3

Level 4

Level 5

A B C D E

The no. of fuzzy region in X1 The no. of fuzzy region in X4 Weight of X1 Weight of X2 Weight of X4

8 9 0.24 0.19 0.35

9 10 0.25 0.20 0.36

10 11 0.26 0.21 0.37

0.27

0.28

factors and collected the first five significant factors for full factors experimental design. At the second stage, the five factors collected from the first stages are the weight of X4, the number of fuzzy region in X1, weight of X1, weight of X2 and the number of fuzzy region in X4. The level designs of each factor are based on the best level in the first stage and then reduce the region between each factor. The code and level of each factor are presented in Table 5. The experiment repeats 405 times. The results of them are recorded in turn and then analyzed. Based on the rule of ‘‘the smaller the better’’, the best parameter combination is A2-B1-C3-D2-E1 (the number of fuzzy region in X1 = 9, the number of fuzzy region in X4 = 9, weight of X1 = 0.26, weight of X2 = 0.2 and weight of X4 = 0.35). As shown in Table 6, the change in the number of fuzzy region in X1 has the biggest influence and the interactions between factor C, D and E are obvious.

Table 6 Detail parameter setting of WEFuNN Factors

The best parameters

The no. of fuzzy region in X1 The no. of fuzzy region in X4 Weight of X1 Weight of X2 Weight of X4

9 9 0.26 0.20 0.35

Demand

Code name

1000000 900000 800000 700000 600000 500000 400000 300000 200000 100000 0

Actual demand The forecast of WEFuNN

1

2

3

4

5

6

7 Month

8

9

10

11

12

Fig. 6. Comparison between WEFuNN and actual demand.

The best parameter combination is put into the WEFuNN model proposed in this study. The forecasting results are presented in Fig. 6. 5.2. Comparison with other forecasting models This study compares the results of WEFuNN in PCB sale forecasting with those of EFuNN, BPN, MRA and GANN. The averages and standard errors of MAPE, MAD and RMSE are viewed as the standard to evaluate the models. 5.2.1. Evolving fuzzy neural network model The Taguchi method employed by EFuNN to design the parameters is similar to that used in Section 5.3. However,

P.-C. Chang et al. / Expert Systems with Applications 32 (2007) 86–96 Table 7 The best parameters setting in EFuNN

Table 10 Forecasting results of BPN

Parameters

The best setup

The no. of fuzzy region in input factor The no. of fuzzy region in output factor The threshold of similarity Threshold of error Studying rate of W1 Studying rate of W2 The number of study time

2 2 0.9 0.1 0.5 0.5 1

Table 8 Forecasting results of EFuNN EFuNN (%) MAPE STD MAX MIN

93

6.44 4.31 13.32 0.66

in this section, the focus is on the performance comparison of EFuNN. Therefore, the details of the best parameters setup are not mentioned. The best parameters are presented in Table 7. The best parameter combination is put into the EFuNN model. There are 38 fuzzy rules developed during the training stage. The forecasting results are presented in Table 8. The forecasting results of EFuNN are: the MAPE of EFuNN is 6.44%, the error standard deviation is 4.31%, the maximum error percentage in 12 months forecast is 13.32% and minimum error percentage in 12 months forecast is 0.66%. 5.2.2. BPN model The neural network is the most popular model used for various application domains, such as sales forecasting of department store, bank personal credit comment, and manufacturing control. All of them predict the results with good efficiency. The Neural Network Professional II Plus is utilized for the experiment of neural network. In addition, the Gradient Steepest Descent Method, the most popular method, is employed to revise the weights of BPN. The Taguchi method employed by BPN to design the parameters is similar to that used in Section 5.3. However, in this section, the focus is on the performance comparison of BPN. Therefore, the best parameters setup of BPN is not mentioned. The best parameters are presented in Table 9 and the forecasting results are presented in Table 10. Table 9 The best parameter design of BPN Parameters

The best setup

The number of hidden layers The neuron number in hidden layers The learning rate Momentum coefficient The number of learning time

1 layer 2 0.4 0.5 100 000 times

BPN (%) MAPE STD MAX MIN

8.76 9.63 25.20 0.49

5.2.3. Multiple regression analysis model Multiple regression analysis is one of the most popular methods used in statistic. Multiple regression analysis is used for testing hypothesis with regard to the relationship between a dependent variable (Y) and two or more independent variables (Xs). It is easy to establish a model when there is a linear relationship between the independent variable and dependent variable. On the other hand, it is very difficult to establish an accurate model within a non-linear relation. In this research, the output factor (Y) is sales forecasting quantity of PCB and the input factors of linear regression are Index of Export Trade (X1), Liquid crystal element (X2), PCB Production Value (X3) and Winter’s exponential smoothing (X4). The regression formula is defined as: Y = a1X1 + a2X2 + a3X3 + a4X4 + b. Among them, a1, a2, a3, a4 and b are calculated via Excel built-in function and then the multiple regression analysis equation is presented as follows: Y ¼ 0:01141 X 1 þ 0:244169 X 2 5:91762 X 3 þ 0:969816 X 4 13140:4 Moreover, those samples are put into the multiple regression equation and the results are shown in Table 11. The forecasting results of MRA are: the MAPE is 9.88%, error standard deviation is 11.03%, maximum error percentage in 12 months forecast is 30.36% and minimum error percentage in 12 months forecast is 0.11%. 5.2.4. Genetic algorithms and neural network The weight revise of the traditional back-propagation network always limits to the searching space of the gradient steepest descent method. It can easily affect the speed of convergence. Accordingly, genetic neural network could offset the limitations of the gradient steepest descent method since the genetic algorithms has strong space searching ability. This study will establish a new model based on the genetic neural network model proposed by Chang, Wang and Tsai (2005). The best parameter setup

Table 11 Forecasting results of MRA MRA (%) MAPE STD MAX MIN

9.88 11.03 30.36 0.11

94


Table 12 The best parameters setting of GANN Parameters

The best setup

Crossover Mutation Replacement Crossover rate Mutation rate Hidden layer Generation

Two point Single point Elite strategy 0.8 0.3 2 300

Table 13 Forecasting results of GANN GANN (%) MAPE STD MAX MIN

3.06 2.61 8.40 0.01

2.

3.

4.

5.

6. by using Taguchi method is shown in Table 12, and the forecasting results are shown in Table 13. As shown in Table 13, GANN is a good forecasting model since the MAPE is 3.06%, error standard deviation is 2.61%, maximum error percentage in 12 months forecast is 8.40% and minimum error percentage in 12 months forecast is 0.01%. 5.3. Integrated comparisons In this section, it will focus on the comparisons between WEFuNN and other forecasting models. As shown in Table 14, this study employed MAPE and MAD to evaluate the degree of accuracy. Among these five forecasting models, WEFuNN is the most accurate forecasting model since the value of MAPE and MAD is 2.11% and 16445 square foot respectively. In addition, RMSE is used as an evaluating index for the degree of precise. The value of RMSE is 24 909 square foot. The results show that WEFuNN has the best degree of preciseness among these five forecasting models. Accordingly, it could be concluded that the WEFuNN forecasting model proposed in this study has considerably accurate forecasting ability. According to the experimental results above, some conclusions could be drawn as follows: 1. The model having other domains taken into consideration rather than just focus on the trend factors and seasonal factors will get a better result no matter in the

average or the standard deviation of error. Therefore, taking other domains into account can reduce the error and standard deviation effectively and then increase the accuracy and stability of the model. The forecasting model having all input factors taken into account and these factors assigned different weight value will get more accurate results than the forecasting model with the same weight value between each factor. Considering the accuracy and the precise of the forecasting model, the WEFuNN is superior to other forecasting models since it has the minimum value of MAPE and RMSE. As for the cost variance caused by the forecasting errors, WEFuNN has the minimum error cost when compared with other forecasting models. The WEFuNN usually takes one epoch to get the results. It is faster than other forecasting models such as BPN. The forecasting ability of Multiple Regression Analysis is the worst one among the five forecasting models as it is only suitable for the linear question and its forecasting ability for the non-linear question is comparatively poor and weak.

5.4. Discussions According to the experimental results shown in the previous section, it could be concluded that the forecasting model proposed in this study is superior to the others. The possible reasons might be as follows: 1. WEFuNN utilized weight is allocated according to the degree of importance in each factor to calculate the similarity. This approach is much better than EFuNN with the same weight in each factor. 2. The original WEFuNN used Fuzzy Distance Function to calculate the value of similarity. Thus, the value of similarity is 1 D when the distance between the two factors is D. However, in this study, exponential transfer function (exp(D)) is employed to transfer the distance to the value of similarity, and the result shows that the outcome is better after using the exponential transfer function. 3. At the recall stage, the method of combining WEFuNN with K-Nearest-Neighbor to get the fuzzy rule is much accurate than the method of WEFuNN which only finds out the most similar fuzzy rule. It is because that

Table 14 The comparison of each forecasting methods

MAPE MAD RMSE

WEFuNN

EFuNN

BPN

MRA

GANN

2.11% 16 445 24 909

6.44% 47 072 59 081

8.76% 72 494 114 785

9.88% 82 673 131 115

3.06% 23 991 33 497


K-Nearest-Neighbor can modify the accuracy of the forecasting results and relatively adapt to the complicated environmental change. 4. As can been seen from the experimental results, the forecasting results of WEFuNN, EFuNN and GANN are better than that of BPN which employed the Gradient Steepest Descent Method. According to Chang, Wang and Tsai (2005), this result is in line with the previous finding that the performance of the Gradient Steepest Descent Method is not good.

2.

3.

6. Conclusions and future researches 6.1. Conclusion The printed circuit board industry in Taiwan has already become the third largest manufacturer among the world. The total output value in the year 2004 is up to NT$C174,126 million which brings considerable contribution to the domestic economy. However, the competition is getting severer as a result of the imbalance between demand and supply, the increase of manpower costs and over-produce. Therefore, capacity planning becomes an important topic today. If it is possible to control the capacity effectively by using scientific knowledge and modernized management skill to assist the industry, it will not only improve its competitiveness, reduce the cost, control the demand but also have positive effects for the relevant industry and the domestic economy. In recent years, the application of soft computing which integrates fuzzy theory and neural networks is getting more and more popular. However, few evidences show that it was employed in PCB sales forecasting. By employing the WEFuNN, this study offers the PCB company in Taiwan a precise monthly forecasting which would facilitate the capacity planning, the preparation of material flows and so on. As can be seen from the results, the MAPE, MAD and RMSE of the WEFuNN forecasting model are 2.11%, 16445 square foot, and 24 909 square foot respectively. It also indicates that this highly accurate forecasting model could be a scientific basis for capacity planning and could further reduce the extra production costs caused by unfavorable forecasting. 6.2. Future researches At the end of this research, there are some interesting findings. These findings might be interested to other academic researchers and could be a reference for relevant research in the future. 1. As for the PCB industry, the environment change of the electronic industry will affect the demand of PCB. This study only takes macroeconomic factors, downstream demand factors and industrial production factors into consideration. Hence, it might be an interesting aspect

4.

5.

6.

95

for the further study to adjust these factors based on the scope of PCB application and the macroeconomic environment at that time in an effort to correspond with the practical industry status. To do so, it might further improve the forecasting ability of PCB. As the amount of data is increasing, other methods can be used to collect the most suitable factor in each domain and then explore which method performs better. There are several forms of membership function such as triangular membership function, Bell-shaped membership function and Trapezoidal membership function. This study mainly employed triangular membership function. Therefore, for further study, other forms of membership function can be used to test the result. This study uses Taguchi method to design the weight of each input factor; therefore, the more the input factors, the more complicated the method will be. It is believed that it can quickly and effectively improve the accuracy of forecasting through integrating heuristic algorithm and WEFuNN to derive the best weight of each factor. There are many advantages of WEFuNN, fast training ability, good adaptation of non-linear problem and high degree of accuracy, for instance. Consequently, this model could be used to carry on the forecasting of other industrial products or to control other production activities. The Euclidean distance method is used in this study to calculate the similarity of fuzzy rule. Other similarity calculation methods can be further applied to cluster the fuzzy rules and the forecasting accuracy is the key measurement to determine the best method applied.

References Abraham, A., & Baikunth, N. (2001). A neuro-fuzzy approach for modeling electricity demand in Victoria. Applied Soft Computing, 127–138. Abraham, A., Baikunth, N., & Mahanti, P. K. (2001). Hybrid intelligent systems for stock market analysis. Lecture Notes in Computer Science (vol. 2074, pp. 337–345). Abraham, A., Steinberg, D., & Philip, N. S. (2001). Rainfall forecasting using soft computing models and multivariate adaptive regression splines. IEEE SMC transactions: Special issue on fusion of soft computing and hard computing in industrial applications, February. Arifovic, J., & Ramazan, G. (2001). Using genetic algorithms to select architecture of a feedback artificial neural network. Physica A, 574–594. Chang, L. (2002). Production Management. Taiwan, TsangHai. Chang, P. C., & Hsieh, J. C. (2003). A neural network approach for duedate assignment in a wafer fabrication factory. International Journal of Industrial Engineering, 10, 55–61. Chang, P. C., & Lai, C. Y. (2005). A hybrid system combining selforganizing maps with case-based reasoning in wholesaler’s new-release book forecasting. Expert Systems with Applications, 29(1), 183–192. Chang, P. C., Wang, Y. W., & Liu, C. H. (2005). Fuzzy back-propagation network for PCB sales forecasting. In L. Wang, Chen, K., & Y. S. Ong (Eds.), LNCS (vol. 3610, pp. 364–373). Chang, P. C., Wang, Y. W., & Tsai, C. Y. (2005). Evolving neural network for printed circuit board sales. Expert Systems with Applications, 29(1), 83–92.

96


Elalfi, A. E., Haque, R., & Elalami, M. E. (2004). Extracting rules from trained neural network using GA for managing E-businesss. Applied Soft Computing, 4, 65–77. Kasabov, N. (1998). Evolving fuzzy neural networks-algorithms, applications and biological motivation. In Proceedings of Iizuka’98, Iizuka, Japan, October. Kasabov, N. (2001). Evolving fuzzy neural networks for on-line knowledge discovery. The Information Science Discussion Paper Series. Kasabov, N., Kim, J., & Watts, M. J. (1997). Funn/2—a fuzzy neural network architecture for adaptive learning and knowledge acquistion. Information Sciences, 101, 155–175. Kim, K. J., & Han, I. (2000). Genetic algorithms approach to feature discretization in artificial neural networks for the prediction of stock price index. Expert Systems with Application, 19, 125–132. Kuo, R. J., & Chen, J. A. (2004). A decision support system for order selection in electronic commerce based on fuzzy neural network supported by real-coded genetic algorithm. Expert Systems with Application, 26, 141–154. Liang, R. H. (1999). Application of grey relation analysis to hydroelectric generation scheduling. International Journal of Electrical Power & Energy Systems, 21(5), 357–364.

Lin, C. T., & Lee, C. G. (1991). Neural-network-based fuzzy inference system. IEEE Transactions on Computer, 40(12), 1320–1336. Liu, Z., Liu, A., Wang, C., & Niu, Z. (2004). Evolving neural network using real coded genetic algorithm for multispectral image classification. Future Generation Computer System, 20, 1119–1129. Montana, D., & Davis, L. (1989). Training feed forward neural networks using genetic algorithms. In Proceedings of 11th international joint conference on artificial intelligence (pp. 762–767). San Mateo, CA: Morgan Kaufmanns. Sexton, R. S., & Gupta, J. N. D. (2000). Comparative evaluation of genetic algorithm and backpropagation for training neural networks. Information Sciences, 129, 45–59. Srinivasan, D. (1998). Evolving artificial neural networks for short term load forecasting. Neural Computing, 23, 265–276. Yao, X. (1999). Evolving artificial neural networks. Proceedings of the IEEE, 87(9), 1423–1447. Zhung, I. F. (1999). Reinforcement neural fuzzy inference networks and its applications. Ph.D. thesis, Department of electronic and control engineering, National Chaio Tung University, Taiwan.