A Hierarchical Self-Organizing Map Model in Short-Term ... - CiteSeerX

A Hierarchical Self-Organizing Map Model in Short-Term Load Forecasting

Ot´ avio A. S. Carpinteiro & Alexandre P. Alves da Silva Instituto de Engenharia El´etrica Escola Federal de Engenharia de Itajub´ a Av. BPS 1303, Itajub´ a, MG, 37500–903, Brazil e-mails: {otavio,alex}@iee.efei.br

Abstract This paper proposes a novel neural model to the problem of short-term load forecasting. The neural model is made up of two self-organizing map nets — one on top of the other. It has been successfully applied to domains in which the context information given by former events plays a primary role. The model was trained and assessed on load data extracted from a Brazilian electric utility. It was required to predict once every hour the electric load during the next 24 hours. The paper presents the results, and evaluates them.

Key words: short-term load forecasting, neural networks, self-organizing map.

1

Introduction

With power systems growth and the increase in their complexity, many factors have become influential to the electric power generation and consumption (load management, energy exchange, spot pricing, independent power producers, non-conventional energy, generation units, etc.); therefore, the forecasting process has become even more complex, and more accurate forecasts are needed. The relationship between the load and its exogenous factors is complex and non-linear, making it quite difficult to model through conventional techniques, such as time series and linear regression analysis. Besides not giving the required precision, most of the traditional techniques are not robust enough. They fail to give accurate forecasts when quick weather changes occur. Other problems include noise immunity, portability and maintenance [1]. Neural networks (NNs) have succeeded in several power system problems, such as: planning; control; analysis; protection; design; load forecasting; security analysis; and fault diagnosis. The last three are the most popular [2]. The NN ability in mapping complex non-linear relationships is responsible for the growing number of its application to the short-term load forecasting (STLF) [3, 4, 5, 6]. Several electric utilities over the world have been applying NNs for load forecasting in an experimental or operational basis [1, 2, 4]. So far, the great majority of proposals on the application of NNs to STLF use the multilayer perceptron trained with error backpropagation. Besides the high computational burden for su-

pervised training, multilayer perceptrons do not have a good ability to detect data outside the domain of the training data. The Kohonen’s self-organizing map has been proposed to overcome these shortcomings [7]. However, the Kohonen’s self-organizing map does not work appropriately for regression problems, because it does not preserve the functionality of the regression surface [8]. This paper introduces an extension of the original self-organizing map to STLF. It has been applied to load data extracted from a Brazilian electric utility. In the next sections, we shall introduce and detail the model, the experiment, evaluate the results, and draw some conclusions.

2

Representation for the Sequences

The input data in the experiment consists of sequences of load data. Eight neural input units are used in the representation. The first unit represents the load at the current hour. The second, the load at the hour immediately before. The third, the load two hours before. The fourth, fifth, and sixth units represent respectively the load at twenty-four hours behind, at one week behind, and at one week and twenty-four hours behind the hour whose load is to be predicted. The seventh and eighth units represent a trigonometric coding for the hour to be forecast, i.e., sin(2π.hour/24) and cos(2π.hour/24). Each unit receives real values. The load data is normalized, varying therefore, from zero to one.

3

The Model

The model is made up of two self-organizing maps (SOMs), as shown in Figure 1. Its features, performance, and potential are better evaluated in [9, 10]. Top

Map

SOM

Time Integrator Λ

Bottom SOM

Map Time Integrator V(t )

Figure 1: The model The input to the model is a sequence in time of m-dimensional vectors, S1 = V(1), V(2), . . . , V(t), . . . , V(z), where the components of each vector are non-negative real values. The sequence is presented to the input layer of the bottom SOM, one vector at a time. The input layer has m units, one for each component of the input vector V(t), and a time integrator. The activation X(t) of the units in the input layer is given by

X(t) = V(t) + δ1 X(t − 1)

(1)

where δ1 ∈ (0, 1) is the decay rate. For each input vector X(t), the winning unit i∗ (t) in the map1 is the unit which has the smallest distance Ψ(i, t). For each output unit i, Ψ(i, t) is given by the Euclidean distance between the input vector X(t) and the unit’s weight vector Wi . Each output unit i in the neighbourhood N ∗ (t) of the winning unit i∗ (t) has its weight Wi updated by

Wi (t + 1) = Wi (t) + αΥ(i)[X(t) − Wi (t)]

(2)

where α ∈ (0, 1) is the learning rate. Υ(i) is the neighbourhood interaction function [11], a Gaussian type function, and is given by

Υ(i) = κ1 + κ2 e−

κ3 [Φ(i,i∗ (t))] 2 2σ2

(3)

where κ1 , κ2 , and κ3 are constants, σ is the radius of the neighbourhood N ∗ (t), and Φ(i, i∗ (t)) is the distance in the map between the unit i and the winning unit i∗ (t). The distance Φ(i0 , i00 ) between any two units i0 and i00 in the map is calculated according to the maximum norm, Φ(i0 , i00 ) = max {|l0 − l00 | , |c0 − c00 |}

(4)

where (l0 , c0 ) and (l00 , c00 ) are the coordinates of the units i0 and i00 respectively in the map. The input to the top SOM is determined by the distances Φ(i, i∗ (t)) of the n units in the map of the bottom SOM. The input is thus a sequence in time of n-dimensional vectors, S2 = Λ(Φ(i, i∗ (1))), Λ(Φ(i, i∗ (2))), . . . , Λ(Φ(i, i∗ (t))), . . . , Λ(Φ(i, i∗ (z))), where Λ is a n-dimensional transfer function on a n-dimensional space domain. Λ is defined as

Λ(Φ(i, i∗ (t))) =



1 − κΦ(i, i∗ (t)) 0

if i ∈ N ∗ (t) otherwise

(5)

where κ is a constant, and N ∗ (t) is a neighbourhood of the winning unit. The sequence S2 is then presented to the input layer of the top SOM, one vector at a time. The input layer has n units, one for each component of the input vector Λ(Φ(i, i∗ (t))), and a time integrator. The activation X(t) of the units in the input layer is thus given by X(t) = Λ(Φ(i, i∗ (t))) + δ2 X(t − 1) 1 Also

known as array, grid, or output layer.

(6)

where δ2 ∈ (0, 1) is the decay rate. The dynamics of the top SOM is identical to that of the bottom SOM.

4

The Experiment on STLF

The experiment was on the prediction, once every hour, of electric load during the next 24 hours. The input data consisted of load data extracted from a Brazilian utility. The data was coded according to the representation presented in Section 2. The training of the two SOMs of the model took place in two phases — coarse-mapping and finetuning. In the coarse-mapping phase, the learning rate and the radius of the neighbourhood were reduced linearly whereas in the fine-tuning phase, they were kept constant. The bottom and top SOMs were tested and trained respectively with map sizes of 15×15 in 700 epochs, and 18×18 in 850 epochs. The initial weights were given randomly to both SOMs. The experiment comprised seven studies. In each study, one neural network was trained to hourly predict the load of a specific day of the week. Thus, the neural network of the first study was required to predict the load of Fridays. The network of the second study was to predict the load of Saturdays, and so on. Table 1 displays the percentage of the error as well as the mean error for each study. Figures 2– 8 plot the actual load, and the forecast load for the seven studies.

4000 3900 3800

Actual Load Forecast Load

Load (MW)

3700 3600 3500 3400 3300 3200 3100 3000 1537

1540

1543

1546

1549

1552

1555

Hour

Figure 2: Actual load and forecast load for the first study

1558

Table 1: Percentage of the error and mean error for each study 
    %  ' ! &  #     *%   *' ! (&  # % %  %% % %( -. /



 

 

   &&  !  ' " !$' " %  &  &(! " % %!  #  &* % )&  #   #  # ' ! % $# " % %( " % &*  &' % && *%  # % &'

  '  '*' % #! "+ &% % '!  %'  % % %( %% & ! ! % ! *' !! % '# (&)(& % ' % % &)   $' %  ! ' ! )&  %( (&) $  )&

!     $ #   )% ' !$' ! % *' % %% %! %  !  &(   %*'  !  $    &%  %# ' )% " % !$% " % #  %$& ",*' ", % ' !!



 



 

"  "  "+ % " &( " && " ' $! ",*' "# ! "#  "+ &' " # " ' "+ )% " !! " # # " %*' ", ! ", $% " $  &(! % #$%  # !(&  &%

 #  " $! " && "% ", &% "% # ", $ "  " &' " # " $%    '(  #  !  ! ",! " $' "  %#  %( !    % !$'

#  $!  (&) # % ! )& ! )% ' &* " ' % ! " ' &)   !   %   #$' ! )' '*%  !# (& % $ "+ !)& % &( ! &'  &(#  $  ##

  *% #)& ! $% ! ! ' ! !(& % % ", % *'  $%  &)  && % )& %(& *'  $  # "% $% ", '$&  ## " )' "% !# *%  !$'  )&

3600 3500 3400

Actual Load Forecast Load

Load (MW)

3300 3200 3100 3000 2900 2800 2700 2600 1561

1564

1567

1570

1573

1576

1579

1582

Hour

Figure 3: Actual load and forecast load for the second study

3300 3200 3100

Actual Load Forecast Load

Load (MW)

3000 2900 2800 2700 2600 2500 2400 2300 1585

1588

1591

1594

1597

1600

1603

1606

Hour

Figure 4: Actual load and forecast load for the third study

3600 3500 3400

Actual Load Forecast Load

Load (MW)

3300 3200 3100 3000 2900 2800 2700 2600 1609

1612

1615

1618

1621

1624

1627

Hour

Figure 5: Actual load and forecast load for the fourth study

1630

3600 3500 3400

Actual Load Forecast Load

Load (MW)

3300 3200 3100 3000 2900 2800 2700 2600 1633

1636

1639

1642

1645

1648

1651

1654

Hour

Figure 6: Actual load and forecast load for the fifth study

3500 3400 3300

Actual Load Forecast Load

Load (MW)

3200 3100 3000 2900 2800 2700 2600 2500 1657

1660

1663

1666

1669

1672

1675

Hour

Figure 7: Actual load and forecast load for the sixth study

1678

3500 3400 3300

Actual Load Forecast Load

Load (MW)

3200 3100 3000 2900 2800 2700 2600 2500 1681

1684

1687

1690

1693

1696

1699

1702

Hour

Figure 8: Actual load and forecast load for the seventh study The results are very promising, specially for several steps ahead forecasting. However, for a few steps ahead, the performance of the proposed model is poor. This behaviour confirmed our expectations. Medium decay rates extend the memory size for past events, and consequently, yield more accurate predictions on large horizons. As a side effect, however, in the predictions of the first hours of a day, medium decay rates led the model to take also into consideration the load from the last hours of the day immediately before, which in general shares little similarity with that of the day in question. More investigation on the effects of time integration is needed in order to produce a better adaptability. It is possible that one unit in the grid which has not responded to any load pattern during the training phase be requested to respond to a particular one during the prediction phase. The occurrence of this situation was rare, although it has produced highly undesirable effects. These effects may be seen, for example, after the points marked with arrows in figure 3. The neural network started producing very bad predictions, and it never recovered. A strategy must be planned to avoid this situation. More, continuous load data was employed as input in all studies. Yet, this data should have been refined to include just the pertinent data in each study. For instance, to hourly predict the load on a specific Monday, continuous load data including that of all days of the week was employed. Thus, the neural model was trained with irrelevant data once load patterns usually vary considerably from one day to the next. We believe that the refinement of the input data will improve the overall performance of the model.

5

Conclusion

A novel artificial neural model for sequence classification and prediction is presented. The model has a topology made up of two self-organizing map networks, one on top of the other. It encodes and manipulates context information effectively.

The results obtained have shown that the artificial neural model was able to perform efficiently the prediction of the electric load, particularly for larger forecasting horizons. We intend to do further research on the model, and study the effects of the time integrators on the predictions.

References [1] A. Khotanzad and et al. An adaptive modular artificial neural hourly load forecaster and its implementation at electric utilities. IEEE Trans. on Power Systems, 10(3):1716–1722, August 1995. [2] H. Mori. State-of-the-art overview on artificial neural networks in power systems. In M. ElSharkawi and D. Niebur, editors, A Tutorial Course on Artificial Neural Networks with Applications to Power Systems, chapter 6, pages 51–70. IEEE Power Engineering Society, 1996. [3] K. Liu and et al. Comparison of very short-term load forecasting techniques. IEEE Trans. on Power Systems, 11(2):877–882, May 1996. [4] A. Bakirtzis and et al. A neural network short-term load forecasting model for the greek power system. IEEE Trans. on Power Systems, 11(2):858–863, May 1996. [5] O. Mohammed and et al. Practical experiences with an adaptive neural network short-term load forecasting system. IEEE Trans. on Power Systems, 10(1):254–265, February 1995. [6] D. Park and et al. Electric load forecasting using an artificial neural network. IEEE Trans. on Power Systems, 6(2):442–449, May 1991. [7] T. Baumann and A. Germond. Application of the kohonen network to short-term load forecasting. In Proceedings of the Second International Forum on Application of Neural Networks to Power Systems, pages 407–412, Yokohama, Japan, April 1993. [8] V. Cherkassky and F. Mulier. Learning from Data. John Wiley & Sons, 1998. [9] O. A. S. Carpinteiro. A hierarchical self-organizing map model for pattern recognition. In L. Caloba and J. Barreto, editors, Proceedings of the Brazilian Congress on Artificial Neural Networks 97 (CBRN 97), pages 484–488, Florian´ opolis, SC, Brazil, 1997. UFSC. [10] O. A. S. Carpinteiro. A hierarchical self-organizing map model for sequence recognition. In L. Niklasson, M. Bod´en, and T. Ziemke, editors, Proceedings of the International Conference on Artificial Neural Networks 98, pages 815–820, Sk¨ ovde, Sweden, 1998. Springer-Verlag. [11] Z. Lo and B. Bavarian. Improved rate of convergence in Kohonen neural network. In Proceedings of the International Joint Conference on Neural Networks, volume 2, pages 201–206, July 8–12 1991.