A Relational Neural Network Database Model Erich Schikuta, Christian Brunner and Christian Schultes Institute for Applied Computer Science and Information Systems, Department of Data Engineering, University of Vienna, Rathausstr. 19/4, A-1010 Vienna, Austria Email:
[email protected] Fax. +43 1 4277 38428
Keywords
Arti cial neural network simulation system, database systems, relational modeling, embedding approach
Abstract
In this paper a framework for the conceptual and physical integration of neural networks into relational database systems is presented. The neural network 'meta model', as the neural network paradigms, the static and dynamic properties, and the pattern description, is realized by a collection of relations and dependencies between them. The speci c network objects (the 'ortho' networks) are tuples in these relations. An object oriented approach is followed for description of the paradigm hierarchies, and it is shown, how it can be mapped to the relational model. Further a handling scheme for the application data set, as input, output and training information, stored in the relational database together with the neural networks is depicted. Finally the design of the new NeuroAccess system based on the proposed framework is presented.
1 Introduction Many di erent approaches for the modeling of neural networks were presented in the literature (e.g. [H+ 92]). Generally the object-oriented approach proved itself as most appropriate. It provides a concise but comprehensive framework for the design of neural networks in terms of its static and dynamic components (the information structure and its methods in the object-oriented notion). The static components comprise the structural parts of a neural network, as the neurons and connec-
tions, higher topological structures as layers, blocks and network systems. The dynamic components are the behavioral characteristics, as the creation, training and evaluation of the network. In our opinion four design characteristics for the development of a modeling framework for neural networks have to be followed: simplicity, originality,
exibility, and performance.
Simplicity The handling of a neural network has to be simple. The environment has to supply functions to manipulate neural networks in an easy and (more important) natural way. This can only be achieved by a natural and adequate representation of the network in the provided environment of the system but also by an embedding of the network into the conventional framework of the system. A neural network software instantiation has to look, to react and to be administrated as any other object. The creation, update and deletion of a neural network instantiation has to be as simple as that of a conventional object. All neural network speci c tasks have to be packed into the basic functionality of the software user interface, e.g. the de nition and manipulation language. Originality A neural network has to be simple, but it has not to loose its originality. A common framework always runs the risk to destroy the original properties of the unique objects. This leads to the situation that either objects of di erent types loose their distinguishable properties or loose their functionality. A suitable framework for neural networks has to pay attention to the properties and to the functional power of neural networks and should give the user the characteristics he is expecting.
Flexibility One property of neural networks is
their beneath unrestricted exibility in providing solution strategies to all types of problems. This covers di erent versions of neural networks of one network paradigm for di erent problems or networks of different paradigms for a common problem. This can ease the work with neural networks dramatically, especially during the training phase of a net. However
exibility has not to be paid with performance. In many software systems the user has to pay for exibility with decreased performance, a typical problem known form relational database systems. This situation leads to the fourth design characteristic.
Performance Performance is a major factor for
the general acceptance of a modeling framework. If the user is burdened by a clumsy and slow system the best modeling environment will loose its attraction and will be replaced by a simpler, but faster system. The software systems has to provide an on-line interface to the user, which guarantees acceptable response times. A neural network training or evaluation phase has to be of reasonable performance to yield a high degree of acceptance. This demands a smart implementation of the algorithms and, if possible, the usage of specialized software or hardware facilities, as database system or parallel processor architectures to name only a few. Summing up, in our opinion all all four mentioned characteristics have to be adhered in the development of an arti cial neural network simulator (ANNS). Problematic is that they are counterproductive, the increase of one of them decreases the other. This correlation is best expressed by gure 1. Therefore a golden mean has to be reached. We see a way in the embedding of neural networks as basic elements into the well-known and standardized environment of database systems (see [Sch96] for object-oriented systems). In this paper we present the underlying data model of the NeuroAccess system, an arti cial neural network simulation system integrated into a relational database system.
2 The NeuroAccess's Database Model The object-oriented model proved extremely appropriate for the speci cation and modeling of neural network systems. Object oriented database systems have proven very valuable to handle and manage complex objects.
One describing property of the object oriented design is the hierarchy of types. A type comprises a set of objects, which share common functions. Generalization and specialization de ne a hierarchical type structure, which organizes the unique types. Functions de ned on a super-type are also inherited by all of its sub-types along the type hierarchy. However, many state-of-the-art and widely used database systems provide a relational data model framework only and do not support the objectoriented paradigm until now. The relational approach [Cod70] is declarative and value-oriented. Operations on relations are expressed by simple and declarative languages delivering their results by new relations. Today the relational approach is the model of choice in the community and provided by beneath all available database systems. The neural network is represented by values in the database system. The semantic information is expressed by relationships between these values.
2.1 Neural Network Objects A neural network object is de ned by its static and dynamic components. These components are dependent on the appropriate network paradigm. Di erent paradigms show di erent static and dynamic properties. In the following we use the object-oriented modeling approach due to its expressive power. We show in section 2.2 how the object-oriented structures are mapped to the relational model, which is the model in focus.
2.2 Type Hierarchy Mapping To provide the expressiveness and exibility of the object oriented framework in a relational system a mapping from the object-oriented neural network speci cation to the relational data modeling environment has to be de ned. In the relational model all speci c neural network types along a type hierarchy are represented by single relations. The relationship between these types is expressed in our approach by a hierarchical numbering scheme stored together with the relation. Di erent types on the same hierarchy are mapped to di erent numbers. Subtypes are expressed by the number of their supertype, a dot, and the speci c number of the respective subtype, similar to a categorization scheme. This approach is depicted by gure 2. The hierarchy paths are attributed with the respective number codes.
2.2.1 Static Components
The static neural network components comprise all information stored in relations, as neural network speci c parameters, links, training objects, evaluation objects, etc. respective to the shown entityrelationship diagram ( gure 3). The neural network object is a sub-type of the general object type of the database system. Subtypes can be classi ed into specialized neural network types according to their network paradigm. The network paradigm is de ned by a specialization, a sub-type of the neural network type. This sub-type (which inherits all characteristics of its super-type) provides the speci c and necessary attributes dependent on the network paradigm. Combined with the de nition of the paradigm are the dynamics (the dynamic behavior) of the network. This approach is re ected in the entityrelationship (ER) diagram of gure 3. An ER diagram is useful for the description of the conceptual schema of the 'reality' in focus. The transformation of the model in ER diagram to a database realization is straightforward. Rectangles represent entity sets, circles attributes, and connections relationships between entities. For a indepth explanation see [Dat86]. The underlined table elements represent the unique keys of the database relations.
NN TYPE. The single nettypes are coded by numbers which are separated by a dots. Thus simulates a hierarchical ordering, which codes the objectoriented paradigm inheritance (e.g.: 1.2.5). NN. This relation de nes the assignment of a network paradigm to a speci c neural net object.
BLOCK. It de nes the layer structure of a net-
work. Neurons are grouped into blocks, which in turn can represent layers of the network. This information is used for the graphical presentation of neural nets. Further a mapping from the input/output stream objects to the input/output neurons can be described.
NN X STRUCT. The structure of a net can be de ned by several structure elds. Number and type of the structure elds depend on the nettype of the neural net. INPUT CONNECT, OUTPUT CONNECT.
This data structures de ne the connections between the elements of the input/output stream and the neurons of the neural net.
INPUT. It de nes the data ow which contains the speci c input values for a neural net object. By using relation INPUT CONNECT it is possible to map elements of the input stream with data width Unit Number to neurons of the neural net object.
TRAIN. It de nes which neural nets will be trained by which input stream. TRAIN X STRUCT,
TRAIN X RESULT.
All necessary parameters of a training can be de ned by several structure elds in these relations. Number and type of the structure elds are depended on the speci c paradigm.
EVAL. All evaluations are kept in the database in
this relation. Together with the neural network object are stored the input values and a speci c training object.
EVAL X RESULT. The results of an evaluation can be described by several result elds. Number and type of the result elds depend on the nettype of the neural net. 2.2.2 Dynamic Components The dynamic components of the neural network object are the typical operations on neural network, the training and evaluation phase. The algorithms for these phases are realized by routines coded in the internal database application code (Visual Basic procedures). Further these routines keep certain consistency assertions on the static components after execution of speci c phases, as insertion of link weights after a training, results after an evaluation phase, and so on.
2.3 Datastream Concept The functional data stream de nition allows to specify the data sets in a comfortable way. It is not necessary to specify the data values explicitly, but the data streams can be described by SQL statements. In the database component of the NeuroAccess system the well known apparatus of the SQL database manipulation language ([MS93]) is at hand. Thus the same tool can both be used for administration and analysis of the stored information. So it is easily possible to use 'real world' data sets as training set for neural networks and to analyze other (or the same) data with trained networks.
2.4 Extensibility
3 Implementation Aspects
An important aspect is the extensibility of the system. This is reached by a modularized paradigm approach. New modules have to follow a speci c programming style to make it possible to integrate them easily into the NeuroAccess environment. Thus users have the possibility to shape the system to their needs by changing existing or adding new paradigms easily. All these implementations can be done without leaving the comfortable database environment. Creating a new nettype, i.e. a new neural network paradigm, is not as trivial as creating a new net or training object. It is necessary to create new tables, forms and modules. This procedure will be summarized by the following steps:
The NeuroAccess system was realized using the MS Access database system running on Windows95/NT1. The Access relations were used to model the static components and the Visual Basic application language for the dynamic components. These application code is stored in the database together with the static information, grouped accordingly to the speci c network paradigm. Until now NeuroAccess supports 10 network paradigms, which are Backpropagation (3 di erent variations), Counterpropagation, Hop eld, Boltzmann, ART1, ART2, Jordan, and Elman networks. In gure 4 a screenshot of the NeuroAccess system is given. The main window for the choice of the paradigm in focus, the object window of a backpropagation network and the respective structure and error curve window of a training phase can be identi ed.
Create a new database entry in table NN TYPE. A record for the new paradigm has to
be inserted into the central paradigm table to make it known to the system (and in turn to the user).
Add new tables. According to the speci ed data model the following tables have to be created EVAL X RESULT, NN X STRUCT, TRAIN X RESULT, and TRAIN X STRUCT. These tables contain the speci c information of the new paradigm for creation of the network, training and evaluation. These are typically structural information, learning rates, activation functions, etc. 'X' stands for the hierarchical code of the new paradigm. Create new forms. Interfaces for the created ta-
bles have to established, i.e. Eval x Result, Net x, NN x Struct, Train x Result, Train x Struct, and View Train x Struct. These forms are necessary to allow the user to apply the new paradigm and to access all describing attributes of the dynamic network operations.
Add new and modify old the language modules. For all dynamic neural network operations respective procedure modules have to be provided. These are procedures for training and evaluation of the network. A standardized call interface has to be followed. Few system procedures have to be changed accordingly to re ect the new paradigm. However it is a well described path to establish a new paradigm and applicable for users with minimal programming experiences at least. 1
4 Conclusions and Future Research We presented in this paper a relational model for the embedding of neural networks into data base systems. Based on this framework the NeuroAccess system was developed, an extensible, comfortable, and powerful neural network tool embedded into the relational MS Access database system. This approach provides a homogeneous and natural environment for the administration and handling of neural networks to the user.
References [Cod70] E.F. Codd. A relational model for large shared data banks. Communications of the ACM, 13(6):377{387, 1970. [Dat86] C.J. Date. An Introduction to Database Systems. Addison-Wesley, 1986. [H+ 92] G. Heileman et al. A general framework for concurrent simulation of neural networks models. IEEE Trans. Software Engineering, 18(7):551{562, 1992. [MS93] J. Melton and A.R. Simon. Understanding the new SQL: A Complete Guide. Morgan Kaufmann Publishers, 1993.
MS Access and Windows95/NT are trademarks of the Microsoft Cooperation
Performance
Flexibility
Simplicity
ANNS
Originality
Figure 1: The correlation of the design characteristics NeuralNet 1 Backpropagation 1.1
1.2
3
2 SOM 2.1
1.3
BP/Momentum ResilientProp Quickprop
RecurrentNets 2.3
2.2
CounterProp ART Hopfield 2.2.1
2.3.1
Boltzmann
ART1
3.1
3.2
Jordan
Elman
2.3.2 ART2
Figure 2: Type hierarchy mapping [Sch96] E. Schikuta. Neudb'95: An sql based neural network environment. In Shun-ichimeri et al., editors, Progress in Neural Information Processing, Proc. Int. Conf. on Neural
Information Processing, ICONIP'96, pages 1033{1038, Hong Kong, September 1996. Springer-Verlag, Singapore.
Figure 3: The entity-relationship neural network model
Figure 4: NeuroAccess screens
