c Allerton Press, Inc., 2013. ISSN 8756-6990, Optoelectronics, Instrumentation and Data Processing, 2013, Vol. 49, No. 6, pp. 569–577. c I.A. Hodashinsky, I.V. Gorbunov, 2013, published in Avtometriya, 2013, Vol. 49, No. 6, pp. 51–61. Original Russian Text
AUTOMATION SYSTEMS IN SCIENTIFIC RESEARCH AND INDUSTRY
Algorithms of the Tradeoff between Accuracy and Complexity in the Design of Fuzzy Approximators I. A. Hodashinsky and I. V. Gorbunov Tomsk State University of Control Systems and Radioelectronics, pr. Lenina 40, Tomsk, 634050 Russia E-mail:
[email protected] Received March 6, 2013 Abstract—Two important stages in design of fuzzy approximators, including structure generation and parameter optimization, are considered. Two optimization criteria, i.e., the accuracy measured by the root-mean-square error and the complexity expressed as the number of fuzzy rules, are proposed. The results of studies of the approximators obtained on real data from the KEEL repository are given, and the results are compared with their analogs. Keywords: fuzzy approximator, structure generation, parameter optimization, metaheuristics. DOI: 10.3103/S875669901306006X
INTRODUCTION The problem of estimating an unknown function based on the table of observations is one of the key issues in the field of modeling of fuzzy systems (FS) and the theory of approximation of functions. The popularity and practicality of FS are due to the following reasons: (1) FS can be identified by combining the observed data and expert knowledge; (2) the nature of fuzzy rules allows describing the behavior of modeled systems in terms of cause-effect relationships; (3) FS are universal approximators capable of representing any continuous nonlinear function with any degree of accuracy [1]. In addition, any FS designed on the basis of real data is given two basic requirements: (1) the system should accurately reproduce information from the analyzed table of observations and have high generalization capabilities; (2) the system should be presented in a format understandable to the user and help identify the most significant dependences and relations between the input and output variables, i.e., fuzzy rules can be interpreted by the user in the context of this application. Interpretability can be explained as an opportunity to describe the behavior of the modeled system in an understandable form. This feature is desirable for all types of applications, but it is particularly important for the knowledge-based systems with a man-machine interaction, such as decision support systems. The knowledge bases of such systems should be understandable for users to increase confidence in systems, their advice, and suggestions. Specialists in fuzzy modeling estimate interpretability through either complexity or semantics [2–5]. Semantics implies giving the functions the sense of membership, when each fuzzy term can obtain a meaningful linguistic meaning, such as “very small,” “small,” “average,” “large,” and “very large.” Complexity is expressed by the number of rules, variables, and fuzzy terms. Note that a simple FS is easier to configure and requires less memory and output time than a more complex FS. Both of these criteria (accuracy and interpretability) are contradictory and should be considered in FS design. FS design is based on multicriteria optimization aimed at finding solutions that simultaneously optimize more than one objective function. In this case, we seek for a tradeoff decision, which is not dominated by 569
570
HODASHINSKY, GORBUNOV
any other decision or is optimal in the Pareto sense. After constructing the area (front), the decision-maker Pareto-face can select only one of the Pareto-optimal solutions. There are many publications on design of two-criteria (accuracy–complexity) fuzzy classifiers, which solve the problem of finding a tradeoff between the two criteria, but we refer here only to [2, 3]. There are not too many works on design of two-criteria fuzzy approximators (FA), and they are based on the approach to multi-criteria optimization based on evolutionary algorithms [1, 4–6]. In this paper, the tradeoff between accuracy and complexity is proposed to be sough in two stages: first, the approximator structure of given complexity is generated, then its parameters are optimized by swarm algorithms. As complexity of our work takes integer values, by restricting its value from above, complexity can be specified as an input parameter of the optimized system that allows us to solve the problem posed as single-purpose optimization. The objective of this paper is to describe the methods and means of tradeoffs between accuracy and complexity in design of fuzzy approximators. FORMULATION OF THE PROBLEM Accuracy is the ability to represent a real system adequately. The commonly accepted measure of accuracy is the root-mean-square error. Complexity is a subjective property associated with such factors as the model structure, the number of input variables, the number of fuzzy rules, the number and form of fuzzy linguistic variables, etc. There is no universal method for measuring complexity of models [2, 3, 7]. In this paper, complexity is defined as the number of fuzzy rules. Furthermore, the FS is subject to the following restrictions: (1) the number of terms for each input variable is within a reasonable range (2 to 9); (2) the membership functions (MF) of fuzzy terms are convex and normalized; (3) the definition domain is completely covered; (4) the membership functions are distinguishable, i.e., two MFs do not take two close values in the definition domain; (5) globally defined MFs are used; (6) there are no rules with similar antecedents but different consequents in the database. Because accuracy and complexity are conflicting criteria, a set of fuzzy systems rather than one FS is generated. A set of nondominated solutions denoted as a Pareto set, whose characteristic property is the optimal relation between the criteria of accuracy and complexity, is chosen from the generated set of systems. A fuzzy approximator is given by the rules IF x1 = A1i AND x2 = A2i AND . . . AND xn = Ani THEN y = ri ,
(1)
where Aji is the linguistic term, which is used to estimate the input variable xj , and ri is a real number, which is used to estimate the output variable y. The approximator performs the mapping f :
