Parallel computations and committee constructions

c Pleiades Publishing, Ltd., 2007. ISSN 0005-1179, Automation and Remote Control, 2007, Vol. 68, No. 5, pp. 912–921.  c V.D. Mazurov, M.Yu. Khachai, 2007, published in Avtomatika i Telemekhanika, 2007, No. 5, pp. 182–192. Original Russian Text 

TOPICAL ISSUE

Parallel Computations and Committee Constructions1 V. D. Mazurov∗ and M. Yu. Khachai∗∗ Ural State University, Yekaterinburg, Russia and Institute of Mathematics and Mechanics, Ural Branch, Russian Academy of Sciences, Yekaterinburg, Russia Received December 18, 2006

Abstract—The paper reviewed the results bearing out the deep-seated relation between the parallel computations and learning procedures for the laminated neural networks one of whose formalizations is represented by the theory of committee constructions. Additionally, consideration was given to two combinatorial problems concerned with learning pattern recognition in the class of affine committees—the problem of verifying existence of a three-element affine separating committee and that of element-minimal affine separating committee. The first problem was shown to be N P -complete, whereas the second problem is N P -hard and does not belong to the Apx class. PACS numbers: 02.10.Ox, 02.60.-x, 89.20.Ff DOI: 10.1134/S0005117907050165

1. INTRODUCTION The laminated neural networks (perceptrons) are among of the most popular mathematical models used to develop algorithms for learning pattern recognition. At that, the learning algorithms for such networks, being parallel by nature, usually do not require parallelization. Nevertheless, despite the intrinsic parallelism of these learning algorithms, the problems of optimization concerned with perception learning often are not free of the difficulties inherent to many combinatorial problems such as high computational complexity, weak approximability, and so on. The present paper can be conventionally divided into two parts. The first part is conceptual and reveals the historical relationship between the parallel computations and the theory of committee decisions describing, in particular, the perceptron learning procedure. The second part is devoted to the new results concerning estimation of the computational and approximative complexity of several combinatorial problems concerned with learning recognition in the class of affine committee classifiers. 2. COMMITTEE APPROACH TO PARALLELIZATION OF THE RECOGNITION LEARNING PROCEDURE The idea of parallelizing the computations of the object characteristics and using the local predicates to identify one or another global properties of objects was used in the pattern recognition from the very origin of this discipline, that is, from the 1950’s. Even earlier Russell and Whitehead posed the problem of generating general notions, universals, on the basis of generalization of the local data. McCulloch and Pitts used the linear threshold functions to realize this idea. Functions of this kind were later christened the neurons. The program of constructing global solutions through 1

This work was supported by the Russian Foundation for Basic Research, projects nos. 07-07-00168 and 07-01399, and the Council for Grants at the President of the Russian Federation, projects nos. NSH-5595.2006.1 and MD-6768.2006.1. 912

PARALLEL COMPUTATIONS AND COMMITTEE CONSTRUCTIONS

913

solution of subproblems was declared in the 1950’s by Rosenblatt when considering perceptrons, a special case of networks with three layers of linear threshold elements and learning in one layer. However, it was soon proved that Rosenblatt’s perceptron cannot extrapolate the precedence data, which gave rise to the need for a more general theory of parallel computing devices enabling precise estimation of the solvability of the problem of data analysis by means of the local predicates. The studies on the committee theory [1, 2] made it possible to prove the necessary theorems. The point is that the committee is a three-layer network with learning in all layers. The problem becomes nonlinear, but methods of its solution were determined. This is especially important because the case in point is parallelization of solutions not only on the completely definite, but also on the nonformalized problems. The neural networks work out solutions by establishing the membership of an object to the given class and summing up the data about processing of the local segments of the data array. If X is the input data array, then the solution f (X) of the corresponding problem Z(X) can be established in two stages: first, the functions f1 (X), . . . , fq (X) (actually they are the functions of part of the array X, the local functions) are calculated, and then the resulting function f (X) = g(f1 (X), . . . , fq (X)) is constructed by adjusting the coefficients of all functions. If at that the functions g and fi are linear threshold elements, then f is the majority committee. This is only one of the possible interpretations of the committee decisions. There are other interpretations concerned with the principle of parallelization of the processes of data and knowledge processing. Committee is just one of the models of the fuzzy set. It is also a generalization of the solution of the incompatible problem, a solution not concentrated at a point; it is a set of points having each its own weight of presence in this set. The study of incompatible systems of linear inequalities by means of the committee constructions which give one of the directions of the notion of decision is important both for the theory of linear inequalities and the applied problems. By the committee of a system of linear inequalities over a linear space is meant the finite set of elements of this space having the following property: the majority of its elements satisfies each system inequality. The committee constructions make up a class of generalizations of the notion of solution for the problems of mathematical programming and pattern recognition which can be either compatible or incompatible. This is a class of discrete approximations for contradictory problems which can also be correlated with the fuzzy solutions. The method of committees defines today one of the lines of analytical research and solution of the problems of effective choice of variants for optimization, diagnosis, and classification. We present by way of example the definition of one of the basic committee constructions: by the p-committee of the system of inclusions for the given number 0 < p < 1 is meant the set of elements such that more than the pth part of its elements satisfies each inclusion. The committee constructions may be also regarded both as a class of generalizations of the notion of solution to the case of incompatible systems of equations, inequalities, and inclusions and as a means for parallelization in the solution of the problems of choice, diagnosis, and forecasting. The committee constructions as a generalization of the notion of problem solution are the sets of elements having some (usually, not all) properties of the solution, they are fuzzy solutions of a kind. For example, the committee of a constraint system is a set of elements such that more than half of its elements satisfies each constraint. The committee construction as a parallelization means appear explicitly in the multilayer neural networks. It was shown in [1, 3] that the method of constructing the committee of some system of affine inequalities can be used to learn a neural network to precise solution of the classification problem. One may conclude from the aforementioned that the method of committees is concerned with one of the important lines of research and numerical analysis both of the diagnostic problems and AUTOMATION AND REMOTE CONTROL

Vol. 68

No. 5

2007

914

MAZUROV, KHACHAI

choice of variants and the problems of adjustment of the neural networks for making them respond as desired to the input information concerning one or another problem of the decision maker. After the experiments of F. Rosenblatt, N. Nilsson and some other American authors studied in the early 1960’s the associative machines and approached closely the notion of the committee of a system of linear inequalities. Finally, in 1965 C.M. Ablow and D.J. Kaylor [4] formulated explicitly the notion of the committee of a system of linear inequalities. After that, this subject was widely studied, but a purely mathematical theory with complete and strict proofs was developed only in Ekaterinburg at the Institute of Mathematics and Mechanics of the USSR (now Russian) Academy of Sciences. It deserves noting that reduction of the solution of a problem to the solutions of its subproblems may be traced by the example of incompatible problems having a transparent applied sense. This line of research may be traced back to the XVIII century. For example, already Legendre, Gauss, and later Laplace, who proposed and elaborated the method of least squares, dealt with the overdetermined and, consequently, usually contradictory systems of linear algebraic equations. Another side of the question of the committee constructions is concerned with the notion of coalitions at generating collective decisions. Here, the situations are sharply different in the case of collective preferences where numerous pitfalls exist and in the case of the rules of collective classification where the procedures have wider possibilities and can be strictly substantiated. Therefore, it is important to reduce the decision making problems to those of classification. Let us consider the coalitions in the problem of collective preference. Let X be the set of variants from which it is required to select a certain variant x using some criteria. Let a set C of experts or decision makers be responsible for this choice. In the case where the choice is based on preferences, each representative f of the set C is in fact a binary preference relation r(f ), which means that the assertion x r(f ) y may be valid for some x, y from X, which implies that “x is preferable to y for f .” The collective preference r = r(C) may be regarded as a function r = ϕ(r(f ) of individual preferences, f running the set C. It is only at first glance that this assumption seems natural, but namely it gives rise to further contradictions. It turned out that the collective preference cannot be a universal rule because it depends on the particular variants x, y, as well as on the preferences r(f ). Stated differently, the rule ϕ cannot be universal and must be local. Historically, in general three paths of research leading to the committee constructions can be identified. (1) From the generalization of the notion of solution: it started with the least-squares method followed by the works of Chebyshev on the approximate solutions of the systems of linear inequalities as applied to the theory of mechanisms, the works of Chernikov and Eremin on the theory Chebyshev approximations for the incompatible systems of linear inequalities, and finally the method of committees for such systems. (2) From the methods of neural network learning: Rosenblatt dealt with the perceptrons with learning in one layer, which enables solution of a narrow class of problems reducible to the linear separation of the finite sets; Nilsson already had heuristic methods of learning the neural networks in two layers, and finally the method of committees enabled one to obtain precise results and substantiated the learning procedures which enable one to solve a wide class of problems reducible to separation of finite sets with a single requirement on nonemptiness of their intersection. (3) The third path is concerned with the voting procedures. The situation in the area of voting is extremely complicated, and here one encounters paradoxes all around. It is well known that the paradoxes may be avoided if the solution of the problem of choice is reduced to a series of classification problems in which case the method of committees provides good results. The three-layer neural network corresponds to the method of committees, and it follows from the committee existence theorems that this network can learn from the precedents to solve any problem if this solution is expressible by a word in some finite alphabet. DecomposiAUTOMATION AND REMOTE CONTROL

Vol. 68

No. 5

2007

PARALLEL COMPUTATIONS AND COMMITTEE CONSTRUCTIONS

915

tion and parallelization of problems are important procedures that can be executed as some basic functions of the methods of recognition and the methods of neural systems. As can be seen from the definition of committee itself, one of the methods of neural system learning, the committee method, in essence is oriented to the parallelization of data and knowledge processing: if a problem may be incompatible, then the committee as a generalization of the notion of solution is a set of elements such that more than half of elements satisfies the problem conditions, that is, the majority of the committee members votes for satisfaction of each of the problem conditions [1]. At that, each member of the committee is responsible for its part of the problem at hand. Let us consider the possibility of decomposing a sufficiently wide class of problems, namely, the problems reducible to DA(A, B, F ), that is, to the following problem of discriminant analysis: it is desired to determine the function f from the functional class F separating the sets A and B so that f (x) > 0 for

x ∈ A,

f (x) < 0 for

x ∈ B,

(1)

which means that, for example, A is the set of the conditions of the problem Z for which the answer must be “Yes” and B is the set of conditions for which the answer is “No.” The class of problems Z reducible to (1) is very wide: these are the problems with parametrizable conditions where the answer is coded by a finite discrete sequence. In turn, system (1) comes to the linear inequalities in the general case where F is a linear space or convex polyhedral set in the linear space. The class Z of problems reducible to (1) becomes even wider if the requirement of search of the function f is relaxed and a separating committee is sought. By the separating committee is meant the set C = [f1 , . . . , fq ] such that each inequality of system (1) is satisfied by more than half of the functions from C. Therefore, the decomposition of the problem Z comes to the decomposition of the problem of discriminant analysis. If the volumes of the sets A and B are sufficiently large, then the successive decompositions A = A1 ∪ A2 , B = B1 ∪ B2 , . . . , Ai = Ai1 ∪ Ai2 , Bi = Bi1 ∪ Bi2 . . . may be required, and so on. 3. COMPUTATIONAL COMPLEXITY OF SOME COMBINATORIAL PROBLEMS CONCERNED WITH THE COMMITTEE RECOGNITION ALGORITHMS Since the late 1980’s the researchers became interested in the computational complexity of the problems of learning the neural network which is optimal in one or another criterion. Of special interest are the results concerning the estimates of computational complexity of learning the simplest networks—the classical perceptrons which represent a two-layer network without hidden layers and having q input and one output neuron. The activation function of the ith neuron has the classical form 

f i (a) =

1, (β i , a) + γ i > 0 −1, otherwise.

Therefore, the perceptron realizes the decision rule F (z | (β 1 , γ 1 ), . . . , (β q+1 , γ q+1 )) : Qn → {−1, 1}. By assuming the sample (a1 , a2 , . . . , am1 , b1 , b2 , . . . , bm2 ) with ai ∈ A and bj ∈ B, and the finite sets A, B ⊂ Qn consisting of the representatives of, respectively, the first and second classes, one can pose the problem of perceptron learning, that is, selection of the parameters β and γ such that F (ai ) = 1 (i ∈ {1, 2, . . . , m1 } = Nm1 ), F (bj ) = −1 (j ∈ Nm2 ). The “learnt” perceptron whose parameters are adjusted by successful learning is usually referred to as correct. The formulations of two combinatorial problems are related with the learning procedure. AUTOMATION AND REMOTE CONTROL

Vol. 68

No. 5

2007

916

MAZUROV, KHACHAI

Problem 1. “EDUCABILITY” ([5]). Given are the natural number q and the sample (a1 , a2 , . . . , am1 , b1 , b2 , . . . , bm2 ). Is there a correct perceptron with at most q input neurons? Problem 2. “OPTIMAL CORRECT PERCEPTRON” ([6]). Given is the sample (a1 , a2 , . . . , am1 , b1 , b2 , . . . , bm2 ). Needed is to determine the parameters of the correct perceptron with the least possible q. It is common knowledge [7] that the problem of EDUCABILITY in the general case is N P complete and remains such for q = 2, whereas for q = 1 it is polynomially solvable, and the problem of OPTIMAL CORRECT PERCEPTRON is N P -hard [6]. The proof of intractability of both problems was obtained in its time as a corollary to the theorem of N P -completeness of the “Quadrant” problem [7]. Similar results also using these findings to substantiate the intractability of the problems of learning the neural networks of more involved architecture are known as well [6]. The present paper demonstrates that the problem of perceptron learning remains intractable even if its architecture is constrained rather strictly. Consideration is given to the problem of learning in the class of perceptrons with odd q and fixed parameters of the output neuron: β = [1, 1, . . . , 1]T and γ = 0, which corresponds to voting by the simple majority rule. The questions of learning such networks can be conveniently formulated in terms of the so-called affine separating committees and committee decisions of suitable systems of linear inequalities. Definition 1 ([8]). By the affine separating committee for the sets A, B ⊂ Qn is meant the finite sequence Q = (f 1 , . . . , f q ) of the functions f i (z) = sgn((β i , z) + γ i ) such that q 

f i (a)  1

(a ∈ A),

i=1 q 

f i (b)  −1 (b ∈ B).

i=1

The notion of the affine separating committee is closely related with that of the committee solution of the system of linear inequalities. Definition 2 ([4]). The sequence Q = (x1 , . . . , xq ), xi ∈ Qn , is called the committee solution of the inequality system (aj , x) > bj if the condition

(i ∈ Nm ),

  q   {i ∈ Nq | (aj , xi ) > bj } >

2

(2)

(j ∈ Nm )

is valid. The sequence Q = (f 1 , . . . , f q ) is the affine separating committee of the sets A and B if and only if the sequence Q = ((β 1 , γ 1 ), . . . , (β q , γ q )) is the committee solution of the inequality system 

(β, a) + γ > 0 (a ∈ A) (β, b) + γ < 0 (b ∈ B)

and defines the weights of the input layer of the corresponding correct perceptron. The problem of MCLE search of the committee solution of system (2) with the least possible number of elements (minimal committee) is known to be N P -hard [9]. It is shown below that —the problem of verifying existence of the three-element affine separating committee (3-ASC) for the given sets A and B is N P -complete; AUTOMATION AND REMOTE CONTROL

Vol. 68

No. 5

2007

PARALLEL COMPUTATIONS AND COMMITTEE CONSTRUCTIONS

917

Example of reducing the problem of 3-COLORABILITY to the problem of 3-ASC.

—the problem of constructing the element-minimal affine separating committee (MASC) for the sets A and B is N P -hard and does not belong to the Apx class. We consider the following combinatorial problem. Problem 3. “THREE-ELEMENT AFFINE SEPARATING COMMITTEE (3-ASC).” Given are the sets A, B ⊂ Qn , A = {a1 , . . . , am1 } and B = {b1 , . . . , bm2 ). Is there a three-element affine separating committee for these sets? Theorem 1. The problem 3-ASC is N P -complete. The proof is based on the technique of polynomial reducibility of the well-known N P -complete problem of graph coloring in three colors to the 3-ASC problem. Problem 4. “GRAPH COLORING IN THREE COLORS (3-COLORABILITY).” Given is the finite graph G = (V, E). Is it colorable in three colors or, stated differently, is there a function ϕ : V → {1, 2, 3} such that ⇒ (ϕ(u) = ϕ(v)) for arbitrary u, v ∈ V , ({u, v} ∈ E)? The sets of points A = A(G) and B = B(G) were shown to be assignable to the finite graph G in a polynomial (depending on its notation) time so that the graph can be colored in three colors if and only if there exists a three-element separating committee for the constructed sets. Example 1. We assign to the graph G = (V, E), V = {1, 2, 3}, E = {{1, 2}, {1, 3}, {2, 3}} (see the figure) the subsets A, B ⊂ Q3 A = {[2, 0, 0], [0, 2, 0], [0, 0, 2]}, B = {[1, 1, 0], [1, 0, 1], [0, 1, 1]}.

(3)

The sets (3) are separable by the committee (f 1 , f 2 , f 3 ), where f 1 (x) = −2x1 + x2 + x3 , f 2 (x) = x1 − 2x2 + x3 , f 3 (x) = x1 + x2 − 2x3 , which induces coloring of G in three colors. Remark 1. One can readily make sure that the 3-ASC problem remains N P -complete if one confines consideration to the sets A ∪ B ⊂ {z ∈ {0, 1, 2}n : |z|  2}. Now we consider the optimization formulation of the problem of learning in the class of affine separating committees. AUTOMATION AND REMOTE CONTROL

Vol. 68

No. 5

2007

918

MAZUROV, KHACHAI

Problem 5. “ELEMENT-MINIMAL AFFINE SEPARATING COMMITTEE (MASC).” Given are the sets A, B ⊂ Qn , A = {a1 , . . . , am1 }, B = {b1 , . . . , bm2 ), and A ∩ B = ∅. Needed is to construct an affine separating committee for the sets A and B with the least number of elements. Theorem 2. The MASC problem is N P -hard. Proof. Validity of the theorem follows from Theorem 1 because the polynomial (Turing) reducedness of the 3-ASC problem to the MASC can be easily verified. The traditional approach to the N P -hard problems includes consideration of the polynomially solvable subclasses of the N P -hard problem, analysis of its approximation properties, and development of approximate algorithms. As usual, by the approximate algorithm (with approximation precision r) for the problem of combinatorial minimization is meant the algorithm enabling one to determine in a polynomial time the admissible solution xapp ∈ M for each particular formulation f ∗ = min{f (x) | x ∈ M } with the condition f (xapp )  r. f∗ The Apx class consists of the problems of combinatorial optimization having an approximate algorithm of a fixed accuracy r. This class comprises many N P -hard problem such as the travelling salesman problem. Unfortunately, examples of problems that do not belong to this class also are known. It seems that the best-known of such problems is that of the greatest clique for which J. Hastad proved that no polynomial approximate algorithm with the approximation precision n1−ε exists for an arbitrary ε > 0 [10]. We make sure that the above MASC problem also cannot be solved with any fixed accuracy. Theorem 3. The MASC problem does not belong to the Apx class. The proof substantiates the Turing-reducedness to the MASC problem of the problem of coloring the uniform two-color graph. Problem 6. “COLORING OF THE 3-UNIFORM 2-COLOR HYPERGRAPH IN k COLORS (3-UHC).” Given are the finite uniform hypergraph Γ = (V, H), |h| = 3, h ∈ H and the natural number k  3. Γ is known to be colorable into two colors. It is required to indicate the coloring of Γ into k colors, that is, a function ϕ : V → Nk such that ({u, v, w} ∈ H) ⇒ (|{ϕ(u), ϕ(v), ϕ(w)}| > 1). The 3-UHC problem is known [11] to be N P -hard and remains such for an arbitrary fixed k  3. We note that, generally speaking, the most precise of the existing approximate algorithms for the MASC problem [12] has precision O(m), although a nontrivial subclass of this problem exists where the algorithm is precise. 4. CONCLUSIONS The results obtained indicate that in the general case the problem of learning optimal recognition in the class of affine separating committees is intractable. Yet the question of estimating the intractability for special cases of such problems under some additional constraints still remains open. We mean, for example, estimation of the computational complexity of these problems under fixed dimensionality of the original space and some other cases that are important for applications. The present authors expect that these questions will be answered in the near future. AUTOMATION AND REMOTE CONTROL

Vol. 68

No. 5

2007

PARALLEL COMPUTATIONS AND COMMITTEE CONSTRUCTIONS

919

APPENDIX Proof of Theorem 1. (1) Obviously, the 3-ASC problem belongs to the N P class because verification of the given function sequence Q = (f 1 , f 2 , f 3 ) for being an affine separating committee of the sets A and B can be carried out in a time depending polynomially on the size of the problem notation. (2) To substantiate the N P -completeness, we prove the polynomial reducibility of the 3-COLORABILITY problem to the 3-ASC problem. Indeed, let the graph G = (V, E) be given defining the condition of the 3-COLORABILITY solution. We assume without loss of generality that V = Nn , assign to the graph G the sets A and B in Qn as follows: 

A=

{2ei }ni=1 , i

where

eij

= δij =

1, i = j 0, i =  j,

(A.1)

j

B = {e + e | {i, j} ∈ E} and demonstrate that the graph is colorable in three colors if and only if for the sets A and B there exists a three-element affine separating committee, that is, there will be pairs (x1 , y 1 ), (x2 , y 2 ) and (x3 , y 3 ) such that the sequence Q = ((x1 , y 1 ), (x2 , y 2 ), (x3 , y 3 )) is a committee solution of the inequality system 

·

2xi + y < 0 (i ∈ V ) xi + xj + y > 0 ({i, j} ∈ E).

(A.2)

·

Let the decomposition V1 ∪ V2 ∪ V3 define the coloring of the graph G in three colors. It is easy to verify that the sequence ((x1 , 0), (x2 , 0), (x3 , 0)) where 

xik

=

2, k ∈ Vi −1, k ∈  Vi

(i ∈ N3 ),

is the committee solution of system (A.2). On the other hand, let Q = ((x1 , y 1 ), (x2 , y 2 ), (x3 , y 3 )) be an arbitrary committee solution of system (A.2). We denote Vk = {i ∈ V | 2xpi + y p < 0 (p ∈ N3 \ {k})}

(k ∈ N3 ).

Since Q is the committee solution of system (A.2), the equality V1 ∪ V2 ∪ V3 = V is true. Without loss of generality we may assume that Vk = ∅ and Vk1 ∩Vk2 = ∅ for arbitrary k and k1 = k2 from N3 . The constructed decomposition defines the desired coloring of the graph G. Indeed, let us assume from the contrary that the edge {i, j} ⊂ V1 (the cases of V2 and V3 can be considered by analogy). The inequalities 2x2i + y 2 < 0,

2x3i + y 3 < 0,

2x2j + y 2 < 0,

2x3j + y 3 < 0

are valid by construction of the set V1 . Consequently, also x2i + x2j + y 2 < 0 and

x3i + x3j + y 3 < 0,

whereas with necessity at least one of the inequalities x2i + x2j + y 2 > 0

or

x3i + x3j + y 3 > 0

is true by virtue of the fact that Q is the committee solution of system (A.2). The resulting contradiction proves the theorem. AUTOMATION AND REMOTE CONTROL

Vol. 68

No. 5

2007

920

MAZUROV, KHACHAI 



2s + 1 Proof of Theorem 3. We demonstrate that for k = for an arbitrary natural s the 3s+1 UHC problem is Turing-reducible to the problem of determining the affine separating committee of 2s + 1 elements of the suitable sets. Indeed, let the uniform hypergraph Γ = (V, H) be given where for each edge h ∈ H V = Nn , H = ∅, and |h| = 3 and the number s ∈ N. Let the decomposition   · 2s + 1 V1 ∪ V2 = V define the coloring of Γ in two colors. It is required to find a coloring of Γ in s+1 colors. Similar to the proof of Theorem 1, we assign to the hypergraph Γ the subsets A, B ⊂ Qn such that A = {3ei }ni=1 ,

where eij = δij ,

B = {ei + ej + ek | {i, j, k} ∈ H} and the system of inequalities 

3xi + y < 0 (i ∈ V ) xi + xj + xk + y > 0 ({i, j, k} ∈ H).

(A.3)

Obviously, these constructions can be carried out in a polynomial time vs. the size of the notation of Γ. We ascertain validity of these assumptions. (a) System (A.3) is incompatible and has a three-element committee solution.   (b) The arbitrary committee solution Q = (x1 , y 1 ), . . . , (x2s+1 , y 2s+1 ) of system (A.3) induces  2s + 1 colors. the coloring of the hypergraph Γ in s+1 (a) Since H = ∅, system (A.3) is incompatible by the Carver theorem. Now, it is easy to see that the sequence ((x1 , 0), (x2 , 0), (x3 , 0)) where 

x1i

=

−1, i ∈ V1 3, i ∈ V2 ,



x2i

=

3, i ∈ V1

and

−1, i ∈ V2 ,

x3 = [−1, −1, . . . , −1]T ,

is its committee solution. Indeed, (x1 , 0) and (x3 , 0) satisfy the inequality 3xi + y < 0 under an arbitrary i ∈ V1 (the case of V2 can be considered by analogy), and (x1 , 0) and (x2 , 0) satisfy an arbitrary inequality xi + xj + xk + y > 0.   (b) Let Q = (x1 , y 1 ), . . . , (x2s+1 , y 2s+1 ) be the committee solution of system (A.3). To each subset P ⊂ N2s+1 , |P | = s + 1, we assign the set VP = {i ∈ V | 3xpi + y p < 0 (p ∈ P )} 



2s + 1 and and denote by P the set {P ⊂ N2s+1 : |P | = 2s + 1}. By construction, |P| = s+1 VP = V . Without loss of generality we assume that VP = ∅ for each P ∈ P and (P1 = P2 ) ⇒ P ∈P

(VP1 ∩ VP2 = ∅). The decomposition ·

·

·

VP1 ∪ VP2 ∪ . . . ∪ VP

2s+1

 =V

s+1

AUTOMATION AND REMOTE CONTROL

Vol. 68

No. 5

2007

PARALLEL COMPUTATIONS AND COMMITTEE CONSTRUCTIONS

921

defines the desired coloring of the hypergraph Γ. Indeed, let us assume that on the contrary the edge {i, j, k} ⊂ VP for some P ∈ P. The inequalities ⎧ p p ⎪ ⎨ 3xi + y < 0 ⎪ ⎩

3xpj + y p < 0 3xpk + y p < 0

(p ∈ P )

are valid by the choice of the edge. Consequently, xpi + xpj + xpk + y p < 0

(p ∈ P ).

On the other hand, since Q is the committee decision of system (A.3), there will be a number p0 ∈ P such that with necessity xpi 0 +xpj 0 +xpk0 +y p0 > 0. This contradiction corroborates correctness of the coloring. Let us assume that on the contrary the MASC problem belongs to the Apx class and there exists a polynomial approximate algorithm with a fixed precision estimate r. Then, with regard for (a) the algorithm constructs a (2s + 1)-element committee solution of system (A.3) corresponding to the Γ, where 2s + 1  3r, thus giving (according to (b)) the hypergraph coloring in  hypergraph  2s + 1 colors. This contradiction proves the theorem. s+1 REFERENCES 1. Mazurov, Vl.D., On the Committee of a System of Convex Inequalities, in Proc. Int. Math. Congr., Moscow: Mosk. Gos. Univ., 1966, no. 14, p. 41. 2. Mazurov, Vl.D., Metod komitetov v zadachakh optimizatsii i klassifikatsii (Method of Committees in the Problems of Optimization and Classification), Moscow: Nauka, 1990. 3. Mazurov, Vl.D., Consistent Completion of Systems of Algorithms to Committee Technologies, Pattern Recognit. Image Anal., 1998, vol. 8, no. 4, pp. 501–506. 4. Ablow, C.M. and Kaylor, D.J., Inconsistent Homogeneous Linear Inequalities, Bull. Am. Math. Soc., 1965, vol. 71, no. 5, p. 724. 5. Judd, J.S., Neural Network Design and Complexity of Learning, New York: MIT Press, 1990. 6. Lin, J.H. and Vitter, J.S., Complexity Results on Learning by Neural Nets, Machine Learning, 1991, vol. 6, pp. 211–230. 7. Blum, A.L. and Rivest, R.L., Training a 3-node Neural Network is NP-complete, Neural Networks, 1992, vol. 5, pp. 117–127. 8. Mazurov, Vl.D., Committees of Inequality Systems and the Problem of Recognition, Kibernetika, 1971, no. 3, pp. 140–146. 9. Khachai, M.Yu., On Computational Complexity of the Minimal Committee and Related Problems, Dokl. Ross. Akad. Nauk , 2006, vol. 406, no. 6, pp. 742–745. 10. Hastad, J., Clique is Hard to Approximate within n1−ε , Acta Mathematica, 1999, vol. 182, no. 13, pp. 105–142. 11. Dinur, I., Regev, O., and Smyth, C., The Hardness of 3-uniform Hypergraph Coloring, in Proc. 43rd Ann. IEEE Sympos. Foundat. Comput. Sci., 2002. 12. Khachai, M.Yu., On the Computational and Approximational Complexity of the Problem of Minimal Separating Committee, Tavrich. Vest. Informatiki i Mat., 2006, no. 1, pp. 34–43.

This paper was recommended for publication by A.I. Kibzun, a member of the Editorial Board

AUTOMATION AND REMOTE CONTROL

Vol. 68

No. 5

2007