Feature Selection: Near Set Approach James F. Peters1 , Sheela Ramanna2? 1
Department of Electrical and Computer Engineering, University of Manitoba Winnipeg, Manitoba R3T 5V6 Canada
[email protected] 2 Department of Applied Computer Science, University of Winnipeg, Winnipeg, Manitoba R3B 2E9 Canada
[email protected]
Abstract. The problem considered in this paper is the description of objects that are, in some sense, qualitatively near each other and the selection of features useful in classifying near objects. The term qualitatively near is used here to mean closeness of descriptions or distinctive characteristics of objects. The solution to this twofold problem is inspired by the work of Zdzislaw Pawlak during the early 1980s on the classification of objects. In working toward a solution of the problem of the classification of perceptual objects, this article introduces a near set approach to feature selection. Consideration of the nearness of objects has recently led to the introduction of what are known as near sets, an optimist’s view of the approximation of sets of objects that are more or less near each other. Near set theory started with the introduction of collections of partitions (families of neighbourhoods), which provide a basis for a feature selection method based on the information content of the partitions of a set of sample objects. A byproduct of the proposed approach is a feature filtering method that eliminates features that are less useful in the classification of objects. This contribution of this article is the introduction of a near set approach to feature selection. Keywords: Description, entropy, feature selection, filter, information content, nearness, near set, perception, probe function.
1
Introduction
The problem considered in this paper is the classification of perceptual objects that are qualitatively but not necessarily spatially near each other. The term qualitatively near is used here to mean closeness of descriptions or distinctive characteristics of objects. The solution to this problem is inspired by the work ?
We gratefully acknowledge the very helpful comments, insights and corrections concerning this paper by Andrzej Skowron and the anonymous reviewers. This research has been supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) grants 185986 and 194376.
of Zdzislaw Pawlak during the early 1980s on the classification of objects [12], elaborated in [13, 17], and a view of perception that is on the level of classes instead of individual objects [10]. In working toward a solution of the problem of the classification of perceptual objects, this article introduces a nearness description principle. An object description is defined by means of a vector of probe function values associated with an object (see, e.g., [11]). Each probe function φi represents a feature of an object of interest. Sample objects are near each other if, and only if the objects have similar descriptions. Ultimately, there is interest in selecting the probe functions [11] that lead to descriptions of objects that are minimally near each other. This is an essential idea in the near set approach [7, 14, 16, 19] and differs markedly from the minimum description length (MDL) proposed in 1983 by Jorma Rissanen [23]. MDL depends on the identification of possible data models and possible probability models. By contrast, NDP deals with a set X that is the domain of a description used to identify similar objects. The term similar is used here to denote the presence of objects that have descriptions that match each other to some degree. The near set approach leads to partitions of ensembles of sample objects with measurable information content and an approach to feature selection. The proposed feature selection method considers combinations of n probe functions taken r at a time in searching for those combinations of probe functions that lead to partitions of a set of objects that has the highest information content. It is Shannon’s measure of the information content [8, 26] of an outcome that provides a basis for the proposed feature selection method. In this work, feature selection results from a filtering method that eliminates those features that have little chance to be useful in the analysis of sample data. The proposed approach does not depend on the joint probability of finding a feature value for an input vectors that belong to the same class as in [6]. In addition, the proposed approach to measuring the information content of families of neighbourhoods differs from the rough set-based form of entropy in [25]. Unlike the dominance-relation rough set approach [5], the near set approach does not depend on preferential ordering of value sets of functions representing object features. The contribution of this article is the introduction of a near set approach to feature selection. This article has the following organization. A brief introduction to the notation and basic approach to object description is given in Sect. 2. A brief introduction to nearness approximation spaces is given in Sect. 4. A nearness description principle is introduced in Sect. 3. A near set-based feature selection method is introduced in Sect. 5.
2
Object Description
Objects are known by their descriptions. An object description is defined by means of a tuple of function values φ(x) associated with an object x ∈ X (see (1)). The important thing to notice is the choice of functions φi ∈ B used to describe an object of interest. Object Description : φ(x) = (φ1 (x), φ2 (x), . . . , φi (x), . . . , φL (x)).
(1)
Table 1. Description Symbols
Symbol Interpretation < O X x F B φ L i φi φ(x)
Set of real numbers, Set of perceptual objects, X ⊆ O, set of sample objects, x ∈ O, sample object, A set of functions representing object features, B ⊆ F, φ : O →
