Performance enhancement of object shape classification by coupling ...

Performance Enhancement of Object Shape Classification by Coupling Tactile Sensing with EEG Monalisa Pal1,a, Anwesha Khasnobish2,b, Amit Konar1,c, D. N. Tibarewala2,d, R. Janarthanan3,e

Dept. of Electronics &Telecommunication Engg., 2School of Bioscience & Engg., 3Dept. of Computer Science. 1,2 Jadavpur University, 3TJS Engineering College. 1,2 Kolkata, 3Chennai, India. a [email protected], [email protected], [email protected], [email protected], e [email protected]

1

Abstract—In this work we establish the fact that using Electroencephalogram (EEG) with tactile signal during dynamic exploration accomplishes object shape recognition better than using the either alone. Adaptive auto-regressive coefficients and Hjorth parameters are used as features which are classified using linear Support Vector Machine, Naïve Bayes, k-nearest neighbor and tree classifiers. Following this, the space complexity to store the high-dimensional tactile features is identified. ReliefF algorithm is used as a dimension reduction technique. A polynomial of order 6 is used to fit an EEG feature to a corresponding tactile feature. These pre-fitted polynomials are used to predict the EEG features in situation where EEG measuring device is not present. Finally we note that using these predicted features along with the tactile features yields enhanced classification accuracies. Keywords—Adaptive Auto-regressive coefficients; electroencephalography; Hjorth parameters; k-nearest neighbor; linear Support Vector Machine; Naïve Bayes; object-shape perception; polynomial fitting; ReliefF; tactile signals; tree.

I.

INTRODUCTION

Touch sensation allows us to manoeuvre the world around us while exploring the same. The attributes that can be identified from touch perception includes object shape, size, surface texture, softness, etc. Consequently, tactile sensing is being applied to the fields of rehabilitation, which aims at constructing artificial hand with tactile sensors, and Human Computer Interaction (HCI) which addresses applications like robotic surgery, tele-operation, virtual gaming, etc. Touch perception can be decoded from brain as well as tactile signals acquired while exploring the objects using the sensor system. Researchers have classified brain responses for different emotion recognition [1-2], motor imagination [3], visual perception [4], haptic perception [5], etc. using EEG as the brain signal measure. Research works also show object shape recognition from tactile images using neural networks [6-7], regional descriptors [8], image gradient [9], etc. Reconstruction of 2-D shapes has also been achieved using Markov models [10]. This work brings together both the approaches and shows that classification using information from both the sources yields better result than using the either sources separately. In the first half of the work, EEG and tactile signal is acquired while dynamically exploring seven objects of different

geometric shapes. Adaptive auto-regressive (AAR) coefficients and Hjorth parameters are used as features and the objects are classified using linear Support Vector Machine (SVM), Naïve Bayesian classifier, k-nearest neighbor (kNN) density estimation technique and tree classifier. Results clarify that using features from EEG and Tactile signals simultaneously produces better accuracy. The highdimensional tactile features are reduced using ReliefF algorithm. However, in applications like tele-operation or use of artificial hand, tactually generated EEG signals are unavailable. In such situations this work proposes polynomial fitting to predict EEG features corresponding to tactile features. Polynomial of order 2, 4, 6, 8 and 10 are fitted and over-fitting is noted as the order increases beyond 6. The predicted EEG features and the corresponding tactile features produce better results than with only tactile features. The results with predicted EEG feature are better than that with the actual dimension-reduced EEG feature. However, this is due to the reduction of the stochastic nature in the predicted EEG features than that in the original EEG features. Section II describes the flow of the work while briefly describing the different feature extraction, dimension reduction and classification algorithms. Results are discussed in section III. Finally, section IV concludes the paper while mentioning future scope of research in this direction. II.

METHODOLOGY

This section gives a brief overview of the different steps taken during the course of the experimentation. A. Data Aquisition EEG and Tactile signals are acquired from 20 subjects (10 male and 10 female) in the age group 25±3 years. The procedure of the data acquisition was explained to the participants and a consent form was duly signed by them. An audio stimulus having the three commands: relax (for 2 seconds), explore (for 10 seconds) and stop (for 1 second), was played for the subjects during which the subjects were blind-folded and were asked to dynamically explore the seven objects of different shapes in a specific order as shown in Fig. 1. Each object was explored ten times by each subject. Thus we have 20×10×7=1400 instances in our dataset.

D. Dimension Reduction, Polynomial Fitting and Feature Selection Tactile signals yield a very high dimensional feature space. This occupies a considerable amount of resources. ReliefF algorithm [17-18] is used as a dimension reduction technique. Here, the quality of a feature is decided based on whether that feature has similar value to the k nearest neighbor of the same class and different value from the k nearest neighbor of the other classes. The value of k is set to 10. Ten features of the highest quality are used. This converts all the feature vectors to 1×10 dimension.

Fig. 1. Objects shapes considered for the experiment: 1-Cone, 2-Cube, 3Cylinder, 4-Sphere, 5-Triangular Prism, 6-Hemisphere, 7-Hexagonal Cylinder

EEG signal is acquired using Emotiv headset [11] having a sampling rate of 128Hz from which six channels are considered viz. O1, O2, P7, P8, FC5 and FC6 which are placed following the standard 10/20 system of EEG electrode placement. On the other hand, the tactile signal is obtained from capacitive MEMS based pressure sensor: the PPS TactArray [12] which consists of a 32×32 grid of sensing elements and has a sampling rate of 10.8Hz. B. Pre-processing The acquired EEG signal has been filtered using a 6th order elliptical band-pass filter to obtain the desired signal in the frequency range 4-16Hz. To overcome the interference from the neighboring brain sources, common average referencing is executed where the average value over all the electrodes is subtracted from each of the electrodes. Thus, for EEG we have 6 time-series corresponding to each of the six channels. For tactile signals, the 32×32 grid of the tactile sensor produces 1024 time-series. C. Feature Extraction Adaptive auto-regressive (AAR) coefficients [13-14] and Hjorth parameters [15-16] are extracted as features from each of the previously mentioned time series. In this work, Kalman filtering is used for estimating the time-varying AAR coefficients for AAR model of order 6. The update coefficient is empirically considered as 0.0085 which permits only small changes in the coefficients with time. These AAR coefficients are averaged over all the time instants of an observation. From each of the time series, six values are obtained for AAR coefficients and three values for the Hjorth parameters corresponding to the triplet of activity, mobility and complexity. In each of the cases, features of all the time series are horizontally concatenated. Using AAR model, this yields a 1×36 feature vector from an EEG observation and 1×6114 feature vector from a tactile observation. Similarly, using Hjorth parameters, a feature vector from EEG signal is 1×18 and from tactile signal is 1×3072.

Tactile signals are less stochastic in nature than EEG signals. Using a tactile feature as the independent parameter, the corresponding EEG feature is fitted by polynomial fitting. Polynomial fitting is studied using 2, 4, 6, 8 and 10th order polynomial. From the root mean square error between predicted and available EEG signal, the number of features is selected to be 7 and the order of the polynomial is chosen to be 6. Thus, all the dataset are now of 1×7 dimension which are the final dimension reduced datasets. E. Classification Classification of eight dataset for both the features is done. These eight datasets consists of the original EEG dataset, the original tactile dataset, the original EEG and tactile dataset horizontally concatenated, the dimension reduced EEG dataset, the dimension reduced tactile dataset, the dimension reduced EEG and tactile dataset horizontally concatenated, the predicted EEG dataset, and the predicted EEG dataset and the dimension reduced tactile dataset horizontally concatenated. Four classifiers are used for this purpose viz. Linear SVM, Naïve Bayesian, kNN and Tree. The classification is done in one-against-one manner. Each of the datasets are divided into train-set, validation-set and test-set which are 70%, 15% and 15% non-overlapping portions of the datasets for training the classifiers, choosing the training parameters and analysing the classifier performance, respectively. For tuning the Support Vector Machine classifier two parameters viz. cost for penalizing training errors (C) and margin between two classes () are considered. C is varied from 40 to 200 in steps of 20 and best performance, after validation, is obtained with C as 100. For , experiments are started with  as 2 and its value is decreased in steps of 0.2 till  becomes 0.4. Best performance is noted when  is 1. During the Naïve Bayesian classification, the features are assumed to follow multivariate normal distribution and their mean and covariance are learned during the process of training. In k-nearest neighbor density estimation, a test sample is assigned a class which occurs most among its k nearest neighbors. The metric used for measuring the distance with the neighbors is Euclidean distance. For this work, k is considered to be 3. Tree classifier [19] creates a binary tree on the training set T with N samples and n classes where each node operates on a single feature yielding the smallest Gini’s diversity index [20] as given by (1) and splits the data into two sets T1 and T2 with

TABLE I.

N1 and N2 samples such that the condition in (2) is maintained. The term p(i) stands for the relative frequency of class i. n



gini T   1   p  i  

Features

i 1



N1 N2 gini T   gini T 1  gini T 2   N N

III.

Classifiers

Adaptive AutoRegressive Coefficients



2

CLASSIFICATION ACCURACIES WITH ORIGINAL DATASET

 Hjorth Parameters

RESULTS AND FINDINGS

Tactile

Linear SVM

55.4762

95.2381

95.4762

Naïve Bayes

54.7619

94.7619

95.0000

KNN

55.4762

89.5238

84.2857

Tree

51.8571

84.0476

86.9048

Linear SVM

51.6667

94.0476

94.5238

Naïve Bayes

56.1905

85.4762

86.4285

KNN

53.3333

76.9048

69.0476

Tree

55.0000

91.1905

93.5714

This section discusses the classification accuracies of the eight datasets and the errors obtained between the predicted and original EEG features. TABLE II. Features

Adaptive Auto-Regressive Coefficients 6

Root Mean Square Error

Dataset

6

Vb

T

V

c

F1,1

0.6054

0.6355

0.6094

0.8462

5.45e+03

5.70e+03

4.98e+03

7.82e+03

F2,2

1.6414

1.9188

1.6333

0.8462

0.1815

0.1988

0.1915

0.2682

F3,3

0.5934

1.7383

0.8333

1.7065

4.89e+03

5.46e+03

4.77e+03

9.34e+03

F4,4

2.0550

1.897

1.8675

9.2577

0.1894

0.2005

0.3186

7.7778

F5,5

0.5486

0.5926

0.6424

4.0985

0.2002

0.2216

0.3376

7.3602

F6,6

1.9133

1.8365

2.1629

1.8322

0.1864

0.1851

0.2721

6.9030

F7,7

3.3333

2.1379

2.6597

5.9265

0.1994

0.1816

0.2257

59.1048

F8,8

0.5775

0.7182

2.0511

3.4931

0.0365

0.0405

0.0380

3.6114

F9,9

1.6742

2.0685

1.7105

0.5831

0.0371

0.0409

0.0373

4.4921

F10,10

1.8403

2.8151

2.0237

17.7357

3.77e+03

6.89e+03

5.46e+03

5.40e+03

Training Dataset (T): 70% of Original Dataset,

T

8

Ta

TABLE III.

Hjorth Parameters

Hjorth Parameters

8

a.

Adaptive AutoRegresssive Coefficients

EEG+Tactile

ERROR BETWEEN ORIGINAL EEG AAR FEATURES AND PREDICTED EEG AAR FEATURES

Polynomial Order

Features

Original Dataset EEG

b.

V

T

Validation Dataset (V): 30% of Original Dataset,

c.

V

EEG feature 1 fitted to Tactile feature 1

CLASSIFICATION ACCURACIES WITH DIMENSION REDUCED AND PREDICTED DATASETS

Classifiers

Dimension Reduced Original Dataset EEG

Tactile

EEG+Tactile

Predicted Dataset EEG

EEG+Tactile

Linear SVM

60.9524

94.5238

95.2381

70.2381

95.5714

Naïve Bayes

67.1429

93.8095

94.1667

81.9048

94.6667

KNN

62.8571

92.8571

67.8571

74.2857

78.2381

Tree

66.4286

86.1905

87.6190

72.3809

87.7619

Linear SVM

53.0952

87.6190

88.5714

74.7619

89.3810

Naïve Bayes

57.8571

84.7619

85.4762

76.4286

86.7619

KNN

56.9048

87.3809

58.5714

71.4286

69.9524

Tree

54.0476

85.9524

90.9524

69.0462

91.7143

The average classification accuracies over all the classes with the initially extracted features forming the original datasets are shown in Table I. It can be noted that with the exception of kNN both the features yields higher accuracies with the EEG and Tactile features taken simultaneously. This calls for a prediction method to obtain similar EEG features with the help of tactile features in absence of the former to improve upon the classification accuracy.

Science, Jadavpur University and Council of Scientific and Industrial Research (CSIR), India. REFERENCES [1] [2]

Table II shows the root mean square error obtained by fitting the EEG features to Tactile features by using polynomials of order 6 and 8. We note that the order of the error on the validation dataset increases from that of the training dataset as order of the polynomial increases from 6 and beyond. Also for Hjorth parameters, features 1, 3 and 10 have very high error and is hence excluded in the further work.

[3]

The results of classification with the dimension reduced dataset and predicted dataset are shown in Table III. The nonstationarity in the predicted EEG features is lesser than that in the original dataset. This accounts for the higher accuracy shown by the predicted EEG features. Also the results supports that proposal that the object shape classification using the predicted EEG from Tactile features along with the tactile features provides superior performance than that with the tactile features alone. Although the accuracies slightly deteriorate from the original dataset, the space complexity with this method decreases drastically.

[6]

IV.

CONCLUSION

The EEG and tactile signals acquired during dynamic tactile exploration of the objects under study, is classified in corresponding object shape classes. The results have supported the hypothesis that though object shapes can be classified from both the signals separately, but the classification accuracy in case of joint EEG and tactile features is higher than that of either alone. In situations where EEG signals availability cannot be ensured, we have predicted EEG features from the original tactile features after dimension reduction. The predicted EEG features and the feature set containing the original dimension reduced tactile features and the predicted EEG features are able to classify the objects shapes better than the original dataset comprising the acquired EEG and the EEG and tactile data concurrently. Thus our EEG feature prediction from the tactile features is an expedient choice where EEG signals are absent, which enhances the classification accuracy at the same time reducing the computation cost since the features the dimension reduced. In future we will be incorporating non-linear approaches of EEG prediction which will retain the stochastic nature of EEG and also reduce the error in prediction.

[4] [5]

[7] [8] [9]

[10] [11] [12] [13] [14]

[15] [16] [17] [18]

[19]

ACKNOWLEDGMENT This study has been supported by University Grants Commission (UGC), India; University with Potential for Excellence Program (UGC-UPE) (Phase II) in Cognitive

[20]

D. O. Bos, “EEG-based emotion recognition”, The Influence of Visual and Auditory Stimuli, pp. 1-17, 2006. Y. Liu, O. Sourina, and M. K. Nguyen, “Real-time EEG-based human emotion recognition and visualization”, International Conference on Cyberworlds, pp. 262-269, October 2010. C. Neuper, R. Scherer, M. Reiner, and G. Pfurtscheller, “Imagery of motor actions: Differential effects of kinesthetic and visual–motor mode of imagery in single-trial EEG”, Cognitive Brain Research, vol. 25, no. 3, pp. 668-677, 2005. T. Ergenoglu et al., “Alpha rhythm of the EEG modulates visual detection performance in humans”, Cognitive Brain Research, vol. 20, no. 3, pp. 376-383, 2004. M. Grunwald et al., “Theta power in the EEG of humans during ongoing processing in a haptic object recognition task”, Cognitive Brain Research, vol. 11., pp. 33-37, 2001. A. Khasnobish et al., “Object Shape Recognition from Tactile Images using a Feed Forward Neural Network”, IEEE International Joint Conference on Neural Networks, pp. 1-8, 2012. G. Canepa, M. Morabito, D. De Rossi, A. Caiti, T. Parisini, “Shape from touch by Neural Net”, Proceedings IEEE International Conference on Robotics and Automation, 1992. G. Singh et al., “Object shape recognition from tactile images using regional descriptors”, IEEE 4th World Congress on Nature and Biologically Inspired Computing, pp. 53-58, November 2012. Singh G. et al., “Object-shape classification and reconstruction from tactile images using image gradient”, IEEE 3rd International Conference on Emerging Applications of Information Technology, pp. 93-96, November 2012. C. Jinhai and L. Zhi-Qiang, “Hidden Markov models with spectral features for 2D shape recognition”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, pp. 1454-1458, 2001. http://www.emotiv.com/ Chameleon Software for TactArray, Operator’s Guide, 2/1/2012. A. Schlögl, F. Lee, H. Bischof, G. Pfurtscheller, “Characterization of four-class motor imagery EEG data for the BCI-competition 2005”, Journal of neural engineering, vol. 2, no. 4, L14. A. Schlögl, K. Lugger, G. Pfurtscheller, “Using adaptive autoregressive parameters for a brain-computer-interface experiment”, International Conference of Engineering in Medicine and Biology Society, vol. 4, pp. 1533-1535, November 1997. B. Hjorth, “EEG analysis based on time domain properties”, Electroencephalography and clinical neurophysiology, vol. 29, no. 3, pp. 306-310, 1970. B. Hjorth, “Time domain descriptors and their relation to a particular model for generation of EEG activity”, CEAN-Computerized EEG analysis, pp. 3-8, 1975. M. Robnik-Šikonja, and I. Kononenko, “Theoretical and empirical analysis of ReliefF and RReliefF”, Machine learning, vol. 53, no. 1-2, pp. 23-69, 2003. I. Kononenko, M. Robnik-Sikonja, and U. Pompe, “ReliefF for estimation and discretization of attributes in classification, regression, and ILP problems”, Artificial intelligence: methodology, systems, applications, pp. 31-40, 1996. S. C. Lemon, J. Roy, M. A. Clark, P. D. Friedmann, and W. Rakowski, “Classification and regression tree analysis in public health: methodological review and comparison with logistic regression”, Annals of Behavioral Medicine, vol. 26, no. 3, pp. 172-181, 2003. L. Jost, “Entropy and diversity”, Oikos, vol. 113, no. 2, pp. 363-375, 2006.