Virtual Hand Prosthesis Moved By Encephalographic ... - Unicauca

4 downloads 5559 Views 280KB Size Report
the potential difference across the cell membrane of neurons and this process is the ... the EEG signals and operate through EmoEngine Emotiv API. These suites are ... dynamic link library edk.dll) and toolkit SDK (Software. Development Kit) ...
CIIMA – INTERNATIONAL CONGRESS OF MECHATRONICS ENGINEERING AND AUTOMATION

Vol. 03, no. 1, 2014, pages 1–5

INVESTIGATION

Virtual Hand Prosthesis Moved By Encephalographic Signals. Correa A. Karin, Master of Engineering in Automation (c), University of Cauca, Popayán, Colombia. [email protected]

Abstract— This paper presents a research project that has the challenge of manipulating a prosthetic hand in a virtual simulation environment using a natural interface based on a BCI (Brain Computer Interface), which should propose a new paradigm for the manipulation of a prosthetic hand. The information provided by encephalographic (EEG) signals captured from an Emotiv® headset is used for the manipulation of a virtual hand prosthesis. This system has fourteen electrodes distributed over the skull of the user, who after a training phase can produce simple commands through the manufacturer's software. Finally, this research presents the results obtained so far, where the user’s encephalographic signals move a virtual hand built in the computer (using free software tools as Qt and VTK). It is expected that the user can train and reproduce various grasps such as cylindrical, spherical and pincer grasp in the virtual hand. KeyWords: robotic prosthetic hand, encephalographic signals, natural interface, brain-computer interface.

I. INTRODUCTION.

I

n Colombia, total or partial amputations of a limb are not only caused by disease, but also by accidents and by the armed conflict. The Colombian Association of Physical Medicine and Rehabilitation estimates that the incidence of amputation in the country ranges from 200 to 300 people per hundred thousand inhabitants [1]. This number tends to increase in population with risk factors such as diabetes, vascular problems chronic diseases. According to the Association of Physical Medicine, surgical amputations are performed with two objectives: first, to eliminate or counteract the cause, to reduce risk and preserve life and second, to allow adequate subsequent rehabilitation to achieve the best fit of prosthesis and restore motor functions associated with the hand as well as possible [2]. Although the use of prostheses is a consequence of the above mentioned drawbacks [3], the development of these has been closely linked with the development of assistive robotics which today constitutes one of the applications to highlight in Bioengineering. Although the history of these devices began over 30 years ago, their evolution has not been what was originally envisaged. There are several reasons for this stand still, for instance, the limited

© 2014 - Universidad del Cauca

Vivas A. Andrés, PhD. In Robotics, University of Cauca, Popayán, Colombia. [email protected]

functionality obtained with respect to the use of healthy arms and hands, not only due to mechanical limitations or manipulative strategies, but also to the user's possibilities to transmit appropriate orders to produce the desired movements [4]. One of the most widely used ways to manipulate the prostheses already implanted in patients is based on using the user's myoelectric signals generated in his muscles to enable the prostheses, however if the patient has current limitations in his remaining muscles and he cannot contract them easily, the user's electromyography signals (EMG) cannot be captured and the manipulation of the prostheses without noninvasive methods is not possible [5]. There are also other drawbacks such as the confusion in the correct identification of the user’s intention of movement and muscle signal degradation in patients whose injury occurred long ago, as the patient loses the ways of communication with the nervous system given the status of his muscle is unknown and must learn to contract it at will, which is not easy [6]. Also, in recent years many efforts have been made in the development of hybrid bionic systems capable of binding, via natural interfaces, the human nervous system with prosthesis, or even with external robotic machines, with the main objective of recovering sensory and motor functions of patients with injuries in the spinal cord or the brain, due to degenerative diseases or accidents that have led to amputations in patients [7]. This research project searches for a novel technique for the manipulation of a virtual automated hand prosthesis using a BCI (Brain Computer Interface). Handling this type of prosthesis based on brain signals becomes important when it is understood that developments in this area are minimal compared to other studies in the medical field and can be a solution to the problems relating to the natural movement of the hands described above [8].

1

Virtual Hand Prosthesis Moved By Encefalographic Signals.

II. HAND PROSTHESIS. Currently there are various types of upper limb prostheses and these vary within the limits of the basic needs of patients who use them. Some prostheses seem very real; others have such an advanced technology that can be considered as robots. Some prostheses do not move at all (aesthetic prosthesis), others can be set only in specific positions and others are mechanical and are controlled by muscles, wires and steel cables [9]. However in all the above types of prosthesis, the control mechanism has been a limiting factor in their functionality. In the field of robotic prostheses, major research, development and applications based in assistive robotics and in the development of these upper limb prostheses, have been conducted in countries like the U.S., Japan, France, Germany and Europe in general [10]. The high accuracy of this technology has aroused great interest in the medical and engineering field. Among these robotic prosthetic hand, the best known are: Michelangelo®, I – Limb Ultra, The Dextrus Hand, BeBionic, Handie, among others [11], [12] y [13]. As for their operation, one of the most used ways to manipulate the prosthesis already implanted in patients is through an application on a mobile device and the other is based on the user’s electromyographic (EMG) signals. The first technique has the disadvantage of the mobile device manipulation by the patient with his healthy hand, a situation that markedly restricts the manipulation of objects that surround him. The second technique is based on using the user’s myoelectric signals generated in his muscles to enable the prostheses [5], however if the user has limitations in his remaining muscles and cannot move them without great effort, the user’s associated bioelectric signals cannot be registered and it is not possible to operate the prosthesis successfully. Additionally, for the use of this technique, it is necessary to keep in mind that the patient may lose several ways of communication with the nervous system due to the amputation, therefore the status of his muscle is not known with accuracy and the patient must go through a long process to learn to contract it voluntarily. Similarly, for developing countries it is more difficult to acquire and apply this technology mainly because of its cost. For example the prosthetic hand BeBionic manipulated through electromyography signals and/or through a mobile application [13] costs about $ 10,000, and this without considering the cost of the rehabilitation treatment of the patient who has suffered injury or amputation. Fig. 1. shows the prosthetic hand BeBionic.

2

Fig. 1. Robotic Prosthesis Hand BeBionic.

The above situations have led to recent research about the manipulation of prostheses using brain-computer interfaces. The last investigations (highly expensive) that have been developed in this field, use invasive BCI interfaces, that is, they involve surgical procedures inside the head of the patient for the connection of sensors that record bioelectric signals produced by the human brain [14]. One of the most renowned projects in this area is called Life Hand II, conducted by European scientists (Italy, Switzerland and the Netherlands) for research processes [15]. This automated prosthetic hand is capable of reproducing the sense of touch, given it connects the patient's nerves to the hand. This prosthetic hand is able to control the strength applied on the grip of objects and it is handled with the interpretation of brain signals. Currently, BCI interfaces which make use of non-invasive physiological sensors are also being used. These use a number of electrodes for reading electrical signals caused by brain activity on the scalp and this type of noninvasive BCI can be obtained as a viable option for the control of devices and applications by interpreting brain signals, which are relatively more accessible than their medical counterparts [7] and [16]. III. ENCEPHALOGRAPHIC SIGNALS (EEG). The electrical signals produced by the brain are generated by the potential difference across the cell membrane of neurons and this process is the basis for the functioning of our nervous system. Registration of these bio signals is what is known as electroencephalogram (EEG) and the rhythms of neuronal activity are communication language typical of neurons. As we have advanced signal processing algorithms that help to understand the meaning of the EEG signals were generated clinical and technological applications that until recently belonged to the area of science fiction [17]. The Fig. 2. shows certain physical or mental conditions that may occur in patients who are diagnosed and is due to the interpretation of EEG.

© 2014 - Universidad del Cauca

K. Correa-Arana y A. Vivas-Albán

The Emotiv® Neuroheadset has autonomy of 12 hours and the manufacturer's application Test BenchTM can display the EEG signals. V. SYSTEM DESCRIPTION. The main objective of this research project is the manipulation of a virtual hand prosthesis hand using EEG signals captured with a BCI low cost, in this case with the Emotiv® neuroheadset, which should propose a new paradigm for handling hand prosthesis, while the disadvantages presented above can be solved.

Fig. 2. Diagnosis of diseases or conditions that can be detected from the making and interpretation of the EEG.

EEG recording of signals is also the basis of the BCI belonging to the group called natural interfaces.

This project uses as a platform of virtual prototype prosthesis developed by the Industrial Automation Group of the University of Cauca. The application uses Qt as a framework of user interface (window), also makes use of graphical display libraries VTK widely used in medical applications. The development was done in C++ language in the Visual Studio 2010 platform. The application to manipulate each of the joints of an anthropomorphic virtual hand, it can be seen in Fig. 4

IV. BRAIN COMPUTER INTERFACE. A brain computer interface, or BCI belongs to the group of natural interfaces, these are used as a means of interaction (HMI), because they allow the manipulation of applications or devices by recording and interpretation of the electroencephalographic (EEG) signals without dependence mechanical devices. The use of natural interfaces for people with disabilities is a novel application, because with these and with the use of robotic prostheses, the user can perform natural movements executed before losing his limb. Below the Emotiv® Headset is presented.

Fig. 4. Prototype anthropomorphic virtual prosthetic hand.

A. Programming Interface Application (API). A. The Emotiv® EEG Neuroheadset. This headset has a BCI and fourteen non-invasive brain electrodes plus two references (10-20 located as standard), which allow the recording of encephalographic signals, specifically the rhythms: Delta (0.5 - Hz), Theta (4 - 8 Hz), Alpha (8-14 Hz) and Beta (14-26 Hz). Then these signals are sent to the computer via Bluetooth. Fig. 3. shows the Emotiv® Headset [16].

Fig. 3. Brain Machine Interface – Headset based Emotiv® technology.

© 2014 - Universidad del Cauca

Emotiv offers a set of suites that allow the interpretation of the EEG signals and operate through EmoEngine Emotiv API. These suites are ExpressivTM, AffectivTM and CognitivTM. The interpretation of the signals according to the suite used is displayed on the Control Panel Emotiv.

Fig. 5. Emotiv Control Panel. Left. Select User and signal quality. Right. Cognitive Suite Panel.

3

Virtual Hand Prosthesis Moved By Encefalographic Signals.

Within the control panel are presented the different suites. Particularly CognitiveTM suite shows a cube that can be manipulated (move back, forward, left, right, etc.) through the EEG signals after a training phase. B. System Integration. The integration of virtual prosthetic hand and Emotiv® Neuroheadset was performed with the API (specifically with dynamic link library edk.dll) and toolkit SDK (Software Development Kit), since the latter provides classes and functions in C++ to manage the suites mentioned above. C. Manipulation of the Virtual Hand Prosthesis.

Fig. 7. Virtual hand prosthesis of fine tip grip and feedback to the user.

If the user is relaxed (corresponding to cognitive training mode neutral action and is recommended to be trained first action) virtual hand prosthesis should be fully open, as shown in Figure 6. D.

For handling the virtual prosthetic hand is chose, this can make the grips: tip, cylindrical grasp and spherical grasp, also must be possible to open and close completely. The hand manipulation is done through the user's thoughts and intention of movement, for this application use CognitiveTM suite where the user must pass a training stage is done, in which different images imagine inferred (related to the different opening and grasp with your hand) and brain patterns are saved and associated with the generation of the various control actions for the handling of the prosthesis in the grips mentioned above. Fig. 6. shows the arrangement of the virtual hand of different types of grips. Fig. 8. User by manipulating the virtual hand prosthesis.

D. Operating Conditions and Restrictions. In order to ensure that the system works properly, the following conditions must be met: All electrodes must be properly recording the EEG signals (green indicators on the Emotiv Control Panel). Each user must go through the training stage to grips mentioned above and should get a skill rating greater than 70%, this task varies by user and can be performed in an average of six hours’ time.

Fig. 6. Arrangement of virtual hand prosthesis for differents grips. A. Fine Tip. B. Cylindrical Grasp. C. Spherical Grasp. D. Relaxed Hand.

For example, if the user moves the bucket to back (along the z axis in the user interface) shown in Fig. 7 after passing the training stage, a control action is generated and this is interpreted by software. This indicates that the virtual hand prosthesis should fit for gripping fine tip. Similarly other control actions are generated for the execution of the other grips mentioned above.

4

The user must have short hair and felt the sensors must be moistened with conductive gel. The Emotiv neuroheadset can be away from the USB receiver maximum distance of one meter to avoid signal loss. The application is designed to work only with one Emotiv neuroheadset time, as only supported training load profile of a user at a time. Manipulation with EEG signals virtual hand prosthesis is aimed at people who had their entire upper extremity and who © 2014 - Universidad del Cauca

K. Correa-Arana y A. Vivas-Albán

was amputated due to an accident or illness, and once learned to move his limb in a natural way and learning associated is stored memory [18]. The number of grips that can be done with the virtual hand prosthesis is restricted to three and these were adds: open and fully closed hand; this limitation is presented to avoid false positives. CONCLUSIONS This paper presents a project about the manipulation of a virtual hand prosthesis using encephalographic signals which poses several scientific challenges. It also stresses that this type of studies provide another way of manipulating hand prostheses by persons with disabilities. The results obtained so far show promising developments of this new technique that uses the BCI, which are expected to become an alternative to the traditional management of hand prosthesis from electromyography signals. The latter, which are the most used method to handle this type of prosthesis, have several drawbacks such as the confusion in the correct identification of the user’s intention of movement and muscle signal degradation in patients whose injury occurred long ago. These drawbacks can be solved using this type of natural interface. Future work will establish clear performance indices of this new technique in order to rigorously evaluate its potential as a control system for robotic prosthetic hand amputees. REFERENCES. [1] D. SMITH, “Prótesis de Extremidad Superior. Segunda parte” inMotion, Vol. 17, No. 4, 2007.

[2] S. MICERA, E. CAVALLARO, R. BELLI, F. ZACCONE, E. GUGLIELMELLI et al. “Functional Assessment of Hand Orthopedic Disorders Using a Sensorised Glove: Preliminary Results”, IEEE International Conference on Robotics & Automation, ICRA 2003, pp. 2212-2217, 2003. [3] P. DARIO, S. MICERA, A. MENCIASSI, M.C. CARROZZA, M. ZECCA, T. STEIGLITZ, T. OSES, X. NAVARRO, D. CEBALLOS. "Cyberhand - A consortium project for enhanced control of powered artificial hands based on direct neural interfaces", 33rd Neural Prosthesis Workshop, USA, 2002. [4] A. KAYSA, W. SUPRIJANTO, A. WIDYOTRIATMO. “Design of Brain-Computer Interface Platform for Semi Real-Time Commanding Electrical Wheelchair Simulator Movement”. 3rd International Conference on Instrumentation Control and Automation, 2013. [5] W. OUYANG, K. CASHION, V. ASARI, “Electroencephalograph Based Brain Machine Interface for Controlling a Robotic Arm”. Department of Electrical and

© 2014 - Universidad del Cauca

Computer Engineering, University of Dayton, Dayton, OH 45410, USA, 2013. [6] S. SUN, C. ROSALES, and R. SUAREZ, “Study of Coordinated Motions of the Human Hand for Robotics Applications,” The 2010 IEEE International Conference on Information and Automation, pp. 776–781, Harbin, China, June 2010. [7] J. MUÑOZ, C. MUÑOZ, O. HENAO. “Diseño de una Estación de Trabajo para Personas con Discapacidad en Miembros Superiores Usando una Interfaz Cerebro Computador”. Tecno Lógicas, Edición Especial, Octubre 2013. [8] A.KAWALA-JANIK, M. PODPORA, M. PELC, P. PIATEK, J. BARANOWSKI. “Implementation of an Inexpensive EEG Headset for the Pattern Recognition Purpose”. The 7th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, Berlin, Germany, September 2013. [9] G. MATRONE, C. CIPRIANI, E. SECCO, G. MAGENES, AND M. CARROZZA, “Principal components analysis based control of a multi-dof under actuated prosthetic hand,” Journal of Neuro Engineering and Rehabilitation, vol. 7, no. 1, p. 16, 2010. [10] T. WIMBOCK, C. OTT, A. ALBU- SCHAFFER, AND G. HIRZINGER,“Comparison of object-level grasp controllers for dynamic dexterous manipulation,” The International Journal of Robotics Research, vol. 31, no. 1, pp. 3–23, 2012. [11] Otto Bock Healthcare Products, “Michelangelo®” (April, 2014). Available http://www.ottobock.com. [12] Touch Bionics, “I-Limb Hand” (April, 2014). Available http://www.touchbionics.com [13] RSL Steeper, “BeBionic” (April, 2014). Available http://rslsteeper.com/ [14] S. KIM, J. SIMERAL, L. HOCHBERG et al, “Point and Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia”, IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol.19, No. 2, April 2011. [15] G. DI PINO, A. BENVENUTO, M. TOMBINI, G. CAVALLO, L. DENARO, V. DENARO et al, “Overview of the implant of intraneural multielectrodes in human for controlling a 5-fingeres hand prosthesis, delivering sensorial feedback and producing rehabilitative neuroplasticity”, 4th IEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics, pp. 1831-1836, Roma, Italy, June 2012. [16] Emotiv EEG Systems (April, 2014). Available http://www.emotiv.com/ [17] N. MOHD, R. JAILANI, H. NORHAZMAN and N. MOHAMAD. “Alpha and Beta Brainwave Characteristics to Binaural Beat Treatment”. IEEE 9th International Colloquium on Signal Processing and its Applications, Kuala Lumpur, Malaysia, 2013. [18] S.GRUDE, M. FREELAND, C. YANG AND H. MA. “Controlling mobile Spykee robot using Emotiv Neuro Headset”. Proceedings of the 32nd Chinese Control Conference, Xi’an, China, July 2013.

5