2005 Conference Proceedings
Seite 1 von 6
2005 Conference Proceedings Go to previous article Go to next article Return to 2005 Table of Contents
ALTERNATIVE CONTROL SYSTEM FOR OPERATING A PC USING INTENTIONAL MUSCLE CONTRACTIONS ONLY Presenter(s) Torsten Felzer Darmstadt University of Technology Department of Mechatronics and Machine Acoustics Email:
[email protected] Rudolf Fischer Email:
[email protected] Thomas Groensfelder Email:
[email protected] Rainer Nordmann Email:
[email protected]
Available at http://www.csun.edu/cod/conf/2005/proceedings/2121.htm
Abstract: This paper presents an extension of the Windows® operating system enabling persons, who cannot use their hands, to operate a computer. The tool translates willfully generated muscle contractions into actions of the computer mouse, which can therefore be controlled hands-free. A collection of programs built around the main processing module provides all functionality needed to make the user of the software almost totally independent from external help while working with the computer, e.g. composing emails or surfing the web. 1. Introduction The common way of interacting with a PC involves employing a keyboard and a manual mouse device, which demands from the user to be able to reliably operate hands and arms. Consequently, a "hands-free" alternative is needed for persons who have - temporarily or permanently - problems with this kind of movement control. This paper introduces such an alternative, called HAMCOS, which stands for "HAnds-free Mouse COntrol System". The next section deals with various other approaches concerning hands-free control reported in the literature. This is followed by a brief discussion of the theoretical concept forming the basis of HAMCOS and a detailed description of all of its components in sections 3 and 4. Finally, the paper is concluded with a short summary and some thoughts on future work in section 5.
file://H:\CSUN05_felzer.htm
27.06.2005
2005 Conference Proceedings
Seite 2 von 6
2. Related Work Numerous approaches concerning hands-free human-computer interaction have been developed in the past. A lot of them take possession of the eyes by examining the line-of-sight (e.g. [3]). However, it is questionable if this keynote really leads to a desirable solution, because the eyes might be needed for something else. A much more promising paradigm is given by so-called brain-computer interfaces (BCI's), which are based on an idea first mentioned in [1]. A BCI (for an overview, see [4]) tries to analyze the ongoing EEG signals (i.e. the brainwaves) of a subject and to recognize recurring patterns. The subject's goal is to produce those patterns at will, in order to launch predefined control commands. The BCI idea sounds great, particularly since it requires merely mental (no physical) activity. Unfortunately, recording EEG signals is extremely sensitive, so that BCI's can only work, if the subject completely renounces any disturbing muscular activity. To cope with this problem, the system introduced in [2] combines EEG with EMG as input signals. The system presented in the following sections exclusively relies on muscle activity. Anyway, even a patient who is able to consciously control only one single muscle group can operate the system. 3. Basic Principle When developing HAMCOS, the goal was to design an input tool that requires as little physical effort as possible, while being very robust and easy to use. As has been described elsewhere (see [5]), it is rather simple to detect muscle contractions (originating from any muscle of choice) by merely inspecting the EMG amplitude. Depending on the question, whether or not the amplitude exceeds a certain threshold, the input signal can be reduced to a stream of single (see fig. 1) or double (see fig. 2) "contraction events" (CE). By considering the time between two CE's, the reduced stream suffices to move the mouse pointer to any location on the PC's desktop. The idea is to keep track of an internal state according to the transition diagram depicted in fig. 3 and to issue mouse actions depending on the one currently selected. 4. Software Solution As the name already indicates, HAMCOS allows controlling the mouse pointer without any manual input. Rather, HAMCOS processes muscle-related signals based on the theoretical concept established in the preceding section. It was decided to use the standard microphone input of the computer's soundcard - which is available in almost any PC - for the acquisition of input data. Therefore, the only hardware requirements are a decent PC and a relatively inexpensive sensor. One of many possibilities is to attach the sensor to the user's forehead with the help of a headband. The effective setup depends on the user's capabilities. The sensor utilized in an exemplary prototype (see fig. 4) actively induces a voltage potential when innervated. By favoring this design over a standard passive EMG electrode, the system is desensitized towards external electromagnetic interference. HAMCOS has been developed on a PC running Windows® XP, and it heavily depends on certain operating system functions. It has to be adapted, in order to work under non-Windows®-based operating systems. The HAMCOS system consists of a main component providing the user interface and a number of auxiliary programs supplying more advanced functionality. The following subsections contain detailed descriptions of the individual components. 4.1 Main Module The main processing module of HAMCOS is associated with the dashboard-like
file://H:\CSUN05_felzer.htm
27.06.2005
2005 Conference Proceedings
Seite 3 von 6
window depicted in figs. 1 and 2, initially displayed in the bottom right corner of the screen. When invoked, it automatically selects the microphone as active recording audio device and starts to "listen" to the incoming signals, which are simultaneously displayed in the central part of the left half of the window. To the right, there are several indicators, e.g. informing the user about the current state or the speed of mouse pointer movements. Beginning in the middle "STOP" state, the user may start moving (with gradually increasing speed) by generating a CE2. Subsequent CE1's cause the system to cyclically wander through the "LEFT", "UP", "RIGHT", and "DOWN" states. After a second CE2, the state returns to "STOP", and the processing module issues a click at the current position of the mouse pointer. Actually, in most cases, it does not suffice to merely know where to issue a (left or right) click, the type of the click (e.g. single or double) might also be important. In those cases, the processing module inserts a click type selection after the stopping CE2: the user may cycle through the possible click types with CE1's and have HAMCOS issue the currently selected click by producing another CE2. Above and below the "central band", the main window contains four buttons, which are kept relatively large to make it easier to click on them. In the type selection step, these buttons are used to display the click type options, and in all other cases, they are assigned to various auxiliary functions, which are described in the following subsections. 4.2 "Keypad/Position" Button The top left button is responsible for two very convenient aids. A single left click on this button (of course by employing the HAMCOS principle itself) causes a pad with buttons for the most important keys - e.g. the "ESC"-key or the "Return"-key - to be ejected. A right click on the "Keypad/Position" button results in sending the main window to one of the other three corners of the screen. This can be very helpful, since the main window always stays on top of any other windows - so that it is always there, when needed - and thus might obscure something the user wants to look at. 4.3 REVERSI Game The bottom left button of the main window is dual-purpose as well. A left click here invokes an instance of the famous REVERSI game, where the user and the PC take turns in placing pieces of two respective colors on an 8x8-board. The idea behind this game implementation is to offer a hands-on opportunity for getting familiar with the HAMCOS system. 4.4 Mouse Editor Right clicking the bottom left button launches an editor application which does not require any keyboard input, though offering almost any functionality of the built-in "Notepad Editor". This application might alternatively serve as a hands-on familiarization tool. In addition (especially since it allows cutting and pasting using the standard "Windows® Clipboard"), it renders a keyboard for text interaction unnecessary. 4.5 Start Screen The top right button of the main window responds to a single click (left or right) by calling the "Start Screen" application, which is sort of a combination of the built-in "Task Manager" and the "Windows® Desktop". This application displays all currently open windows as well as running
file://H:\CSUN05_felzer.htm
27.06.2005
2005 Conference Proceedings
Seite 4 von 6
processes and buttons dealing with the state of the OS. It also provides customizable shortcuts for arbitrary applications. 4.6 Configuring HAMCOS The main window's final (bottom right) button may be clicked any time to edit the current configuration. The user gets the chance to fine tune some internal parameters, such as an amplification factor or the acceleration of the mouse pointer. This might be necessary, as the particular computer hardware and the physical ability of the user both make a difference (in general). Even a tremor can be compensated by adjusting the threshold. Customizing the shortcuts displayed by the "Start Screen" is also one of the tasks to be accomplished here. 5. Conclusion A system allowing to control the mouse pointer on a PC desktop merely by issuing contractions of any muscle of choice has been presented. While being easy to use and requiring extremely little physical effort, the system is almost free of external noise, thus representing a valuable assistive device for those persons with physical disabilities who are able to exert reliable control over at least one muscle group. HAMCOS not only demonstrates that the hands-free use of a computer is indeed possible or how it might be done in theory, but it represents a practical realization of the muscle control idea. The wheelchair control device described in [6] is based on the same basic principle as HAMCOS. Both applications are undergoing some major improvements in the near future. References: [1] Vidal, J., "Toward direct brain-computer communication," in: Annu. Rev. Biophys. Bioeng., pp. 157-180, 1973. [2] Patmore, D.W., Putnam, W.L., and Knapp, R.B., "Assistive cursor control for a PC Window environment: Electromyogram and electroencephalogram based control," in: Center on Disabilities Virtual Reality Conference - Proceedings, 1994. [3] Park, K.S. and Lee, K.T., "Eye-controlled human/computer interface using the line-of-sight and the intentional blink," in: Computers and Industrial Engineering 30(3), pp. 463-473, 1996. [4] Wolpaw, J.R., Birbaumer, N., et al., "Brain-computer interface technology: A review of the first international meeting," in: IEEE Transactions on Rehabilitation Engineering 8(2), pp. 164-173, 2000. [5] Felzer, T. and Freisleben, B., "Controlling a computer using signals originating from muscle contractions," in: METMBS '02 - Proceedings, pp. 336-342, CSREA press, 2002. [6] Felzer, T. and Freisleben, B., "HAWCOS: The 'hands-free' wheelchair control system," in: ASSETS 2002 - Proceedings, pp. 127-134, ACM press, 2002. Figure captions:
file://H:\CSUN05_felzer.htm
27.06.2005
2005 Conference Proceedings
Seite 5 von 6
Fig. 1: HAMCOS' main window showing a single contraction event CE1
Fig. 2: Double contraction event CE2
Fig. 3: Transition diagram
file://H:\CSUN05_felzer.htm
27.06.2005
2005 Conference Proceedings
Seite 6 von 6
Fig. 4: Hands-free text entry by spelling out words Summary: The system's demonstration will cover the sensor gathering muscle-related signals, the main module driving the computer mouse accordingly, and the framework actually extending the OS
Go to previous article Go to next article Return to 2005 Table of Contents Reprinted with author(s) permission. Author(s) retain copyright.
file://H:\CSUN05_felzer.htm
27.06.2005
