Control of 2DOF Periferal Devices by Means of ...

Control of 2DOF Periferal Devices by Means of Intuitive and Ergonomic Head Movements M. Bureau a, C. Rodríguez-de-Pablo a, J.M. Azkoitia a, G. Eizmendi a, I. Manterola a, M. Perez b, J. Medina b a Fatronik Foundation, Research Technology Center, Donostia, Spain b Guttmann Universitary Institute for Neurorehabilitation-UAB, Badalona, Spain

Abstract. This article presents the use of accelerometer and gyroscope to track pitch and yaw head movements in static and dynamic references in order to control two degrees of freedom peripheral devices. The interface is expected to be a communication alternative for many users with mobility impairments in upperextremities, enabling interaction with most of external devices needed to develop a normal life activity, gaining autonomy and independence. We select the wheelchair and mouse pointer as the most useful and representative peripheral devices. At this stage, the system is patent pending with request P2006600109 on January 18th 2006. Keywords. Accelerometer, gyroscope, intuitive, ergonomic, universal interface, low-cost.

Introduction Nowadays, many of the daily life activities depend on technology based devices and we live surrounded by “friendly technology tools” such as TVs, computers, smart homes and others that make our life easier. The interaction with the so called “digital devices” becomes critical for people with mobility impairments. Currently, multiple solutions are available for alternative Human Machine Interaction (HMI) but they are specific, expensive, not compatible and not standardized. The challenge of the work presented in this paper is the development of a low cost Head Movement Tracker (HMT) that allows controlling different peripherals. This will be an alternative solution for upper-limb impaired people (SCI C4-C5 / amputee). The solution described in this paper takes into account feedback form users and rehabilitation professionals obtained from the first prototype [1] evaluation in the Guttmann Institute in Badalona. The conclusions showed the need to define a more intuitive and ergonomic head movements tracking algorithm, to achieve high precision and reduce neck pains after long time use. However, it validated the usefulness of the system as a universal interface to improve autonomy. The technical method use kinematic sensors (accelerometers and gyroscopes), to monitor the head’s yaw and pitch angles to achieve combined, discrete and continue control of two degrees of freedom (DoF) in static and dynamic applications.

1. State of the art 1.1. HMT objectives definition First of all, it is important to define intuitiveness, ergonomic and universality terms which are applied for the HMT. 1) To be intuitive, HMT must double the user head’s looking direction to the adequate machine dof control. The pitch head movement can control either, machine pitch or XG or ZG translation, while the yaw head movement can control either yaw or YG translation, and the pitch head movement can control only the pitch machine movement. Additionally, HMT must provide continue control. 2) To be ergonomic HMT should limit the neck muscle activities to reduce the use of high accelerations; speeds, extreme positions and long time static non comfortable positions. 3) To be universal HMT should be able to interface different devices and the user should be able to commute from one to another device with autonomy and simplicity. 1.2. Related works There are various technologies being used such as infrared [2], ultrasound [3]-[4], image processing [5] and kinematics sensors [6]-[7]. Infrared, image processing and ultrasound technologies work with at least one external component looking the operator which means less flexibility of the system. Combination of kinematics sensors are actually more flexible but provide, either adequate machine dof control but discrete detection [6], or, head posture but non adequate dof control [7]. We try to give a step forward regarding the HMT objectives definition and the related work, developing an alternative solution targeting these challenges: universality, low cost, ergonomic and intuitive.

2. Technical solution Within this work we develop a system witch compute head posture through kinematics sensors when the rest of the body is also moving in all directions in the earth reference (i.e. driving the wheelchair). The technical solution consists of two groups of kinematic sensors, the first one mounted at the back of the user’s head and the second one in the wheelchair as reference. The method consist of calculate the pitch and yaw angles between these two groups of sensors. The system integrated thus, two hardware units; the head-unit and the peripheralunit. The solution operates wireless technology, which brings mainly two advantages on it; the comfort due to the movement freedom and the longer distance we can control peripherals. 2.1. Hardware Design The accelerometer and gyroscope are MEMS technology sensors, selected due to their size (very small compared with other type of technology) and price (as they are massively used in the automotive safety industry). A typical yaw head velocity is around 220º/s [1], and a wheelchair rotation is in the same range of motion so a

gyroscope with a range of 500º/s is adequate to track the sum of head and wheelchair rotation velocity. The main characteristics are shown in Table I. Sensor

Model

Type

Output (V)

Sensitivity

Range

Accelerometer

DXL330

Triaxial

0-3.3

300mV/g

±3.6g

Gyroscope

IDG-300

Biaxial

0-3.3

2 mV/º/s

±500º/s

Table 1. Sensors characteristics, IMU 5DOF [8]

The wireless electronic elements (transmitter and receiver) are Cypress [9] PRoC “CYWUSB6953”. This 2.4GHz DSSS radio transceiver operates in the unlicensed Industrial, Scientific and Medical (ISM) band, and has a transmission range of 50 meters and a 62.5kbps throughput. Apart from the small size and economical criteria, the low energy consummation during a long time uninterrupted 50Hz data transition was important to select this WirelessUSBTM technology among other radiofrequency, Bluetooth or Zigbee. The interface integrates also “CY8C29466” Cypress PSoC microcontrollers to communicate with the wireless module via UART protocol, and reads analog sensor data and computes pitch and yaw head angles. 2.2. Angles processing The method consists in computing angles between two references which are moving related through a known kinematic model. A simplified head model represented by two rotary joints, the first one through the ZW axis and the second one through the YH (as show in figure 1) was chose to simulate the head positioning when the user looks the direction he/she wants to go.

Figure 1. Head kinematic model

The model above has the following change of reference equation:

X H = cos(θ ) ⋅ cos(ψ ) ⋅ X W − cos(θ ) ⋅ sin(ψ ) ⋅ YW + sin(θ ) ⋅ Z W (1) Regarding the equation 1, and assuming that both groups of sensors are looking the same vector acceleration, we can calculate the θ angle with data obtained from different sensors. ψ angle is required for calculus of θ angle.

The ψ angle is calculated with the integration of the result of the following equation 2 which needs rate gyroscopes information mounted in the ZW and ZH axes.

 ψ&  ψ& =  H  − ψ& W  cos(θ ) 

(2)

Because of the better stability of the θ calculus (no drift, and no influence of ψ when θ=0), we always compute θ before ψ and start the θ calculus with ψ=0. This universal interface is intended to be used for interaction with many common devices which are not accessible for impaired people. Many applications could be imagined. Any application would need an adaptation, in order to correlate headmovements with specific orders in the control system of the device. In some cases, head-movements will be used to control 2D positioning; for example when navigating across a menu in a digital TV, or when controlling a wheelchair, or when controlling the cursor in a PC. In other cases head-movements will be used to activate/deactivate options; for example, switching on/off lights in a smart home, or selecting an option from a menu. In every case, the processing of the signals from sensors located in head-unit is the same, and won’t need any further programming.

3. Control Design In order to demonstrate the operational use and universality of the interface, two representative peripherals have been selected to develop complete applications for this universal interface: 1) Control of an electrical wheelchair 2) Movement of a PC mouse pointer

Figure 2. Wheelchair and PC mouse pointer control demonstration.

In both cases, enabling alternative interaction with the device becomes critical for people with disability. Both, electric wheelchair and computer are key elements for independence and autonomy of people with severe disability. Own control of electric wheelchair gives big independence in terms of mobility and access to computer and internet opens a big window to the digital world. Concerning this, the possibility of controlling the wheelchair and accessing the computer with same interface will open big opportunities to people with disability.

The challenge of the control system is to deal with precision, reactivity and ergonomics. In the next section we present two different controls, the first one applied to the electrical wheelchair and the second one at the PC mouse pointer. 3.1. Electrical wheelchair As explained before, in this application the movements of the electrical wheelchair are controlled by means of head movements. This relation is developed assuring an intuitive interpretation for user. The pitch axis controls translational speed and the yaw axis the rotational speed. A calibration determines maximal and minimal pitch and yaw motions of each user’s head to obtain better precision and ergonomics. A direct proportional control of the rotational speed is applied and the biggest advantage of this solution is that the user looks where he wants to go. The same control was not applied for the translational speed control due to the user obligation to spend long time in non comfortable positions to maintain a constant velocity. We designed a discrete control with 10 incremental velocities in both positive and negative translation to keep precision. Next velocity is engaged after positioning the head in the range limit during a predefined period of time. This method avoids repetitive movements. Results in [1] illustrate that any movement in one axis has a reflection in the other. A 42% of the minimal and maximal pitch particular user’s head was experimentally found as the best threshold value. This value avoids undesired pitch threshold detection and head extreme pitch pain full positions. The velocities computed in the microcontroller are converted in adequate signals connected to an Omni+ controller and Pilot+ power module. These Penny and Giles drives technology modules are wide extended in electric wheelchairs for people with severe disability. 3.2. PC mouse pointer The developed HMT provides also access to the PC mouse pointer, demonstrating the versatility of the interface. From the hardware point of view, the reception-unit, integrates the same wireless receiver as in the wheelchair controller and a RS232 transceiver, to transform the pitch and yaw from the receiver into RS232 serial protocol signals. The mouse pointer positions are accessed with the User32.dll library. The pitch value controls the vertical position and the yaw value the horizontal position of the cursor. In the same way as in wheelchair, a calibration determines maximal and minimal pitch and yaw values. A proportional control is applied in both axes. To improve precision and limit the use of extreme positions and long time non comfortable positions, a moving window has been implemented. Minimal and maximal thresholds are determined in both axes. The internal thresholds zone is used to move the cursor inside the windows and the external thresholds zone to move the windows inside the screen.

4. Conclusions and further work The control of electric wheelchair has been tested by a researcher and a SCI C3 patient collaborator with part of the official slalom homologated by the Spanish Sport

Federation for people with Cerebral Palsy (FEDPC) [10]. The control of pointer in PC screen has been tested by 6 healthy persons in two applications, firstly in a developed cursor moving game to measure time to realize a predetermined sequence of actions, and secondly in windows XP environment with the freeware Point-N-Click software to realize action as click, double click. A “system usability scale” questionnaire and a “game pointer PC time” results allow concluding that the proposed head model permits a really intuitive and effective control, easy and rapid to learn. The principal drawback is the noticeable yaw drift due to the integration of equation 2. Concerning the ergonomic point of view, no pain has been reported but fatigue during the use of the Point-N-Click software. Crossing the screen at each click and double click, multiplies the head movements, causing fatigue. Regarding these conclusions, further work has been oriented in the following directions: 1) The yaw drift will be corrected by the implementation of magnetic sensors. The absolute values of the magnetic sensors will compensate the relative values of the gyroscopes. 2) Integrate an external head wearable commuting system to control any event-driven user intention (i.e click, double click, activation, deactivation, emergency stop). This head information tracker can be EMG, voice or tongue based. Other line of work is development of multiple integration use. This means that headunit should decide/select which reception-unit communicates with and send the proper information each time in the proper direction.

References [1]

M.Bureau, J.M. Azkoitia, G. Ezmendi and al. “Non-invasive, wireless and universal interface for the control of peripheral devices by mean of head movements” Proceedings of the 2007 IEEE 10th International Conference on Rehabilitation Robotics, June 12-15. [2] Yu-Luen Chen; Fuk-Tan Tang; Chang, W.H.; May-Keun Wong; Ying-Ying Shih; Te-Son Kuo, “The new design of an infrared-controlled human-computer interface for the disabled”, IEEE transactions on neural systems and rehabilitation engineering, vol. 7, Issue 4, Dec. 1999. pp. 474 – 481. [3] http://www.ritsumei.ac.jp/se/~tejima/theme-e.html [4] D. L. Jaffe. “An ultrasonic head positon interface for wheelchair control.” Journal of Medical Systems, vol. 6, no. 4. pp. 337-342, 1982. [5] Bergasa L.M.,Mazo M., Gardel A., Barea R., Boquete L., “Commands generation by face movements applied to the guidance of a wheelchair for handicapped people” ICPR 2000: 4660-4663 [6] Y.-L. Chen, S.-C. Chen, W.-L. Chen, and J.-F. Lin, “A head oriented wheelchair for people with disabilies” Disability and Rehabilitation, vol. 25, no. 6, pp. 249-253, 2003. [7] C. Mandel, T. Röfer, U. Frese, “Applying a 3DOF orientation tracker as a human-robot interface for autonomous wheelchairs’’, Proceedings of the 2007 IEEE 10th International Conference on Rehabilitation Robotics, June 12-15. [8] http://www.sparkfun.com/commerce/product_info.php?products_id=741 [9] http://www.cypress.com [10] Federación española de Deportes de Paraliticos Cerebrales, FEDPC. http://www.fedpc.org/upload/reglamentos/reglamento.doc