International workshop on physicality and tangibility in interaction, i3 annual conference, Siena, Italy, 1999
MOVY - a sourceless and wireless input device for real world interaction
Peter Henne, Thomas Schardt, Eckhard Meier, Wolfgang Broll German National Research Center for Information Technology Institute for Apllied Information Technology (FIT.CSCW) Schloss Birlinghoven, D-53754 Sankt Augustin, Germany Tel: +49-2241-14-2717 Fax: +49-2241-14-2084 EMail:
[email protected]
Keywords: Inertial tracker, tangible, input devices
Abstract The number and computational capacity of portable computers or embedded systems have grown extremely during the last years. Since these systems massively influence our every day life, new intuitive interaction techniques are needed, going far beyond common input devices such as mouse or keyboard. Adapted to people's natural behavior, these techniques must enable the users to focus on their intrinsic tasks instead of distracting them. In this paper we present MOVY, a miniaturized sourceless and wireless device for the detection of position and orientation. Due to its spatial resolution MOVY can be used to realize high accuracy input devices, such as needed for gesture analysis or universal pointing devices. Additionally the great range of MOVY provides also support for applications in the field of pose or position detection. In contrast to conventional tracking systems, MOVY is unsusceptible for magnetic fields or optical occlusions and can easily be combined with portable systems. Mounted on a ring or worn in a pocket its usage is very simple and does not hinder the user.
International workshop on physicality and tangibility in interaction, i3 annual conference, Siena, Italy, 1999
International workshop on physicality and tangibility in interaction, i3 annual conference, Siena, Italy, 1999
MOVY - a sourceless and wireless input device for real world interaction Peter Henne, Thomas Schardt, Eckhard Meier, Wolfgang Broll German National Research Center for Information Technology Institute for Apllied Information Technology Schloss Birlinghoven, D-53754 Sankt Augustin, Germany EMail:
[email protected]
Keywords: Inertial tracker, tangible, input devices
1. Introduction Natural human behavior, activity and communication is coined and influenced by the physical and social environments people live in. From childhood on, we are sensing, experiencing and acting in our real world. We become familiar with our surroundings and intuitively know how to interpret and react on different situations. Moreover, not only acting is based massively on the perception of our world, communication is also. Cultural and social settings largely affect our communicative and collaborative behavior that is an essential part of our everyday life. Implicit contextual and gestural information are important for cooperative processes, in which we are involved every day. The situation immediately changes if we are using common computer systems that are based on the generic screen, keyboard and pointing device. The world of computer programs is completely synthetic and separated from reality. Natural behavior is replaced by human computer interaction, that most often is reduced to pointand-click and typing activities. The handling of these systems completely differs from people’s natural behavior. People’s elemental abilities and habits, important in their everyday life, are ignored and replaced by artificial interaction techniques. Users have to adapt their activities to the user interface, instead of the system adapting to the user. Moreover, because of the single user character of these systems, special facilities for communication and collaboration have to be made available explicitly by groupware applications. The MOVY tracking device enables us to partially overcome this kind of constrictive human computer interaction. The retrieval of spatial information allows us to directly integrate real world actions of the user in our applications. Consequently, the user is no longer bound to the interface and hence, gains additional flexibility and degrees of freedom. Our goal is to make the computer system vanish in the background, thus users can better focus on their work without beeing deflected by the computer interface. The detection of the position and activity state of people by using the MOVY device, in combination with basic gesture recognition mechanisms is our basic approach to realize an externalized real world interface, that allows users to corporately work within their physical environment.
2. The MOVY Inertial Tracking System MOVYs are a class of self-contained orientation and movement trackers, which incorporate the features of sourceless operation and wireless communication with the host system. A MOVY system in principle consists of two components: the MOVY Sender and the MOVY Receiver. The MOVY sender is a miniaturized circuit, containing different sensors for data acquisition, and a radio sender to transmit the data to the MOVY Receiver. The receiver unit transfers the incoming data via a RS-232 interface to the host system, where the data can be processed. The usage of a radio link offers the advantage of a reliable and fast connection between sender and receiver. The choice of the sensors used to realize a sourceless data acquisition, is closely related to the application. All devices developed so far, are designed as tangible input interfaces. Since human’s movements rather consist of fast short-term actions instead of smooth long-term actions, we can take advantage of the fact that the position of a moving object can be derived from its acceleration. In addition, also the orientation of an object according to two of the three coordinate axes in space can be determined, if the gravitational acceleration of the earth is taken into account. Consequently the developed MOVYs are based on Figure 1: A naked MOVY prototype accelerometers. The more complex MOVY devices additionally contain angular rate sensors and/or an electronic compass (fig. 1). This approach in principle differs from commercial tracking systems [2,7], which use an artificial magnetic field to determine object positions.
International workshop on physicality and tangibility in interaction, i3 annual conference, Siena, Italy, 1999
2.1 Orientation tracking and the 3-DOF MOVY The smallest developed MOVY system is an orientation tracker with two degrees of freedom (2-DOF-devices). As mentioned above, it contains two accelerometers measuring the gravitational force g. These sensors are orthogonal oriented according to the x- and y-axis of the world coordinate system (fig. 2). Thus one accelerometer is used to measure MOVY’s rotation around the x-axis (pitch) and the other one is used to measure the rotation around the y-axis (roll). If no additional acceleration in x- or y-direction is present, pitch and roll can be computed using the following equation:
? ? asin
g
G
(Equation 1)
Figure 2: World coordinate system
0
Where ? is the angular excursion (pitch or roll), g is the force measured by the sensor and G0 is the earth’s gravitational force. Obviously, the sensors must be calibrated once to get 1.
G0' , the sensor measured gravitational force:
G0' ? g ; for roll = pitch = 90°
2.
g 0 , the sensor measured idle state force:
g 0 ? g ; for roll = pitch = 0°.
The output voltage of the accelerometer is directly proportional to ? . A micro controller converts these voltages and sents them by a small transmitter to the MOVY Server, which transfers them via the RS-232 interface to the host. The dynamic response of the sensor is 50 measurements per second at a resolution of 0.4 degrees (10 Bits). After a precise calibration the accuracy of repeatability and the absolute error are less than 1 degree. A problem occures, if angles greater than 90 degrees should be detected. Due to the ambiguity of the detected values, the position of the tracker cannot be determined clearly. To solve this problem, the host application has to log the sequence of received data and must handle the axis crossing. Due to the described limitations and the fact that the 2-DOF MOVYs cannot sense rotations around the z-axis (yaw), they can only be used for very easy tracking tasks. To enhance MOVY to a 3-DOF orientation tracker, the system must be equipped with an additional sensor to detect rotations around the z-axis (yaw). Therefore two different types of sensors can be used. The straightforward approach is the usage of an electronic compass, which detects absolute positions in the magnetic field of the earth. A disadvantage of this device is that the frequency of data acquisition is limited to 5 Hz, which is to low for many applications. Using an angular rate sensor (gyro) instead of a compass, the yaw angle can be obtained by the one-time-integration of the output signal of the gyro. Though the detection frequency of the gyros is 50Hz, the problem of long-term zero-drift arises: In idle state, the sensor produces an output signal different from zero. Depending on physical factors, like temperature, the frequency and amplitude of this signal varies. Since the integration of the output signal leads to an integration of the error, the system starts drifting. This drifting can be suppressed by adding a high-pass filter with a low cut-off frequency, but this again limits the resolution and leads to additional errors. In consequence the frequent recalibration of the system, i.e. with a compass, is necessary. A first 3-DOF MOVY was developed using two accelerometers, oriented in x- and y-direction for the detection of pitch and roll, and a compass, using two magnetic sensors in x- and y-direction for the measurement of yaw. It should be noted that yaw cannot be determined if the angles of pitch or roll are 90 degrees, and has low accuracy for pitch or roll near 90 degrees. A third magnetic sensor could solve this problem, but leads also to a larger size of MOVY. 2.2 Short-range navigation and the 6-DOF MOVY New questions arise when trying to enhance the 3-DOF orientation tracker to a sourceless 6-DOF device. In a first attempt we have used three gyros and three accelerometers, each of them oriented according to the three coordinate axes. The basic working principle is to use the gyros for the orientation tracking and the accelerometers for position tracking. However, the tasks of the different sensors are interweaved. As described above, the acceleration detected by the accelerometer is a combination of the earth’s gravitational/magnetic field and the acceleration acting uppon the sensor due to sensor movements. For the computation of the movement, the exact orientation of the sensor must be known. This information can be obtained by analysing the output of the gyros. We have also made experiments to eleminate the gravitational influence of the accelerometer signals without the use of gyros, using the response of the idle accelerometers to determine the position of the tracker. However, this method did not produce satisfying results, because a movement of the tracker, after a successful calibration, changes also the trackers orientation. Since the calibration is orientation-depended, it becomes invalid and must be done again.
International workshop on physicality and tangibility in interaction, i3 annual conference, Siena, Italy, 1999
However, the gyros also need to be recalibrated frequently, using the output of the accelerometers when MOVY is idle (no changes in position or orientation). Because the calibration of the roll-gyro cannot be done using the accelerometers, an additional compass is needed for this task. To get an impression of the required accuracy of the computations, we assume a constant maximum error (resolution, drift, movement) of an accelerometer to be e = 0.01m/s2, i.e. one milli-G. With increasing time t this error accumulates to a maximum error displacement D = 1/2et2. After 1 second the value of D is only 0.005 meters, but after 10 seconds D increases to 0.5 meters! Even if we know that this is really a worst case assumption, we have to conclude that the values for the position have to be recalibrated every few seconds. So problem 2 is how to recalibrate the location-values of a 6-DOF-MOVY.
3. Sample Applications A number of applications based on conventional input devices can take advantage of the MOVY. In particular application areas that are related to spatial or gestural information representations can benefit by this kind of input device. In this section, we describe a set sample applications that aim at the extension and simplification of conventional interaction and communication techniques. 3.1 The MOVY-Ring A simple modification of the MOVY is the MOVY-Ring. By using only two sensors of the MOVY and externalizing them on a ring allows us to directly record the bending of the users forefinger. The ring is connected to the MOVY Sender by light cables, which are strapped to the wrist of the user. That way, we can easily determine the pointing direction of the user’s finger and use it as a straightforward pointing device. A sample application of the MOVY-Ring is the usage as an enhanced laser pointer device, e.g for PowerPoint presentations. The orator gets the opportunity to highlight important items accordingly to his speech by simply using his finger to control the pointer’s position on the projection screen. In addition, he easily can use it to control his slides and animations by gestures. Another field of application of the ring is to use it as an interaction device in Virtual Reality applications. In virtual environments the pointing direction of the participant can be visualized by a virtual beam heading in the direction of his forefinger. The closest virtual object hit by the beam automatically gets selected, and can be manipulated in further steps. The major disadvantage of solely using a two-sensor tracking device is the unknown position and orientation of the user himself. As long as the active person keeps staying at the same place, the ring works correctly as absolute pointing device. Conversely, if the person wearing the ring is walking around, only relative measurements of the finger’s direction are possible. To remove this drawback, an additional 6-DOF MOVY device is required to track the movements of the users hand. Since in most applications the user stays in front of the projection screen, appearing discrepancies can easily be compensated. 3.2 Distributed virtual environments. Several shared virtual environments based on the Internet have been established through the last years [1,3,6]. Most of them use visual representations of their participants by avatars. These virtual characters are placed at the current viewpoint of each user and in this way impersonate the real person. In consequence, every kind of activity or communication has to be indicated by the avatar of the particular person. In practice, the visualization of the user is rather inexpressive. Even though the visual appearance of the avatar is individually configurable, its static and fixed representation cannot give the impression of natural behavior between participants. To counteract on this problem, some virtual environments try to add additional signs of life to their avatars. In AlphaWorld [1] and Sony’s Figure 3: Communicating avatars in the Community Place [6] for instance, communication is visualized by SmallView application the appearance of typed text above the corresponding avatar. Other environments such as OnLive Traveller provide a speech-to-lips synchronization and a number of predefined facial expressions of the user. Most participants however, do not use such predefined actions, since they have to be activated explicitly. Nevertheless, people’s body language, gestures and facial expressions are essential elements for most types of communication and cooperative processes. The pure indication of presence is not sufficient. If people have a meeting or presentation in a 3D environment, it is important to get additional information about their actual state and mood. Within our multi-user VR toolkit SmallTool [5] (fig. 3) the MOVY device enables us to track user
International workshop on physicality and tangibility in interaction, i3 annual conference, Siena, Italy, 1999
movements and to map them into actions of the corresponding avatar. This is done by directly mounting the MOVY on the arms of the participants. The acquired data is analyzed subsequently and converted into gestures or activity state information such as sitting, standing or walking. That way, we easily can enhance the conventional static avatar representations by symbolic acting capabilities. 3.3 Conferencing Another way of using the MOVY is to support distributed communication in the field of conferencing applications. Audio conferences as well as simple chat applications in principle do not provide any kind of awareness information about its participants. In contrast, video conferencing applications additionally present a continuos stream of visual impressions, which implies a high bandwidth of data transmission and hence limits the number of participants. Further more, not all of the visual information provided by the video are relevant to the other conference members. For example, while following a presentation, it is not interesting to know where another listener is currently looking at. Additionaly in some applications the transmission of video is not desired, due to the privacy or anonymity of the participants. The conferencing prototype application developed in our institute is based on a static pictorial visualization of each participant. The MOVY again is utilized to track activities of each conference member. If a predefined gesture has been recognized, the picture of the participant is replaced by another one, showing this person in an analogue posture. This technique massively reduces bandwidth requirements, because only notifications about changing activities have to be sent.
4. Current Work In our current work we use the 6-DOF MOVY within the CAMELOT project [4]. The goal of this project is the development of the mobile collaborative augmented reality environment Virtual Round Table (VRT). The basic idea of the Virtual Round Table is the homogenous visualization of a synthetic scene within the real world workplace of the users. The projection of virtual 3D-objects into the real environment of the users is realized using light-weight see-through glasses. The user can manipulate virtual objects, using placeholder objects. Placeholder object are physical objects, such as cups, lighters or erasers, which act as physical representatives for virtual objects. Thus each user gets the impression of a workspace that includes both, physical as well as virtual objects. To realize the synchronization of the user’s view to the virtual scene, his or her viewpoint and viewing direction have to be tracked in real time. We use the 6-DOF MOVY for this task due to its significant advantages over existing 6-DOF tracking devices. Nevertheless further development of the MOVY is required to achieve a more precise tracking. Future enhancements include an increase in the number and type of sensors as well as a significant miniaturization. Further challenges include the development of a algorithm for automated calibration of the system.
5. Conclusions In this paper we have presented the design and some sample applications of the new inertial tracking device MOVY. The main features of the device are sourceless and wireless operation as well as the modularity of the design. Sourceless operation was obtained by using accelerometers, angular rate sensors and compasses for the acquisition of position and orientation data. To send the incomming data to the host system, we have used a radio-link. Taking advantage of the modularity of the design different devices have been developed, each of them adapted to the requirements of a certain application. Although MOVY has met the requested requirements, we also have recognized some problems in accuracy and in the combination of different sensor types. We have described how these problems can be fixed and have given an outlook of future application, in which we will use the MOVY.
References [1] ActiveWorlds. [www] http://www.activeworlds.com [2] Ascension Technology Corporation, Flock of Birds, 6 DOF magnetic tracker, [www] http://www.ascension-tech.com [3] Blaxxun Interactive Inc. [www] http://www.blaxxun.com [4] CAMELOT, A Collaborative Augmented Multi-User Environment with Live Object Tracking, [www] http://orgwis.gmd.de/projects [5] W. Broll: SmallTool - A Toolkit for Realizing Shared Virtual Environments on the Internet. Distributed Systems Engineering, Special Issue on Distributed Virtual Environments (1998), No. 5, pp. 118-128. [6] R. Lea, Y. Honda, K. Matsuda, and S. Matsuda. “Community Place: Architecture and Performance”. In Proceedings of the VRML’97 Symposium, ACM SIGGRAPH, 1997, pp. 41-49. [7] Polhemus Incorporated. StarTrack, UltraTrack, FastTrack 6 DOF magnetic trackers. [www] http://www.polhemus.com
