entertainment. Some of these applications require their users ... application, a âheavyâ box is carried by two participants that use Phantom [13] haptic devices to.
Evaluating Decorators for haptic Collaboration over Internet Shervin Shirmohammadi, Nancy Ho Woo DIstributed COllaborative Virtual Environments Research Laboratory (DISCOVERLab) School of Information Technology and Engineering University of Ottawa, Ottawa, Canada [shervin|nancy]@discover.uottawa.ca
Abstract One of the known problems with shared object manipulation in collaborative virtual environments (CVE) is the disruptive effect of network lag in collaboration sessions. It is widely recognized that delay and jitter cause significant problems for CVEs. Most solutions to this problem revolve around techniques to compensate for this lag at the networking level. More recently, the usage of visual queues indicating network lag to the user have shown to be effective. In this article we examine the latter for tele-haptic applications, and we find out an appropriate visual queue specifically for the low delay and jitter requirement necessary for closelycoupled collaborative tasks..
Keywords: tele-haptics, shared object manipulation, decorators, network lag, collaborative virtual environments, computer human interaction.
Introduction Shared object manipulation in Collaborative Virtual Environments has been a subject of interest for many years. In such environments, users immerse in virtual reality systems where they can manipulate objects, such as drawing the plans of a building or playing a game, in a collaborative manner. Applications of such CVEs range from telelearning and collaborative design to gaming and entertainment. Some of these applications require their users to perform tasks in a synchronous manner, sometimes in a closely-coupled form that requires precise coordination between the parties, who are connected to the network from geographically distributed locations. One of the problems, which has been studied and addressed to some extent in recent years for CVEs, is network lag. Shared object manipulation is achieved by sending each user’s interaction with the object to
other participants over the network. Because of network limitations and traffic conditions, some of these interaction updates are lost, or delayed. In reality, adverse network characteristics are present in any networked distributed application, and CVEs are no exception. Due to its requirements for closely-coupled tasks, synchronized collaboration is specially susceptible to delay and jitter. Therefore, special techniques and solutions are used in order to provide better quality of service for collaborative virtual environments and to help users collaborate in a more natural way in spite of the network lag. These solutions can be categorized into two main groups: networking-level approaches, and user-interaction based approaches. In this article, we combine both categories of solutions and apply it to a highly synchronous collaborative tele-haptic application, and we demonstrate that the said hybrid solution which combines both methods will give better results than each method on its own. In this article, we apply the general concept of “decorators”, as introduced in [2] specifically to tele-haptic applications for tightlysynchronous collaboration. Based on our subjective evaluations, we also find a novel characteristic of the user interaction-based approach, which we further demonstrate objectively.
Related Work Over the past few years, many studies have been performed to examine the effect of delay and jitter on synchronized tasks performed by users in such environments [3] [5] . It is generally agreed that in order to perform closely-coupled tasks in CVEs, an end-to-end delay of no more than 200 msec is required [6] . In addition it has been shown that jitter, which in simple terms can be thought of as the variation of delay, has a significant adverse effect on the quality of a collaborative session, being even more detrimental than delay: a 10 msec jitter can
result in a collaboration environment which is almost as bad as one with a 200 msec delay but no jitter [6] . As mentioned earlier, much research has been done to compensate for network lag in collaborative environments. Some of these studies concentrate on network loss [10] while others try to address the jitter problem [12] . Most of these existing approaches focus on solutions based on network communications, either at the transport-layer, the network-layer, or the application-layer in the form of framing of update messages [4] [8] [11] . These solutions use special techniques to reduce delay or to compensate for loss or jitter. Recently; however, researchers have begun to examine this problem from a computer-human interaction point of view, as opposed to a network communications perspective. One of these methods is the use of visual ornaments in the representation of an object in order to reveal the network lag to the users, so the users can be aware of the network lag [2] [7] [9] . The most recent work by Gutwin et al refers to these visual queues as Decorators [2] , which are graphical representations that visually inform the user about the delay or jitter; e.g., when delay increases or decreases these decorators change their color accordingly, sending visual feedback to the user about the condition of the network. In essence, instead of applying a compensation method at the network or transportlayer to deal with network lag, the idea of a decorator is to reveal to the users the presence and magnitude of the lag so that the users themselves can better understand the consequences and apply their own coping strategies in a more intuitive way. In this article, we deploy the concept of decorators in the context of a tele-haptics application that requires closely-coupled collaboration among its users.
The Collaborative Application
Tele-Haptic
A snapshot of the application used for evaluation is shown in figure 1 on the next page. In this application, a “heavy” box is carried by two participants that use Phantom [13] haptic devices to interact with the CVE. The participants’ objective is to complete a task whereby the shared object, the box, is to be moved to a predetermined target destination. Haptic devices were used both to represent the heaviness of the box by means of force feedback, and to enrich the overall user interaction since experience has shown that force-feedback devices, which will give a sense of “touch” to the user by applying physical force, further improve the quality of such object manipulations [1] .
As seen in figure 1 on the next page, each participant controls a virtual magnetic hook, which should be connected to predefined knobs on the box, in order to carry the box. The knob is the only position on the box that the magnetic hook attaches to, and it’s the only way to move the box. The box can then be carried by pulling or pushing on the magnetic hook while it’s in contact with the knob. The application was designed so that both users have to simultaneously lift the box, each at a different end, and together carry the box by smoothly moving it towards the predetermined destination. Once at the destination, the users have to fit the box within a fixed perimeter that is about 10% larger than the size of the box. Both the carrying task and the fitting task require closely-coupled collaboration. If one user moves the box too fast or too slowly compared to the other user, the box will fall due to its heaviness, simulating a situation that happens in the real world. The application is able to simulate network lag, where latency and jitter can be introduced to test more realistic network conditions. User errors, due to network lag, would cause the box to fall, which required the users to go back to the box and synchronously lift it and move it. These errors would increase the duration of task completion, which will be discussed in the experiment section.
Decorator Implementation Inspired by using the cursor as a decorator in [2] , the magnetic hook that each user controls was chosen as a decorator for this research. This is analogous to using the cursor as it’s the primary visual element controlled by the user. The coloring scheme of the hook, in presence of various network delay values, is shown in Figure 2 on the next page. The choice of colors was based on traffic lights: it was assumed that people intuitively perceive green as “ok”, yellow as “so-so”, and red as “not ok”; other colors in between have been chosen to complement the main colors. The above scheme is referred to as the multi color mode in this article. Based on our subjective results, which will be discussed in the analysis section, we also came up with a 2-color mode: black indicating no network lag, and red indicating that there is network lag. The justification for such coloring scheme is as follows: Due to the closely-coupled nature of collaborative tasks in the application, with the low delay and jitter requirements discussed previously, there is very little time for one user to react to the actions of another user. If the user’s reaction to the
box movement is not fast enough, the box will fall as mentioned before. Therefore, we hypothesize that what is crucially important to the user is the knowledge that there is network lag, and not how much lag. When the user notices the visual information about network lag, he/she reacts by briefly pausing. This gives enough time for the application to get back in sync at both ends, allowing the shared object to be perceived correctly by both
box
users. The actual color of the decorator, which indicates the amount of delay, will not affect the user due to the very short duration of reaction time. So the existence of a color change indicating network lag, without indicating the actual network delay, is sufficient. In this experiment, we also introduced a 2color decorator that would indicate lag if the delay between 2 consecutive update messages exceeds its natural rate by 10 msec or more.
knob
hook destination
Figure 1. Tele-haptic application
Figure 2. Colour scheme for the decorator
Performance Experiments Initially, two types of testing were conducted: the first was without the aide of a decorator, and the second was with a decorator. In addition, both the multi-color and the 2-colr decorator scheme were deployed and tested. A set of network configurations were used in each type of testing. Because we are dealing with highly-synchronized tasks in a CVE here, delays of 0 to 300 milliseconds were tested, with jitter values of 5, and 10 msec. With the exception of some 500 msec tests to test the thresholds of the application, higher delay or jitter values were not tested. Although some literature have tested higher delay and jitter values, those have been done for 2D and cursor-based collaborations, and not for CVEs with closely-coupled tasks such as the said tele-haptic application. The sending module of the application was designed to be able to simulate a delay of X milliseconds with a jitter of Y milliseconds. In a given collaboration session, each user will run his/her instance of the application with the same network configuration to ensure the network characteristics are symmetric.
Eight (8) sets of trails were performed, each using two users as described before. Each set contained 3 tests: no decorator, multi-color decorator, and 2-color decorator. The users were first trained until they were comfortable with the application. In addition, the system was tested under X=0 and Y=0 (ideal situation) to ensure the users were able to work flawlessly under the absence of network lag. Each trial took about half a day, including the resting time in between tests. These resting times were placed between each pair of tests in order to ensure the subjects were not physically or mentally tired after the first few tests. In total, there were 24 distinct tests performed. As we shall see next, the use of decorators does indeed improve user collaboration in the application. This is specially true for cases with very higher delays, where the absence of a decorator renders the collaboration impractical. In addition, we will see that the 2-color decorator is in practice as effective as the multi-color decorator. These results are discussed next.
Results Task completion time was the primary measurement of this experiment. This is the duration, in seconds, that it takes for 2 users to complete one
test. For each test, the amount of time taken to complete a task was measured. Corresponding tests from each of the 8 trials were then averaged to give a mean completion time for that specific test point. These results are shown in figures 3 to 5.
60
55
Decorator (multi color) No Decorator
Task Completion Time (sec)
50
Decorator (2 colors) 45
40
35
30
25
20 0
100
200
300
500
Delay (m sec)
Figure 3. Task Completion Time under UDP (jitter = 5 msec)
40
Decorator (multi color) Task Completion Time (sec)
No Decorator 35
Decorator (2 colors)
30
25
20 50
100
200
Delay (m sec)
Figure 4. Task Completion Time under UDP (jitter = 10 msec)
Analysis
Looking at figures 3 and 4, we can see that using decorators yields shorter completion times compared to not using decorators, which in turn can be interpreted as decorators having a mitigating effect on the users’ perception of network lag. Note that for a given delay value, there is not a very huge difference in completion time between different tests, and this is in fact consistent with observations made in [2] . Nevertheless the difference is significant enough to justify the usage of decorators. Another interesting observation that can be made from figures 3 and 4 is that in each figure, the performance of the 2 color decorator is “comparable” to that of the multi color decorator and has practically the same effectiveness when compared to the performance without the decorator. It can be concluded that due to the highly-synchronized nature of the application, the color change in itself is a sufficient visual indicator to the user.
Conclusion Using decorators as network lag mitigation tools were studied in the context of tightly-coupled collaborative tasks in virtual environments. A telehaptic application with such characteristics was developed as a test bed for performance evaluation of this method. It was shown that using decorators improves the overall collaboration quality of such applications. This can be of special interest to developers of collaborative applications in VR. We also hypothesized and objectively demonstrated that under low delay and jitter conditions, as required by closely-synchronized CVE applications, a 2 color decorator has practically the same effect as a multi color decorator. This was attributed to the user’s high levels of concentration and his/her need to take quick reactions in the presence of network lag, without being interested in the actual value of the lag.
Acknowledgments The authors acknowledge the financial support of Natural Sciences and Engineering Research Council of Canada, the design and implementation contributions of Stephane Dodeller, and the experimentation contributions of Sara Alavi and all volunteers for this project.
References
[1] Jung Kim et al, “Transatlantic Touch: A Study of Haptic Collaboration over Long Distance”, Presense, Vol. 13, No. 3, June 2004, pp. 328-337. [2] C. Gutwin et al, “Revealing Delay in Collaborative Environments”, Proc. ACM CHI 2004, pp. 503-510. [3] D. Wang et al, “The Effect of Time Delays on TeleHaptics”, Proc. IEEE HAVE 2003, pp. 7-12. [4] G. Kessler, and L. Hodges, "A Network Communication Protocol for Distributed Virtual Environment Systems", Proc. IEEE Virtual Reality Annual International Symposium, 1996, pp. 214-221. [5] K. Hikichi et al, “The Evaluation of Delay and Jitter for Haptics Collaboration over the Internet”, Proc. IEEE Globecom 2002, pp. 1492-1496. [6] K. S. Park and Robert V. Kenyon, "Effects of Network Characteristics on Human Performance in a Collaborative Virtual Environment", Proc. IEEE Virtual Reality, Houston, March 1999. [7] I. Vaghi et al, “Coping with Inconsistency due to Network Delays in Collaborative Virtual Environments”, Proc. ACM VRST ’99, pp. 42-49. [8] L. Gautier et al, "End-to-End Transmission Control Mechanisms for Multiparty Interactive Applications on the Internet", Proc. IEEE Infocom '99, New York, U.S.A, 1999. [9] M. Fraser et al, “Revealing the Reality of Collaborative Virtual Reality”, Proc. ACM Collaborative Virtual Environments, San Francisco, California, United States, 2000, pp. 29-37. [10] S. Shirmohammadi and N.D. Georganas, "An End-toEnd Communication Architecture for Collaborative Virtual Environments", Computer Networks, Vol. 35, No. 2-3, Feb. 2001, pp. 351-367. [11] S. Shirmohammadi and N.D. Georganas, "Collaborating in 3D Virtual Environments: A Synchronous Architecture", Proc. IEEE Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE 2000), NIST, Maryland, U.S.A., June 2000, pp. 35-42. [12] S. Dodeller and N.D. Georganas, “Transport Layer Protocols for Telehaptics Update Messages”, Proc. 22nd Biennial Symposium on Communications, Queen's University, Canada, May 31 - June 3, 2004. [13] T. Massie and K. Salisbury, “The PHANToM Haptic Interface: A Device for Probing Virtual Objects”, Proc. Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems, 1994.
