Integration of Physics Based Simulation with Haptic Interfaces for VR Applications Jinwook Kim, Irawati Sylvia, Heedong Ko, Hyoung-Gon Kim
Makoto Sato
Imaging Media Research Center Korea Institute of Science and Technology {zinook, sylvi, ko, hgk}@kist.re.kr
Precision and Intelligence Laboratory Tokyo Institute of Technology
[email protected]
Abstract We present a novel approach to integrate physics based simulation with haptic interfaces for virtual reality applications. Proposed approach categorizes rigid bodies into two set of the visual and haptic rendering groups. Dynamic states of most objects which belong to the visual rendering group are simulated according to the visual rendering update rate which is typically 60 Hz. Only the objects in the haptic rendering group are simulated in a higher update rate which is required for the stable haptic interaction. In our case, dynamic states of the haptic rendering group are updated 1KHz. In the sense that collision and contact responses for each group can be efficiently handled with respective simulation algorithms (the penalty vs. analytic methods), our approach does not only save the computational cost but improves the simulation stability. As a case study, we apply our approach to CAVE like environment.
1
Introduction
In order to deliver more immersive and realistic experiences in VR environments, various peripheral devices and software techniques have been developed. Among those trials, physics based animation and haptic interfaces have long been brought into focuses. Even though realistic animation generated in a physically simulated manner gains more interests in graphics area and film industry, it has been applied only in part to real time applications such as VR environments and interactive computer games due to its computational expenses. However recent research breakthrough and rapid growth of computing power enable interactive simulations of physics phenomenon and the application area is being enlarged to complicated systems such as articulated dynamic systems and fluid systems. Haptic interfaces are peripheral devices which are widely used in VR applications to add the sense of touch [1]. However most researches have been done to realize the contact feeling of only static objects like unmovable walls. On the other hand, typical VR applications are expected so as that users can manipulate and interact with dynamic objects which are governed by the laws of physics. One of the major difficulties in applying haptic interfaces to VR applications can be found in integrating haptic rendering process with physics simulation of dynamic objects. Most approaches used for non penetrating rigid body simulation are based on penalty methods [2] or analytic methods [3]. Analytic methods calculate the impulse between colliding rigid bodies and instantly change velocities of the bodies. Several researches have been reported to apply analytic methods to contact simulation of rigid bodies [4]. Note that collision responses are calculated by the collision impulse and these values are hardly realized with existing haptic interfaces. Table 1 Comparison of two approaches for collision/contact simulation Penalty methods
Analytic methods
Computation cost
Expensive
Moderate
Collision impulse
Approximated to a finite value
Infinite
Contact force
Continuous
Discontinuous
Preferred application area
Haptic rendering
Real time application
Published on the 11th International Conference on Human-Computer Interaction 2005, Las Vegas, Nevada USA, 22-27 July 2005
On the other hand, penalty methods introduce restoring forces when objects interpenetrate. Typically elastic spring models are used to calculate the restoring forces. In that case, if the rigidity of colliding objects is stiff, smaller time step to integrate the dynamics is necessary for stable integration. As a result, penalty methods are computationally expensive compared to analytic methods. However the penalty method approximates the collision impulse to be finite. Hence the methods can be successfully applied to a haptic rendering simulation which requires high update rate and a finite impulsive force at collision. Table 1 shows a comparison of two approaches. We propose an algorithm to separate dynamic objects in VR applications into a haptic rendering group and a visual rendering group. All the objects which collide or contact with an end effector of a haptic interface are categorized into a haptic rendering group. A visual rendering group consists of the other objects. Each group is governed by its own simulation algorithms and the dynamic states of each group are updated respectively. Physics simulation of objects in a visual rendering group is sufficiently performed according to a rasterizing update rate. If we assume that typical display systems are rasterized 60 times per second, penalty methods are prone to fail in simulating contact of very stiff objects because its integrating time step of 1/60 sec may be too large in this case. Hence we apply an analytic method to a visual rendering group. Also discontinuous property of generated contact force does not matter as long as it looks plausible. Haptic rendering groups can be simulated using a penalty method and also updated constantly in a rate of 1KHz to realize a natural sense of touch [5]. In the following sections, we discuss the physics simulation used in our case and an algorithm to classify objects dynamically into haptic/visual rendering groups. A case study of applying our approach to CAVE system is also presented. Finally, we summarize our results in a conclusion.
2
Physics Based Simulation
All the objects in VR applications are expected to obey the Newton’s physics law and be affected by gravity. Also the objects should not interpenetrate each other. Fluids or gaseous materials could be simulated to present users more natural and realistic experiences. However VR applications should be run in an interactive manner. Hence we focus only on rigid body simulation to reduce the computational complexity. We have developed VirtualPhysics, a library for realtime rigid body dynamics simulation. VirtualPhysics is designed for simulating behaviours of articulated rigid body systems in an interactive manner. Using VirtualPhysics, programmers can easily implement simulation based applications with moderate understanding of related physics and mathematics [6]. The simulation world consists of several systems, where the system is a minimal set of rigid bodies that should be managed together in the dynamics analysis. A system could consist of several rigid bodies and joints which connect adjacent rigid bodies. Then the physics module builds collision geometries for dynamic objects. Though bounding boxes of dynamic objects are generated for collision geometries by default, user can override the collision geometries with a set of predefined primitive geometries such as a box, sphere, capsule and terrain height field to represent the model in more details. In this paper, we will describe a condition of consistent collision as a contact. In general, a collision of rigid bodies that move toward each other can be handled by applying an impulse to the bodies. This results an instant change of velocities of the bodies. On the other hand, contact condition is explained with contact forces not impulse. The contact force influences the dynamic system in the next simulation period. To calculate the collision/contact responses, two major approaches of the penalty methods and the analytic methods are used.
2.1
Penalty Methods
Penalty methods introduce a repulsive force at collision points. A typical model for penalty methods is that the repulsive force to be proportional to the penetration depth. Hence it can handle both the collision and the contact condition within a same framework. The proportional gain, which is often considered as an elastic coefficient of the artificial spring, governs the behaviors of collision/contact response. To simulate hard contacts and collisions of fast moving objects, the elastic coefficient should be considerably high. This leads very stiff dynamics equations. Solving stiff dynamics has been discussed widely in the field of numerical analysis [7, 8]. One of the difficulties in integrating stiff dynamics equations using the explicit time steeping algorithms is a narrow stability region. For the
systems to behave stable, applying very small time step is inevitable and this approach is prone to be suffered from expensive computational cost. The other common way to solve this is using the implicit time stepping algorithms which require derivatives of the dynamics equations. Despite of these disadvantages, penalty methods are easy to implement and the resulting repulsive force can be considered to be continuous along simulation time steps. In this regard, penalty methods are preferred in an area of haptic interaction. To present a natural sense of touch, haptic interaction needs updating hatic feedback force hundreds times per second.
2.2
Analytic Methods
Analytic methods treat collision and contact response differently. We iterate simulation through time by integrating the first order differential equations representing the Newtonian dynamics. If the penetration of bodies is found and the relative velocities of the bodies are toward each other, then the iteration stops and the process of collision response changes the velocities of bodies instantly to prevent the interpenetration of rigid bodies. Discontinuous changes of the velocity are calculated from the impulse which is governed by the Newtonian restitution. If the colliding bodies move only in the tangential direction, the process of contact response is called to generate resting contact forces. The contact force can be efficiently calculated by solving the linear complementarity problem [4]. Analytic methods offer several advantages over penalty methods. It is computationally cheaper and produces right behaviors. However existing haptic interfaces can not generate a proper collision impulse. Also the hard contact condition can results an instant occurrence of contact force even with the moderate choice of the restitutional coefficient. These observations reveal that analytic methods are not adequate for implementing haptic interactions.
3
Dual Simulation of Visual/Haptic Rendering Groups
For visually plausible behaviours, integrating dynamics can be sufficiently performed every rendering frame and integration time steps are updated adaptively according to the visual rendering frame rate. However, in order to apply the results to haptic rendering, the update rate of simulation should depend on the required update rate of haptic interfaces. Absolutely the haptic rendering requires much higher update rate for natural sense of touch compared to the visual rendering. As a result, the haptic processing can suffer from expensive computational cost, if there are too many dynamic objects in the scene. Hence it will be more efficient if objects which are involved with a haptic rendering process are updated in a required update rate of 1KHz and the other objects are simulated in a rasterizing update rate of 60Hz. Also the collision/contact response algorithm for those objects can be efficiently chosen according to whether the collision impulse or the contact force should be presented using the haptic interface. Figure 1 shows a proposed simulation process. Key idea is grouping objects into two groups and updating dynamic states of those groups respectively according to the required update rate. At the first stage, dynamic objects are categorized into two set of haptic and visual rendering groups. The algorithm for grouping will be discussed in the following section. Then collision detection among objects in the visual rendering group is performed. If the collision and contact is found, the analytic method to solve the collision/contact condition is employed. The next stage is to advance the simulation time for objects in the visual rendering group by integrating the dynamics equations. In the Figure 1, ∆tv represents a time step used in the integration. Usually ∆tv can be safely chosen according to the rasterizing update frequency. Before advancing to the next step, the current simulation time is stored for future necessity. The following processes are done for objects in the haptic rendering group. Since our grouping algorithm guarantees that objects in the haptic rendering group can not collide with objects in the visual rendering group during the next visual simulation step ∆tv, processes of collision detection and response can be done only for the haptic rendering group objects. Moreover in most cases, a number of objects in the haptic rendering group is much bigger than a number of visual rendering group objects. Hence a considerable amount of computational cost can be saved. Processes inside the dashed line in Figure 1 are iterated with ∆th, a time step of the haptic rendering update rate. As the inner iteration progresses and an elapsed time reaches the visual rendering time step, our simulation flow gets out of the dashed line in the figure and repeats the overall iterations.
~ t
~ t
Figure 1 Dual simulation process Note that the overall iteration is performed with a frequency of 60Hz according to the rasterizing update rate and the haptic iteration is done 1KHz for stable operations of haptic interfaces. Since the visual update rate is considerably small compared to the haptic update rate, dynamics states of the visual rendering group objects can be assumed to change little during the haptic iteration process.
4
Categorization of Objects into Visual/Haptic Rendering Groups
In order to apply different update rate and collision/contact models to each dynamic object, we propose an algorithm to categorize rigid bodies into the haptic and visual rendering groups. We define the haptic rendering group as a set of objects that can possibly collide with an end effector of a haptic interface during the next simulation period of ∆tv. The visual rendering group consists of the other objects. At the initialization step, a bounding sphere including all the geometries representing a rigid body is defined for each body. When the body moves during a given period of time, then a swept volume of the bounding sphere is a cylinder with hemispheres at the ends. Figure 2 depicts a bounding sphere of the body and its swept volume of the bounding sphere. Then intersection tests between these swept volumes can determine possible collisions during the next simulation period. Hence at the grouping process, we perform intersection tests for the swept volume of the bodies representing an end effector of a haptic interface with the other swept volumes. The intersection test of two capped cylinders can be done by finding the closest points on the line segments and testing an intersection of bounding spheres at those points. Note that the length of line segments can be computed by velocities of the objects and the radius of a capped cylinder is same to the radius of a bounding sphere of the objects.
t = t0
t = t0 + tv
Figure 2 Swept volume of a bounding sphere
Figure 3 Grouping objects
In Figure 3, a body representing an end effector of the haptic interface is indicated as ‘E’. Using our algorithm, all the objects that are touchable by a haptic interface can be easily categorized into a haptic rendering group.
5
Case Study
In order to evaluate our approach, we make a simple application which simulates a dark room with some holes on the floor, balls and other toys, as shown in Figure 4. The goal of this game is to put the right ball into the right hole until all the holes contain the balls. The user can move the balls by controlling the end effector. When the end effector touches the object, the force feedback will be sent to the user. Besides that the user may also play with the toys to see and feel how the laws of physics work in our virtual environment. The fan is modelled using a revolute joint where a consistent torque is applied. Hence the fan is trying to rotate but the friction force will settle the system at a stable condition. The chain is modelled with several rigid bodies connected by universal joints. The chain shown in the right side of Figure 3 is modelled as a 22 degrees of freedom articulated system. Each joint is implemented using a universal joint which is a 2 DOF joint. The user can influence the movement of those objects by touching them using end effector. Their movement will change according to the force given by a user. We implement our virtual world in the CAVE like environment equipped with a SPIDAR 3 DOF haptic interface [9]. CAVE system is suitable to give users very wide angle of FOV(filed of view) and eventually a fully immersive experiences. However we found difficulties in integrating classical types of haptic interfaces which have a robot arm structure. SPIDAR which is composed of 4 strings attached to the corners of the CAVE and an end effector, provides scalability from the desktop size to the human scale size and does not interfere with users’ visible volume. Figure 5 shows our experimental environment. The dominant hand of the user grabs the end effector and controls its movement to experience the haptic feedback from the SPIDAR. The other hand holds the WAND to control the spotlight direction to help the user to see in the dark room. The system also provides a stereo graphics which is seen through the stereoscopic glasses. In our application, the average visual rendering frame rate is 60 Hz and haptic rendering frame rate is 1KHz. We found that by using different update rate for visual and haptic rendering, our system works more efficient. It can produce a realistic physics based simulation and also a continuous force feedback to the user. We have demonstrated our simulation to some visitors. Most of them are interested with our demo especially in playing with the dynamic objects, such as fan, chain, and torque balance, to influence their movement direction and feel the contact force with those objects. They were immersed into the virtual environment and felt that the interactive physics based simulation is more realistic and natural. The sense of touch also helped the user to control the movement of the objects.
Figure 4 The physics based haptic simulation
6
Figure 5 The experimental environment
Conclusion
We present a methodology for integrating physics based simulation with haptic interfaces for virtual reality applications. Physics based simulation can give users more natural and realistic behaviors of rigid bodies. However an interactive simulation of the laws of physics for typical VR applications requires a considerable amount of computational cost. Moreover haptic interaction demands higher update rate of hundreds of frames per second. To be efficient in the physics based simulation with haptic interfaces, our approach categorizes virtual objects into haptic and visual rendering groups. Dynamic states of most objects that belong to the visual rendering group are calculated according to the visual rendering update rate which is typically 60 fps. Only the objects in the haptic rendering group are simulated as frequently as the stable haptic interaction requires. In our case, dynamic states of haptic rendering group are updated 1KHz. In the sense that collision and contact responses for each group can be efficiently handled with respective algorithms, our approach can not only save the computational cost but also improves the simulation stability. Our approach could be successfully implemented in a CAVE like environment.
7 1. 2. 3. 4. 5. 6. 7. 8. 9.
References William, M., Randolph, S., Finch, M., Verth, J. V. & Taylor II, R. M. (1996). Adding force feedback to graphics systems: issues and solutions. Computer Graphics: Proceedings of SIGGRAPH. Terzopoulos, D., Platt, J. C. & Barr, A. H. (1987). Elastically deformable models. Computer Graphics: Proceedings of SIGGRAPH. Hahn, J. K. (1988). Realistic animation of rigid bodies. Computer Graphics: Proceedings of SIGGRAPH. Baraff, D. (1994). Fast contact force computation for nonpenetring rigid bodies. Computer Graphics: Proceedings of SIGGRAPH. Colgate, J. E., Brown, J. M. (1994). Factors affecting the z-width of a haptic display. Proc. IEEE Intl. Conf. on Robotics and Automation. Kim, J. (2005) Virtualphysics, the realtime dynamics simulation library. Retrieved January 5, 2005, from http://virtualphysics.imrc.kist.re.kr Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1996). Numerical recipes in C (2nd ed.). Cambridge: Cambridge University Press. Negrut, D., Haug, E. J., & German, H. C. (2003). An implicit Runge-Kutta method for integration of differential algebraic equations of multibody dynamics. Multibody System Dynamics, 9(2), 121-142. Sato, M. (2002). Development of string-based force display: SPIDAR. VSMM 2002.
