International Conference on Particle-Based Methods Particles 2009 E. O˜ nate and D.R.J. Owen (Eds) c
CIMNE, Barcelona, 2009
PHYSICS ENGINE BASED PARALLELISED DISCRETE ELEMENT MODEL S. M. LONGSHAW∗ , M. J. TURNER∗ , E. FINCH† AND R. GAWTHORPE† ∗ School
of Computer Science and Research Computing Services The University of Manchester Manchester, United Kingdom e-mail:
[email protected] web page: www.rcs.manchester.ac.uk
† Basin
Studies, School of Earth, Atmospheric and Environmental Sciences The University of Manchester Manchester, United Kingdom e-mail:
[email protected] web page: www.seaes.manchester.ac.uk/research/groups/basin/
Key words: DEM, Parallelisation, Physics Engine Abstract. Recently software libraries known as physics engines have begun to gain interest within the computing communities. They are designed primarily for use within the computer game industry but also currently find credence within real-time animation and robotics simulation. The technical requirements of a Discrete Element Model (DEM) are markedly similar to that of a gaming rigid body problem. A DEM needs to calculate which elements are in contact and integrate forces in time to avoid inter-body penetration accordingly. The contact detection techniques utilised in a physics engine are relatively complex and as a result, many DEM simulations do not make use of these optimised techniques. Physics engines provide a well documented and maintained software library upon which a researcher could build a complex rigid body simulation such as a DEM. This also has the effect that DEM simulations that utilise a known physics engine are being calculated using standardised code, something which is considered best practice amongst the computing community when dealing with well defined algorithms. This paper discusses the suitability of the modern physics engine architecture for calculating a DEM. It also details a novel distributed parallelisation technique to allow a physics engine to be used to calculate a DEM with more elements than the majority of current physics engines will allow. The technique is described and a pictorial example given.
1
S. M. LONGSHAW, M. J. TURNER, E. FINCH AND R. GAWTHORPE
1
Introduction
Discrete Element Modelling (DEM) (Cundall, 19711 ), utilises the physical interactions between a collection of disparate bodies, to simulate a larger system.It is particularly suited to simulation scenarios which are likely to develop discontinuities in the structure of its material or where the system is discritised anyway, such as with granular flow. DEMs require contact detection and the ability to solve the laws of motion. More complex models consider additional forces such as friction as well as other restraining forces. These requirements are similar to the problems faced by game designers when utilising physics. It is generally accepted by most Computer Scientists that, wherever possible, forming a well defined library, upon which more complex software is then built, is good practice. For example libraries such as OpenGL and DirectX are used to render graphics primitives. However, rigid body physics calculations are typically re-written by each researcher as required. It is proposed in this paper that in certain cases, currently available physics engines may in fact produce simulation results that are sufficiently accurate. Furthermore, by simplifying the process of implementing a rigid body based simulation, through using a physics engine, it is hypothesised that the complexity and physical correctness of the design of some models may increase. 2
Physics Engines
Physics engines are all designed to perform the same basic task of simulating a system of dynamic rigid bodies in a physically accurate way. Physics engines currently aimed at the gaming and animation sectors include NVIDIA PhysX2 and Bullet3 . Others such as ODE4 aim themselves more towards pure simulation, for example within robotics simulations due to its open sourced nature, transparency of function and accurate constraint solver. 2.1
Accuracy
The majority of physics engines are designed around the needs of the game developer; as such their chosen methods are typically iterative in nature wherever possible. This allows simulation accuracy to be adjusted according to the amount of processing time available. The majority of engines are transparent in the algorithms they utilise, with some allowing a choice to be made. A set of simple shapes such as boxes or spheres are often used while performing collision detection. This means a collision between two spheres can be as accurate as floating point numeracy allows, while a complex triangulated mesh will be tested for collision by creating a conglomerate of simple collision shapes in order to approximate it’s volume. For many applications including DEMs, a well devised collision shape conglomerate can provide adequate accuracy while improving calculation times significantly. Generally physics engines operate to single precision floating point, which for a well defined model can be accurate enough, however some engines such as Bullet3 also allow for double precision. 2
S. M. LONGSHAW, M. J. TURNER, E. FINCH AND R. GAWTHORPE
2.2
Suitability
The simplest DEM implementations typically utilise spheres, which in turn allow the use of easily programmed methods such as Euclidean distance checks. Integration is normally performed via a small fixed time-step integration scheme such as Verlet or RungeKutta Order Four (RK4). As DEMs grow in size to contain millions of elements, each conceptually connected to every other, these simple methods even when well parallelised, are too na¨ıve. Physics engines often operate in a three stage pipeline (figure 1). For a
Figure 1: The three stages of the general pipeline most physics engines adhere to. Stage one utilises an Axis Aligned Bounding Box Sweep and Prune algorithm to determine which pairs may come into contact. Stage two utilises this information to perform an exact geometry check against all pairs, typically by treating the check as a Linear Complementarity Problem (LCP). In stage 3 forces are calculated to ensure bodies do not inter-penetrate at the points found in stage 2, finally all forces are integrated in time.
particular DEM application the advantages and disadvantages of different physics engines need to be considered. Where ultimate individual body accuracy is not a requirement, but access to a well defined code base with which to build a complex physical scenario is desirable, the majority of current physics engines are appropriate. In situations where control over final numerical integration is paramount but contact detection is only to be performed on geometric shapes or simple convex hulls, then utilising the contact portion of a physics engine would prove beneficial. However there exists one major drawback with most physics engines in that they heavily restrict the scale of simulation (PhysX has a limit of 64,000 individual rigid bodies). Bullet theoretically allows for in excess of 1 million, although speed of execution is reduced. 3
Parallelisation of a Physics Engine
DEMs can exceed 1 million elements and have run times given in weeks rather than hours. A solution to the problem of limited scale within physics engines, is to parallelise the process over a distributed computing resource. When considering a physics engine, it can be considered to be a closed loop system; parameters are entered at the start of the loop, the three processing stages are completed and results are given at the end of the loop. One parallelisation technique is to intrinsically parallelise the methods utilised in the processing pipeline4 , but this requires the re-engineering of existing engines, so a more elegant solution is to split a large DEM into multiple smaller DEMs and utilise boundary conditions in order to make them behave as a whole. 3
S. M. LONGSHAW, M. J. TURNER, E. FINCH AND R. GAWTHORPE
In order to achieve this, we split the overall global DEM co-ordinates into smaller sub-DEM co-ordinates, the size of which is determined by how many elements we want in each sub-DEM. For example if we had 1,000,000 elements overall and access to 50 processors, then we would split into 50 models of 20,000 elements. As the overall DEM progresses, the local boundaries are updated at each time-step to reflect the current maximum and minimum positions of the elements it contains. To propogate interaction across the processes, a ‘Ghost’ copy of elements within a ‘Ghosting Zone’ are re-created in the neighbouring sub-DEM (see figure 2(a)). Contact forces generated at stages 2 and 3 of the physics pipeline (see figure 1) are copied from elements to ghosts and from ghosts to elements.
(a)
(b)
(c)
Figure 2: (a): DEM model splitting, enlarged view of ghosting zone, (b): Example DEM containing 298,102 spherical elements split into 16 sub-DEMs running via NVIDIA PhysX2 and (c): Resultant ghosts from (b)
4
Conclusion
This paper has provided an overview of the advantages and disadvantages of utilising a modern physics engine in the design of a DEM. It has also presented a novel parallelisation technique, which allows the limited scale of simulation enforced by most engines to be overcome, as well as providing a method for the parallel execution of a DEM being calculated using a closed loop blackbox physics engine. REFERENCES [1] P. A. Cundall. A computer model for simulating progressive large scale movements in blocky rock systems. Proc. Sympo. Int. Soc. Rock Mech., 129–136, (1971). [2] NVIDIA. Homepage of the NVIDIA PhysX engine, http://www.nvidia.com/object/nvidia physx.html [3] E. Coumans. Homepage of the Bullet engine, http://www.bulletphysics.com/wordpress/ [4] R. Smith. Homepage of the ODE engine, http://www.ode.org/ [5] P. Kipfer. LCP Algorithms for Collision Detection Using CUDA. GPU Gems 3, 723– 740, (2008). 4
