Anatomically-based models for physical and geometrical ...

Anatomically-based models for physical and geometrical reconstruction of animals Victor Ng-Thow-Hing November 7, 1998

Contents

0 Overview 1 Introduction

1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . 1.1.1 Scope . . . . . . . . . . . . . . . . . . . . . 1.2 Problem Statement . . . . . . . . . . . . . . . . . . 1.3 Restrictions . . . . . . . . . . . . . . . . . . . . . . 1.4 Contributions and Results . . . . . . . . . . . . . . 1.4.1 Muscle shape primitive . . . . . . . . . . . 1.4.2 Lagrangian Physics formulation . . . . . . . 1.4.3 Biomechanical Muscle Model . . . . . . . . 1.4.4 Development of System . . . . . . . . . . . 1.4.5 Skin Synthesis . . . . . . . . . . . . . . . . 1.5 Chapter synopsis . . . . . . . . . . . . . . . . . . . 1.6 Statement of Proposed Results and Contributions .

2 Previous Work

2.1 Anatomy Background . . . . . . . . . 2.2 Geometric Deformations . . . . . . . . 2.2.1 Volume-oriented deformations . 2.2.2 Surface-oriented deformations . 2.2.3 Feature-oriented deformations . 2.3 Physically-based Deformable Objects . 2.4 Articulated Figure Animation . . . . . 2.5 Biomechanical Muscle Models . . . . . 2.6 Chapter synopsis . . . . . . . . . . . .

3 B-spline solids

3.0.1 Lattice Design . . . . . . . . . 3.1 Operations . . . . . . . . . . . . . . . 3.1.1 Volume Computation . . . . . 3.1.2 Data- tting . . . . . . . . . . . 3.1.3 Point membership test . . . . . 3.1.4 Display acceleration techniques

. . . . . . . . . . . . . . . 1

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

3 3 3 3 6 6 7 7 7 7 7 8 8 8

8

8 9 9 9 9 10 10 10 11

11

11 11 12 12 14 14

3.2 Chapter synopsis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4 Modelling Musculotendons 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8

Numerical integration concerns . . . . . . . . . . Generalized Coordinates . . . . . . . . . . . . . . Intrinsic stability of a deformable mass actuator . Volume Preservation . . . . . . . . . . . . . . . . Shape Preservation . . . . . . . . . . . . . . . . . Contractile Forces . . . . . . . . . . . . . . . . . Collision and non-penetration forces . . . . . . . Chapter synopsis . . . . . . . . . . . . . . . . . .

5 Bones, Ligaments, Fatty-Tissue, Skin 5.1 5.2 5.3 5.4

Bones . . . . . . Ligaments . . . . Skin . . . . . . . Chapter synopsis

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

6 System Design 6.1 6.2 6.3 6.4

Core design . . . . . . . . . . . . . . . . . . . Base primitives . . . . . . . . . . . . . . . . . Application to Anatomical Modelling System Chapter synopsis . . . . . . . . . . . . . . . .

7 Results 7.1 7.2 7.3 7.4

Contracting Soleus Muscle . Shoulder reconstruction . . Horse motion . . . . . . . . Chapter synopsis . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

8 Conclusion and Discussion

. . . .

. . . .

. . . .

. . . .

. . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

15

15 15 15 15 16 16 17 17

17

17 18 18 18

18

18 19 19 19

20

20 20 20 20

20

8.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 8.2 Current Progress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2

0 Overview This document introduces a method for modelling vertebrate animals using a combination of solid primitives for muscle, tendon and ligament tissues and a software architecture that allows these elements to be dependent on each other physically and geometrically. The title of the thesis will be: Anatomically-based models for physical and geometric reconstruction of animals. There will be eight chapters to the thesis: 1) Introduction. 2) Previous Work 3) B-spline Solids 4) Musculotendon Model 5) Bones, Ligaments, Fatty Tissue, Skin 6) System Design 7) Results and 8) Conclusion and Discussion. The following text outlines the structure of presentation of the research completed for the thesis.

1 Introduction This chapter will introduce the problem of modelling animals in computer graphics and in physical simulation.

1.1 Motivation

The need for modelling animals has great creative appeal and utility for us. In particular, we will examine its potential impact in dierent elds of endeavour: anatomy, biomechanics, and the entertainment industry.

1.1.1 Scope This work attempts to make advances in several areas. We will demonstrate clear contributions the presented techniques have.

 Anatomy Anatomists have studied muscle by dissecting cadavers and recording measure-

ments of muscle ber length. A large disadvantage is that because the ber tissue is inactive, it is impossible to observe muscle contraction. The best clinical tool available is ultrasound imagery which is currently subject to noise and only allows a limited view of the process from one vantage point. We will demonstrate how we can reconstruct muscles from medical imagery or serially dissected specimens. In addition to the surface boundary of the solids, we explain how ber orientations are preserved, allowing new directions for the functional study of muscle. A geometric model that only has a boundary parameterization would not be capable of representing the internal bers in the volume. We will show how we can simulate muscle contraction while preserving overall volume. In addition, we demonstrate that the changes in ber orientation observed in our synthetic contraction simulations match closely those observed in ultrasound images of contracting muscle in vivo (in the living organism). Designing a modelling system that is closely modelled after true anatomy, allows reconstructions of an animal's visual appearance from its skeletal remains. For example, we can attempt to reconstruct the body shape or deduce the biomechanics of extinct animals from their fossils.  Biomechanics In biomechanics, the work in dynamic simulation has traditionally used piecewise line segments to model the major lines of action of muscle acting on skeletons. There are several de ciencies that we overcome. The site of tendon (the intermediary tissue between bone and muscle) attachment to the underlying bone is usually represented as a single point 3

of contact. We introduce B-spline solid models as true solid primitives that allow areas of attachment to be correctly displayed, capturing the correct situation that exists in nature. The axial curves of the solid model can have non-zero curvature, improving the accuracy of the geometric con guration of the muscle's lines of action. This allows moment arm characteristics to be accurately assessed. We represent muscle as a true solid that has its own mass properties and a measurable volume that is analytically exact. Linear models of muscle only attempt to model the longitudinal forces of muscle bers that are assumed to run in parallel. Actual muscle bers have a complex ber orientation where dierent groups of bers lie in dierent orientations. Linear models do not adequate capture this situation. By embedding a physical model for muscle force that can generate active contractile forces as a result of control inputs, biomechanists and neurologists can begin to experiment with motor control theories for human locomotion. Currently, such models can only be conducted with inaccurate linear force models or through analysis of measured human motion. A computer model allows the theorist to synthesize dierent control models for feasibility testing.  Entertainment Being able to model synthetic animals for lm or interactive entertainment has many advantages. Computer generated animals can be directly controlled by a human operator without the unpredictability associated with real animals. The inherent dangers of working with wild animals and potential hardships on the animal's health are eliminated. With knowledge of existing anatomical con gurations in animals, ctional creatures with similar body structures can be designed and animated with potentially greater realism. In the area of entertainment, more emphasis is placed on visual accuracy rather than physical correctness or muscle architecture. Nevertheless, our approach has several advantages over other methods for attempting to represent the anatomical features of animals. Previous models have focused on simulating skin deformations automatically from muscles with shape changes being based on deformation techniques that neither accurately depict the correct anatomical con guration of the body nor adequately capture the dynamic solid tissue nature of muscle. While this approach may be satisfactory for an initial visual approximation for animation, there are several phenomena which may be excluded: 1. Tendons and ligaments de ne shape When a skeletal muscle contracts to create motion, the contractile force is transmitted through to the tendons. These tendons can become stretched taut and can visibly protrude out of the skin. This is apparent when you clench a st or straighten out your ngers. The palmaris longus and exor carpi radialis tendons will be clearly visible (Figure 1). 2. Isometric contractions and negative work In animal motion, muscles often actively contract while exhibiting no limb motion. Visually, muscles appear to tense and sometime bulge, conveying to the viewer the anticipation or exertion of eort during a motion. Many schemes that simulate muscle deformations correlate the deforming of muscle to joint movement. However, since by de nition, isometric contractions do not result in any joint motion, no noticeable deformations will be generated. In reality, as a muscle contracts, there is compliance in the attached tendon. This compliance allows a muscle to change shape while the aected limbs are held stationary. 3. Inertial eects Many muscle models feature deformations that are exclusively a function of the kinematic state without consideration of time derivatives or mass properties. 4

flexor carpi radialis palmaris longus

Figure 1: The palmaris longus and exor carpi radialis tendons are visible when the skin is pressed tightly against them. As a result, stress waves[6] that propagate through muscle or local inertial eects that can cause muscles to demonstrate damped oscillations will not be reproduced with pure geometric models. This can be seen as \jiggles" that occur in the muscles after a sudden stop of a quick motion. 4. Collision and interpenetration Nature has designed muscles to be compactly placed on the skeleton of animals. As a result, numerous muscles often press against each other, surrounded by an envelope of fat tissue and skin. Consequently, shape changes in one muscle will result in pressure changes on surrounding muscles. Biomechanists have realized the importance of this phenomenon when they study compartmental syndrome (the injuries in muscle that are caused by the built up pressures of muscle tightly packed against each other). These geometric and physical consequences have not been modelled. Previously, muscle volumes have been allowed to intersect with each other. With the goal of visualizing these features and simulating the mechanics of musculotendon systems, this thesis develops a methodology and useful set of physical and geometric primitives that can be applied in the elds of anatomy, biomechanics and entertainment. The challenge will be to create a muscle model that can represent a large array of shapes and levels of complexity. Anatomy and biomechanics require a higher standard of accuracy than applications for entertainment purposes.

5

1.2 Problem Statement

Given these considerations, the problem can be stated as: Given a skeleton of an animal consisting of bone geometry, produce an additive process that allows the layering and attachment of muscles, tendons and ligaments to the underlying skeleton (Figure 2). For physical simulation, these elements can be made to exert force on the skeleton and undergo shape changes consistent with the behaviour of the material they are meant to model. This includes shape changes due to external forces or internal potential energies. In the case of muscle tissue, these forces are generated internally and we wish to be able to accurately model the muscle architecture ( ber orientation) as well as its volume preservation characteristics. The muscles produced should be controllable to the extent that a force model allows a control input variable to be speci ed that directly aects the active contractile force of muscle.

Figure 2: We would like to layer muscles on top of the skeleton.

1.3 Restrictions

To model the entire anatomy of an animal would be an extremely complex task. For our purposes, we are concerned only with modelling the elements of anatomy that are the causal agents for motion 6

and body shape de nition: skeletal muscle, tendons and ligaments. Less consideration will be given to fat and skin. Smooth (stomach lining) and cardiac muscle (the heart) will not be considered. The stiness of bone is measured by its Young's Modulus, which is about 1011dyn=cm2[1]. In contrast, tendon has a strength of around 9  108dyn=cm2, which is several orders of magnitude more compliant than bone. Consequently, bones will be treated as rigid objects.

1.4 Contributions and Results

The proposed solution to this problem is the design and implementation of an anatomically-based modelling system. A fundamental concern is to develop an integrated physical and geometric model for muscle. Muscles are the primary actuators of locomotive force in animals and also de ne the shape of the animals body. Previous work has only focused on one side of this dual nature of muscle. Biomechanists have constructed force models for linear muscle model without 3-D volume. In contrast, computer graphics has only considered modelling gross geometric eects of muscle deformation on skin with little concern for physical simulation. An important rst step to modelling both aspects simultaneously is to choose and de ne equations of motion for the parameters of a geometric muscle model while keeping in mind that the muscle must be able to collide and maintain non-penetration constraints with other objects.

1.4.1 Muscle shape primitive Consequently, we chose to model muscles as solids, instead of a boundary surface representation. All points that consist of the solid can be enumerated with a parameterization of a B-spline triple tensor product formulation. This representation has allowed us to model the internal ber arrangements of actual muscle, providing an important visualization tool and link for validation of our muscle with actual measurements from cadaveric specimens.

1.4.2 Lagrangian Physics formulation As the model's shape can be represented uniquely by a set of state variables, we developed a set of equations of motion derived from Lagrangian dynamics. This has provided a convenient way of decomposing the various forces into separate parts that can be adjusted. Volume-preserving forces can co-exist with external forces such as gravity and spring forces. Constraints can be applied to portions of the solid material to create point on plane or xed points constraints.

1.4.3 Biomechanical Muscle Model

The physical parameters of our solid primitive allow us to extend the range of applicability of the model from muscle to tendon to ligaments. This model for soft tissue can be used in a dynamic anatomical system. The well-known force-length-velocity relationship of the contractile element of muscle will be integrated into our three-dimensional volume model for muscle.

1.4.4 Development of System We have developed a software architecture that can allow the integration of all the bone, muscle and other soft tissue elements to create a simulated dynamical system. Consequently, a system was designed using object-oriented techniques to provide a exible environment to construct our dynamic anatomical systems. The system was co-designed with Petros Faloutsos and is referred to as DANCE (Dynamic Animation and Control Environment). 7

1.4.5 Skin Synthesis

The nal stage is to envelope the bone and muscle constructs with a skin model. The skin should be deformed by the underlying muscle motion and rigid motion of the limbs of the skeleton. The eects of the fat layer that lie underneath the epidermal and dermal layers of skin and over the musculature should be accounted for.

1.5 Chapter synopsis

The introductory chapter motivates the need for an anatomical modeller. The relevance of the work is explained in the context of anatomy, biomechanics and computer graphics. The problem statement and its scope is de ned. In particular, emphasis is placed on the current de ciencies of present and past techniques and our solutions for ful lling the desired goals.

1.6 Statement of Proposed Results and Contributions

This thesis provides new work in several areas. The development of a physically-based model for muscle will be introduced. The model has a solid parameterization that dierentiates it from traditional surface representations for muscle used in computer graphics. In addition, internal muscle bre orientations can be visualized within the solid model as well as interactive deformation and nesting of solids within each other. The proposed addition of biomechanical muscle force properties will allow these muscles to be physically functional and extends existing biomechanical force models from a one dimensional domain to a three dimensional solid. Finally, an extensive software architecture is presented that allows the integration of many anatomical elements into a common framework where they can interact physically with each other. This allows important and previously ignored aspects of anatomical modelling such as collision resolution between muscles and bones to be visualized.

2 Previous Work As this work will draw upon several dierent bodies of work, this chapter will be organized into several sections that review and explore the previous and current work that is relevant to dierent aspects of our problem. Before delving into the techniques that are used to model our anatomical system, it is necessary to include a background primer on anatomy.

2.1 Anatomy Background

A brief background on the major anatomical components will be given. These elements include the muscle, tendon, ligament, aponeurosis, and the skeleton. The composition and physical properties of these elements will be discussed followed by a review of our current understanding of their functional purpose and interactions between each other. For example, we will review what is known about the composition of muscle and some of the observed behaviour of this material under dierent conditions such as isometric contractions, isotonic contractions, and fully activated muscle. The connection between muscle, aponeurosis and tendon will be illustrated and examined. The concept of pennation and muscle architecture will be reviewed as well as its relevance to force production in muscle. Several papers that attempt to model muscle architecture will be reviewed at this time. 8

In this section, we will locate and describe a speci c muscle group that we worked on for our model. These muscles and tendons are the soleus, gastronemius and tendocalcaneous (Achilles tendon). This example will serve to illustrate muscle architecture concerns as well as providing a real anatomy context for our work to prevent the thesis from getting too abstract about muscle models. The unique shape and ber orientations of soleus and gastrocnemius will help readers to realize the instrinsic complexity of muscle tissue and remove some of the preconceptions of muscle shape. For example, muscle is often depicted incorrectly as having a belly and two tapered tendon ends.

2.2 Geometric Deformations

In order to simulate the changing shape of muscle and subsequent skin deformations, it is necessary to review the current techniques being used to model muscle-based deformations in computer graphics. The techniques we will cover will be divided into three categories: volume-oriented deformations, surface-oriented deformations and feature-oriented deformations.

2.2.1 Volume-oriented deformations The subsection will be a mini-review of free-form deformation techniques (FFDs) that are based on triple tensor product parametric spline formulations. We will discuss the relative bene ts and disadvantages between Bezier, B-spline and NURBs (non-uniform rational B-spline) basis functions. The dierent choices of lattices will be reviewed, such as the traditional cubic lattice, extended FFDs, and subdivision volumes FFDs. Applications of FFDs for animation such as in Chadwick's animation system[4] and the more recent derivative work in Kinetix's Character Studio called Physique. For the latter system, an attempt will be made to understand how physique works. I believe it is using some form of special FFD lattice.

2.2.2 Surface-oriented deformations This category includes all deformation techniques which involve the creation of a surface and the modulation methods for adjusting the surface's shape. This includes simple control point manipulation of parametric surfaces, and techniques that generate an isosurface from implicit surfaces. Examples of this work include Wilhelms and Ferdie Scheepers work on using implicit surfaces as muscle-approximating shapes, and Shen and Thalmann's[14] work on generating a parametric surface from the isosurface of implicit surfaces.

2.2.3 Feature-oriented deformations This work includes techniques which attempt to produce noticeable features in the surface that are qualitatively dierent from the general smoothness of a shape. For example, Singh's work on wire based deformations can add discontinuous surface characteristics like cusps. Recently, work with subdivision surfaces have allowed arbitrary topological surfaces to be modelled which are important for modelling appendages such as ears and noses. Wilhelms[17] has created mesh structures that attempt to model anatomical features by attaching skin points to underlying musclebased geometry. The work with hierarchical B-splines (H-splines) by Forsey[8] will be discussed.

9

2.3 Physically-based Deformable Objects

To motivate the development of a physical muscle shape primitive, this section will review the involvement of physics for modelling deformable shapes and their dynamic behaviour (i.e., shapes that take into account mass and forces). The Lagrangian Dynamics Formulation will be reviewed and serve as the common physics starting point for introducing several techniques for deforming and constraining viscoelastic eects in non-rigid bodies. The work on spring-damper systems will be reviewed, including discussion of applications on modelling various animals. The inherent de ciencies of such systems will be discussed and motivate a subsequent introduction of continuum mechanics that attempts to accurately model the solid mechanics of continous materials. Terzopoulos' elastic body work and Lagrangian formulation of solids will be reviewed. The linear dynamic formulation popularized by Witkin et al.[18] will be reviewed. Gascuel's soft body dynamics with implicit surfaces will be covered[9]. Other Lagrangian based formulations with FFDs will be mentioned. There will also be a discussion of volume preservation techniques and several dierent strategies that have been used to model such phenomena. Rappeport's work on constrained volume deformations, and Gascuel's local volume preservation work with implicit surfaces will be reviewed as well as potential energy strategies for volume. The area of collision resolution will be explored to determine what techniques have been used in the past for modelling deformable objects that reaction to collision detection. I will talk about Lagrange multipler techniques for linear constraints, constrained optimization with nonlinear constraints and projected constraint methods.

2.4 Articulated Figure Animation

Articulated gures have become a useful representation for modelling jointed animals. This section will brie y discuss the two major formulations for deriving the equations from articulated gures: reduced coordinate formulations and maximal coordinates. We will then talk about the current technique in interactive applications to bind a polygonal skin mesh with a weighted sum of the dierent coordinate frames of reference of these articulated gure skeletons. The discussion will then talk about the need for motion speci cation and review the three major strategies that have emerged. These are 1) spacetime constraint methods which attempt to adjust an initial trajectory in spacetime to minimize an objective function subject to physical constraints, 2) controller synthesis techniques that look for globally optimal motion solutions and 3) handcrafted controllers that are constructed to perform speci c motions. The use of joint torque motors in most of this work will be pointed out and lead to an inquiry into more biologically-motivated actuators.

2.5 Biomechanical Muscle Models

A review of the fundamental work of Hill, Zajac and perhaps Hatze for modelling muscle forces and their relationships to length, velocity, and activation will be conducted. Applications of these force models in simulation work in biomechanics will be explored. This work includes nite elements models such as those utilized by Chen and Zeltzer[5] and larger multi-segment models used by Pandy[12] and von Bogert[2]. We will also examine some anatomical systems from that use linear force models such as SIMM[7] and the topological modelling system of Kalra et al.[10]

10

2.6 Chapter synopsis

This chapters role is to make the reader aware of the vast literature that has been done for dierent aspects of our main problem statement in our thesis. In isolation, they do not provide a complete solution, but dierent techniques can be adopted and modi ed to provide the ingredients of an anatomically-based modelling and animation system. This chapter will serve to illustrate how a disparate set of ideas can be combined to create a powerful technique. Providing the global perspective of our problem helps the reader to place each piece of the problem in its proper context. This chapter has the potential to be very large and I may move the background material on anatomy and biomechanical muscle force models to a separate chapter.

3 B-spline solids This chapter focuses on the mathematical formulation and properties of B-spline solids. B-spline solids are the main primitive for muscle and tendon. A preliminary discussion of design considerations that lead us to choose B-spline solids will start this chapter. Notation that will form a large part of subsequent discussion in later chapters of the thesis will be introduced here and be consisently used throughout the thesis. We will explain the dierent components of the B-spline parametric solid equation, such as control points and parametric points. We will de ne these parametric points as material coordinates to be consistent with the continuum mechanics terminology. I may decide to introduce standard Einstein summation notation to prevent notational clutter. However, Einstein summation notation can be misleading when a reader rst encounters it and may confuse the reader as he or she attempts to follow the mathematical development of our model. To prepare a discussion of the Lagrangian formulation, I will use generalized coordinates to discuss two dierent choices of the state variables (control points and spatial points) for B-spline solids. The relative advantages and disadvantages of each will be discussed. The idea of spatial coordinates and material coordinates will be presented to avoid ambiguity when referring to particles of the solid. Spatial and material derivatives will also be presented in preparation for subsequent discussion with dierential equations of B-spline solids.

3.0.1 Lattice Design

The choice of indexing of the lattice of control points of the B-spline solid will in uence the family of shapes that we can model. We will illustrate how our B-spline solids can model cylindrical, toroidal, cubical, and spherical solids. For cylindrical and spherical shapes, equality constraints have to be enforced on control points of these solids. B-spline surfaces can be modelled as the degenerate case of a B-spline solid with only a 2-D array of control points. This illustrates that B-spline solids are a true generalization of B-spline surfaces and curves. The properties of solids which have multiple knot points will be considered. For example if multiple knot points are used at the ends of a knot vector, the outer boundary of a B-spline is equivalent to a B-spline surface and the volume can be completely determined by the boundary control points.

3.1 Operations

This subsection will explain several useful operations that have been developed that make the use of B-spline solids a powerful primitive for muscle modelling. Each operation will include sample uses of the technique with respect to building the anatomical modelling system. 11

3.1.1 Volume Computation

We derive the equations for computing the exact volume of B-spline solids. For solids that have multiple knots on the boundary, we can apply the Divergence theorem to compute the volume based only on the boundary control points. I will provide a constructive proof in the appendix on how the terms that involve internal control points reduce to zero. The volume computation has physical meaning as long as the B-spline solid does not intersect with itself. If self-intersection occurs, the requirement of maintaining a positive orientation on the boundary surface when facing the exterior region of the solid is violated. Consequently, negative terms appear in the volume formula. In fact, if the solid's boundary surface orientation is reversed, the volume quantity will be the exact negative of its positively oriented counterpart. The gradient of the volume will be de ned with respect to the control points and the spatial points. This technique will be compared to volume calculation of polyhedra meshes which can also be used to approximate the solid's volume. However, the B-spline solid volume formula solves the volume exactly and is the limit case of the polyhedral mesh method as the number of polygons increase and it approaches in nitesimal small polygon area elements.

3.1.2 Data- tting We have developed a multi-stage procedure for allowing us to specify an initial shape for B-spline solids by creating an initial con guration for the control points. This will include a description of tting solids from pro le curves and contour curves generated from the Visible Human dataset. The technique involves building a linear system that solves the control points based on sample points taken from our input data. This procedure will be illustrated by a detailed case study on tting actual anatomical data to the B-spline solids from two dierent sources: Visible Human axial (transverse) images (Figure 3) and 3-D digitized points taken from photogrammetric techniques on serially dissected muscles ~~~ CVSF(u,v,w) ~ a(w) ~ l(w)

~ c i (v) ~ l(w)

~ ci+1(v)

~ a(w)

P A

(1)

(2)

(3)

Figure 3: B-spline solids can be tted to contour curves extracted from transverse images of the Visible Human dataset. 12

(Figure 4). This discussion will serve to emphasize the importance of the initial quality of the input data and its in uence on the accuracy (external and internal) of the B-spline solid muscle t.

Figure 4: Specimen of serially-dissected soleus muscle There will be a discussion on the visualization of streamlines within the B-spline solid. This will demonstrate an important advantage of our use of a solid representation for muscle instead of a boundary representation as well as demonstrating the eectiveness of our technique to capture 3-D muscle bers within the volume of the reconstructed solid (Figure 5).

.

Figure 5: Visualizing streamlines within the soleus muscle model

A discussion of how the sample points are reused as spatial points will lead to the development of a restricted form of direct manipulation. In addition, these spatial points can also be given a mass value so that they can be used in a linear dynamics for physical simulation.

13

3.1.3 Point membership test

An important operation is being able to determine if a spatial point resides within a solid. Although a nonlinear root nding technique could be utilized, we prefer to use a nonlinear least squares technique for several reasons. If a point resides outside of the solid, the nonlinear least squares method will produce the point in the solid which is closest to the given point, giving us a convenient nearest-point operation. Our non-linear least squares technique can use line search methods that produce a globally convergent algorithm. In contrast, Newton's method can diverge. We will outline a nonlinear least squares technique based on the Levenberg-Marquardt method for solving this problem with superlinear convergence. Aspects of this problem involve suitable choice of an initial guess and constraining the search for valid material points to the valid parametric domain of the solid. This operation will be an important element of the collision detection and resolution process between two muscle and muscle-bone pairs. The point membership test enables B-spline solids to be nested within each other, similar to the embedding process with free-form deformations[13] (Figure 6).

Figure 6: B-spline solids can be nested within each other.

3.1.4 Display acceleration techniques

This subsection will outline a method of precomputing basis function products for very quick display and manipulation of B-spline solids. This involves tessellation of the solid for display and caching values to allow quick updates of the tessellation as control points or spatial points are adjusted. The same technique is applied to normal vector computation for lighting calculations and streamline visualization. Re-tessellation at dierent sampling densities allows a simple levelof-detail representation scheme for the solids.

14

3.2 Chapter synopsis

This chapter introduces the mathematical notation that will be used in the next chapter for developing physical equations of motion for the solid muscle. The major operations and properties of B-spline solids are described and will be the key ingredients for developing several subsequent algorithms.

4 Modelling Musculotendons This chapter focuses on the physics formulation of the B-spline solid for dynamics simulation. We review the Lagrangian dynamics formulation, including the complete general representation of the Lagrangian equations that include external forces, potential energy forces, and constraint forces.

4.1 Numerical integration concerns

Dierent types of numerical integration will be discussed from the straightforward explicit Euler method to implicit Backward Euler methods for handling sti equations. We discuss the results of our experiments with each of these techniques and the tradeos due to computation required for each method. In general, the implicit Backward Euler implementation can allow stinesses up to several orders of magnitude greater than explicit integration methods. However, more computational eort is required to provide the derivatives for the implicit method.

4.2 Generalized Coordinates

After the Lagrangian formulation is introduced in general, we cast this problem using two dierent interpretations of generalized coordinates: control points and spatial points. The dierences in physical interpretation of each of these schemes will be discussed, especially regarding the propagation of forces from the application point on the solid. The spatial points are less prone to stiness in the equations of motion because they can be made independent of each other. The local in uence of control points on a solid shape results in a larger propagation of forces from an application point to the surrounding material.

4.3 Intrinsic stability of a deformable mass actuator

The use of muscles as force actuators has several important consequences for the issues of control and the inherent stability of the dynamical system. The fact that muscles mainly exert force unidirectionally requires that they must work together with other muscles in antagonistic and synergistic groups. This allows muscles to stien joints of a limb. In addition, the mass properties of muscle change the inertial properties of the limb. In fact, they eectively make the rigid skeleton a non-rigid object with a changing centre of mass for the limb. As limb motion occurs, inertia will cause the mass points in the solid muscle to continue along their motion trajectory, altering the explicit control forces needed to control a limb because secondary torques and forces will be generated. The natural damping of the muscle may help to reduce natural oscillations in limb motion.

4.4 Volume Preservation

We illustrate how we use our equation for volume to compute a force based on the gradient of a quadratic energy equation. The rate of convergence to volume preservation can be controlled 15

by stiness and damping parameters. We will also explain how setting elements of the gradient volume vector to zero can allow other constraints to be applied to the solid while maintaining volume simulataneously. We discuss the robustness of this technique by examining the nature of the generated potential forces. An argument will be introduced over why exact volume conservation at every timestep may not be desirable. This leads to a comparison between iterative and single pass methods. The nonlinear equation for volume makes a single pass method is unlikely. Several strategies will be explained over how volume conservation at every time step can be maintained. We argue that if we can maintain a relatively low error tolerance, interactive rates can be achieved while preserving volume subject to other forces on the solid.

4.5 Shape Preservation

The shape preserving properties of the solids are clearly de ned by the stress tensor of the solid. We outline several strategies we employed for maintaining shape memory in the B-spline solids. This includes simple spring-mass networks, to stress-tensor derived potential forces.

4.6 Contractile Forces

Using a set of embedded springs in the solid, we can induce parameterized contractions in the solid while maintaining constant volume. In fact, we illustrate this with simulated contraction experiments performed on human soleus muscle and compare the results with ultrasound measurements of real soleus contracting in vivo (Figure 7). The spring-mass network method is desirable when exact transitions between rest lengths are desired.

Figure 7: Simulating muscle contractions in synthetic soleus muscle. For higher accuracy, we will develop a method using more complex serial-parallel viscoelastic units used commonly in biomechanical force models. We model the active contractile element of muscle using known relationships between length-shortening-velocity and force in muscle. The physical constraint of internal active forces not accelerating the muscle's center of mass is examined and accounted for. In addition, the dierent physical properties of adjoining tendon and aponeurosis are modelled and incorporated with the muscle dynamics. 16

We intend to verify our biomechanical model by reproducing classical experiments like isometric contractions, activation to tetanus, quick-release experiments and isotonic contraction. The challenge will be to nd the correct parameters for our model. The biomechanics literature provides some values, but those parameters may only be tuned for a linear force model. The geometric con guration of our model will be more complex and parameter searching may be required.

4.7 Collision and non-penetration forces

The point-in-solid operation that we developed previously can now be used for collision detection tests. As muscle groups can be located near each other spatially, we can restrict the number of object-object pairs to test as we can be certain that some muscle pairs will never come in contact with each other. This will accelerate the collision detection process. Perhaps a good data structure is a graph based mechanism or adjacency matrix. To meet our requirements of proper collision resolution between muscles and the underlying skeleton, we use the projective constraint method (reaction constraints) to project out undesired force components which would cause interpenetration situations. However, we handle the bonemuscle and muscle-muscle interaction cases dierently. As most bone geometry will be represented as polyhedral meshes, the contacting surface of a given spatial point on muscle with bone will be planar. We have formulated a point on plane constraint that will maintain non-penetration of the muscle into the bone while allowing slippage of the muscle along the bone. In addition, the proper reaction forces of the muscle acting on the bone are properly modelled. For muscle-muscle interaction, we need to determine the material coordinates of each participating muscle. The shared spatial position will now be subject to a moving point-to-point constraint that will be maintained by constraint forces. If the constraint forces are observed to be pulling the muscles together instead of pushing against each other, the constraint will be removed to let the muscle's drift apart. Slippage is possible because the contact point forces are recalculated at every time step. The quick point-in-solid test (Section 3.1.3) makes these checks relatively quick.

4.8 Chapter synopsis

This chapter is an exposition of the physical model of muscle and how it ts into the common Lagrangian dynamics framework. The biomechanics-derived force models are included in this chapter to tie together the continuum mechanics and muscle force models as an integrated unit.

5 Bones, Ligaments, Fatty-Tissue, Skin This chapter rounds up the modelling of the other secondary tissues of the anatomical modeller.

5.1 Bones

The bones are assumed to be fairly rigid. We show how we calculate the volume, mass and moments of inertia using algorithms that integrate over the bone's surface[11]. We can perform an eigenvector analysis on the resulting inertia tensor to get a new inertia tensor that only has the principle moments of inertia on the diagonal. The object axes are realigned to the main eigenvectors of the matrix.

17

The constraints for joints are discussed. The physical simulator we are using implements a reduced coordinate method that allows various joint to be modelled. We will detail how we implement the shoulder joint, which is a complicated four link closed model. The dierent articulation points of this joint will be discussed.

5.2 Ligaments

The collagen and elastin components of ligament tissue allow a degree of compliance and add joint stability. We will show how we model the stress-strain behaviour of ligaments. It is most likely we can use a linear spring or a linear spring with dierent stinesses at dierent strain levels.

5.3 Skin

The exact choice of skin geometry has not been selected yet, but we are contemplating several choices.  B-spline sheets A set of B-spline patches can be attached to the muscles and stitched together to form a quilt like skin. This method is relatively easy to implement, but may provide problems at the seams.  Subdivision surfaces With the existence of parameterizable subdivision surfaces[15], we can attempt to generate a set of points over the entire body that can form a large linear system to solve for the control points of the subdivision surface. More points can be placed in areas that require ner detail or higher curvature. We will discuss how the skin is attached to the underlying bone and muscle. An intermediate fat layer that lies between the upper dermal skin layers and the musculature can be added to model the physical eects of fat deposits at various areas of the body.

5.4 Chapter synopsis

This chapter details the geometric and physical models for the bones, ligaments and skin. As they have mainly passive physical properties, we can group their model descriptions together in one chapter.

6 System Design This chapter describes the system architecture of the modelling system and its core DANCE (Dynamic Animation and Control Environment) foundation. The main design goals of DANCE are outlined and we describe how each of these are satis ed with the resulting system design. As the design is object-oriented, I will use the UML (Universal Modelling Language) design speci cation for documenting the classes and their relationships. Dierent aspects of the design will be explained.

6.1 Core design

The DANCE core consists of a scripting interface, and knowledge of a set of base primitives which are all descended from a common DanceObject. These base primitives are the actuators, articulated objects (including links and joints), and geometry. There are some secondary primitives such as lights and views which allow lighting and multiple camera views in the scene respectively. 18

6.2 Base primitives

Several of the base primitives are abstract classes that can be subclassed for variety and customization. It is this ability that allows the DANCE architecture to be extremely exibile in building a wide variety of potential applications. We decided to implement several of the objects as dynamically-linked plugins. This allows other developers to independently enhance DANCE without knowledge of the core design. They need only follow the speci ed interface of the class. We will illustrate how the abstract actuator class allows a wide variety of physical phenomena to be modelled. Actuators can be used to model anything that exerts forces on the system. Gravity elds, muscles, interactive springs, ground contact forces can all be implemented as actuators. We will brie y outline how the articulated objects are structured and the geometry instancing mechanism developed to allow shared geometry or display lists for economical memory storage.

6.3 Application to Anatomical Modelling System

We demonstrate the application of the DANCE system in constructing an anatomical modelling system. The particular application will be used as a teaching tool in a biomechanics class at the University of Toronto. Aspects of the user interface regarding the construction and manipulation of the skeletal system will be presented (Figure 8). For example, the direct manipulation interface will allow muscles to be interactively attached to the bone using 3-D painting techniques that take advantage of the hardware depth buer. Physical simulation examples will be exhibited. Examples of the scripting interface and the independent GUI will be shown.

Figure 8: Image of the anatomical modelling system.

6.4 Chapter synopsis

This chapter will cover several software engineering principles and demonstrate that a system design is an essential part of successful integration of all the elements together into a working system. The 19

use of UML was chosen because it is considered an industry standard (one of many!) and allows the design to have greater likelihood of being understood or used by a software engineer.

7 Results This chapter illustrates several dynamic anatomical systems that were created with the anatomical modeller.

7.1 Contracting Soleus Muscle

This example demonstrates that our methodology can be applied to anatomical simulations requiring high accuracy. We show how a soleus can contract and verify that volume is preserved and pennation angles change in agreement with observed data.

7.2 Shoulder reconstruction

We reconstruct a shoulder motion of a human joint. This example will use muscles, bones, and ligaments together in a complex joint assembly.

7.3 Horse motion

We will apply muscles to a horse and attempt to stabilize a stance. According to van den Bogert[16], the structure of a horses leg allows it to be relatively stable without need for external control. We wish to verify this.

7.4 Chapter synopsis

The purpose of this chapter is to present a cross-section of dierent examples that illustrates the unique contributions of the anatomical muscle modeller. The contracting soleus muscle will emphasize the low-level applications of the model for detailed muscle architecture functional studies. The shoulder reconstruction which illustrate how one of the most complex joints in the human body can be handled by our system. The reasoning is that if we can model a complex joint, then simpler joints naturally can be created. The nal example will illustrate that the technique will work with a whole-body animal simulation.

8 Conclusion and Discussion We review the major contributions of this work.  A muscle model allowing accuracy at various levels of detail has been developed. This model is suitable for gross shape modelling to accurate muscle architecture at the internal ber level due to the B-spline solid formulation.  Volume preservation and collision resolution are modelled. This combination of features has not been captured in previous muscle models.  Muscles have a controllable activation and are physically capable of applying forces on adjacent muscles and bones. 20

 A unique system architecture is demonstrated that allows modular extensions of the anatomical system and a work ow that allows a user to construct an anatomical system by adding components. This allows applications such as biomechanical simulation, muscle reconstruction and simple simulated surgery to be performed.

8.1 Future Work

One limitation of the parametric B-spline solid formulation for muscle is the lack of intrinsic support for arbitrary topological shapes. It may be desirable to model branching structures that are commonly found in nature with a single solid. Perhaps a subdivision technique similar to that used in generating Catmull-Clark surfaces[3] can be applied. With the existence of this system, there is an opportunity for many novel simulations for biomechanics, and anatomy. Neurologists and motor control researchers can perform and test theories of neural control of muscle, of which the nature is still widely unknown. Functional studies of muscle force based on detailed ber orientations captured with the B-spline solid model can be conducted. Automatic strategies of muscle parameter estimation, perhaps based on optimization can be explored. Simulated evolution or synthetic construction of active animals is another fascinating area to investigate. Increased accuracy can be added by creating new plugins with the DANCE architecture to eventually create a detailed virtual surgery simulator.

8.2 Current Progress

As of this date, chapters 1,2,3, and 6 can be written up completely. The majority of chapter 4 is complete except for the work on modelling contractile forces with biomechanical force models. The remaining items to be completed are:

 Integrate the force-length-velocity relationship into the Lagrangian equations of motion for

muscle and develop the control interface for muscle. This is the contractile force portion of chapter 4. There is a potential risk that the natural frequencies for the equations of motion of the muscle may be signi cantly higher than those of the skeleton, causing sti equations for numerical integration. The addition of damping properties instrinsic to muscle may help to alleviate this problem if it occurs. Implicit integration techniques applied separately to the contractile force model can combat stiness and allow moderately-sized timesteps for simulation. Another potential diculty is the ability to successfully nd a set of muscle parameters that produce stable simulations. (one month)  Integration of muscle model into DANCE. This was partially completed, but recent changes to the muscle model and DANCE architecture will require some re-integration. (one month) item Construct examples detailed in results section. Once the system has been built, these examples can be directly built from the system. (two weeks).  Develop a skin model. This can range from a simple parameteric surface where selected points are attached to the muscle or a general subdivision surface that can handle arbitrary topology and may have physical properties. This is for chapter 5. There is a chance that the resulting skin model may not be visually satisfactory for the entire body. A possible solution would be to selectively apply skin patches on areas of the body and use specialized geometric models for other regions like the nose and ears. (one month) 21

 Ligament modelling is possible with the existing implementation, but it has not been explicitly

done. I am in the process of nding a suitable physical model which will probably be a relatively simple linear spring or biphasic spring. (two weeks or less)  Write the thesis. If this proposal is approved, I intend to immediate start writing all chapters that can be completed (the four I mentioned above). I plan to write in parallel to the implementation of the remaining tasks. I believe this will help to consolidate my research notes and allow me to analytically review the progress of my work. (two months).

References [1] R. McNeill Alexander. Animal Mechanics. Sidgwick and Jackson, 1968. [2] A. J. van den Bogert, H. C. Schamhardt, and A. Crowe. Simulation of quadrupedal locomotion using a rigid body model. Journal of Biomechanics, 22(1):33{41, 1989. [3] E. Catmull and J. Clark. Recursively generated B-spline surfaces on arbitrary topological meshes. Computer-Aided Design, 10:350{355, September 1978. [4] John E. Chadwick, David R. Haumann, and Richard E. Parent. Layered construction for deformable animated characters. In Jerey Lane, editor, Computer Graphics (SIGGRAPH '89 Proceedings), volume 23, pages 243{252, July 1989. [5] David T. Chen and David Zeltzer. Pump it up: Computer animation of a biomechanically based model of muscle using the nite element method. In Edwin E. Catmull, editor, Computer Graphics (SIGGRAPH '92 Proceedings), volume 26, pages 89{98, July 1992. [6] Julian L. Davis. Introduction to Dynamics of Continuous Media. Macmillan Publishing Company, 1987. [7] Scott L. Delp, J. Peter Loan, Melissa G. Hoy, Felix E. Zajac, Eric L. Topp, and Joseph M. Rosen. An interactive graphics-based model of the lower extremity to study orthopaedic surgical procedures. IEEE Transactions on Biomedical Engineering, 37(8):757{767, 1990. [8] David R. Forsey and Richard H. Bartels. Hierarchical B-spline re nement. In John Dill, editor, Computer Graphics (SIGGRAPH '88 Proceedings), volume 22, pages 205{212, August 1988. [9] Marie-Paule Gascuel. An implicit formulation for precise contact modeling between exible solids. In Computer Graphics (SIGGRAPH '93 Proceedings), volume 27, pages 313{320, 1993. [10] P. Kalra, P. Beylot, P. Gingins, N. Magnenat Thalmann, P. Volino, P. Homeyer, J. Fasel, and F. Terrier. Topological modeling of human anatomy using medical data. In Computer Animation '95, pages 172{180, 1995. [11] Brian Mirtich. Fast and accurate computation of polyhedral mass properties. Journal of Graphics Tools, 1(2):31{50, 1996. [12] M. G. Pandy, F. C. Anderson, and D. G. Hull. A parameter optimization approach for the optimal control of large-scale musculoskeletal systems. Transactions of the ASME, 114:450{ 460, November 1992. 22

[13] Thomas W. Sederberg and Scott R. Parry. Free-form deformation of solid geometric models. In David C. Evans and Russell J. Athay, editors, Computer Graphics (SIGGRAPH '86 Proceedings), volume 20, pages 151{160, August 1986. [14] Jianhua Shen and Daniel Thalmann. Interactive shape design using metaballs and splines. In Implicit Surfaces '95, pages 187{193, 1995. [15] Jos Stam. Exact evaluation of catmull-clark subdivision surfaces at arbitrary parameter values. In Computer Graphics (SIGGRAPH '98 Proceedings), volume 32, pages 395{404, 1998. [16] A. van den Bogert. Computer Simulation of Locomotion in the Horse. ADDIX, 1989. [17] Jane Wilhelms. Animals with anatomy. IEEE Computer Graphics and Applications, 17(3):22{ 30, 1997. [18] A. Witkin, M. Gleicher, and W. Welch. Interactive dynamics. In ACM Computer Graphics (Proc. 1990 Symposium on Interactive 3D Graphics), volume 24, pages 11{21, 1990.

23