Threading neural feedforward into a mechanical spring: How biology

Biol. Cybern. 92, 229–240 (2005) DOI 10.1007/s00422-005-0542-6 © Springer-Verlag 2005

Threading neural feedforward into a mechanical spring: How biology exploits physics in limb control Karl T. Kalveram1 , Thomas Schinauer1 , Steffen Beirle2 , Stefanie Richter3 , Petra Jansen-Osmann1 1

Department of Cybernetical Psychology and Psychobiology, University of Duesseldorf, Universit¨atsstr.1, ¨ 40225 Dusseldorf, Germany Institute of Environmental Physics, Im Neuenheimer Feld 229, 69120 Heidelberg, Germany 3 ¨ Institute of Aerospace Medicine, German Aerospace Centre DLR e.V., Linder Hohe, 51147 Cologne, Germany 2

Received: 3 December 2003 / Accepted : 14 December 2004 / Published online: 14 March 2005

Abstract. A solution is proposed of the hitherto unsolved problem as to how neural feedforward through inverse modelling and negative feedback realised by a mechanical spring can be combined to achieve a highly effective control of limb movement. The revised spring approach that we suggest does not require forward modelling and produces simulated data which are as close as possible to experimental human data. Control models based on peripheral sensing with forward modelling, which are favoured in the current literature, fail to create such data. Our approach suggests that current views on motor control and learning should be revisited.

1 Introduction Recent reviews (Kawato 1999; Sabes 2000) reveal that the currently favoured concept of movement control typically includes negative feedback control based on peripheral sensing, a forward model which bridges the delay in the feedback lines, feedforward control through an inverse model which gets its state input also via peripheral sensing and forward modelling, and neural merging of feedback and feedforward signals. The concept abandons the assumption that the stretch reflex (Sherrington 1906; Marsden et al. 1972; Vallbo 1974; Nichols and Houk 1976) plays an essential role in movement control, but it preserves the fundamental idea of negative feedback control traced back to explicit peripheral measurements and further adds the notion of adaptive inverse control (Widrow and Walach 1996; Neilson et al. 1997). The concept also includes the acquisition and tuning of the inverse model by an algorithm going through the forward model (Wolpert et al. 1995). Even a candidate for the anatomical location of both internal models is currently in discussion: the cerebellum (Wolpert et al. 1998; Spoelstra et al. 2000; Timmann et al. 2000). Thus the concept, as sketched in Fig. 1, suggests an integrative approach to motor control, Correspondence to: K. T. Kalveram (e-mail: [email protected])

expressed in terms of cybernetics, uses predicted measurements based on explicit peripheral sensing, and is widely accepted (Kalveram 1992; Stroeve 1997; Kalveram 1998; Wolpert and Kawato 1998; Bhushan and Shadmehr 1999; Kawato 1999; Desmurget and Grafton 2000; Flanagan et al. 2003), though not completely uncontroversial (Ostry and Feldman 2003). Nevertheless, the peripheral sensing approach outlined above neglects to mention that the mechanical properties of the limb-muscle system itself can also establish a negative feedback control loop which, however, does not demand explicit peripheral measurements (Hill 1938; Asatryan and Feldman 1965; Bizzi et al. 1976; Polit and Bizzi 1979; Sternad 2002; Sainburg et al. 2003). In this ‘equilibrium point hypothesis’, the loop gain is provided by the mechanical joint stiffness, and the setpoint by the joint’s mechanical equilibrium position (this is where all muscularly induced forces acting on the limb add up to zero, and where the limb comes to rest if persistent external forces are absent). Assuming that this suffices for movement control, a goal-directed movement could be performed by simply turning the intended position of the limb into the equilibrium position to be attained next. The task dynamics approach (Saltzman and Kelso 1987) or the spinal force field hypothesis (Mussa-Ivaldi and Bizzi 2000) extends this equilibrium point hypothesis (Feldman 1966) to limbs with several joints and to goals located on planes or even in space. Again, the basic idea is that the nervous system specifies muscular forces which superimpose onto a force field having an equilibrium point at the goal position such that the limb’s position is automatically driven to – or, loosely speaking, attracted to – this location, independent of the starting position. At first glance, neither an inverse nor a forward model seems necessary, so that the equilibrium point hypothesis appears to be the point of departure in modelling motor behaviour (Bizzi et al. 1992). However, if gravitational or other persistent external forces are acting upon the limb, equilibrium position and actual resting position deviate considerably. Assuming for instance an angular setpoint of 90◦ to the direction of gravity, a human forearm (mass = 1 kg placed at

230

Fig. 1. The peripheral sensing approach, expressed as a schematic diagram of the internal models used in controlling a multijoint arm. The desired motion of the arm X* is fed into an inverse model of the arm, which acts as a feedforward controller, producing a command signal Uff . There is also evidence of late influence from a feedback control signal, Ufb . The two commands are combined, through simple addition perhaps, to yield the final control signal U. The feedback pathway is illustrated in grey to reflect the fact that it plays a subordinate role as a result of feedback. (NB: The word ‘subordinate’

indicates that the peripherally sensed signals cannot be directly used for motor control, only via forward modelling). The true state of the arm, X, is estimated with a combination of the visual and proprioceptive feedback. The state estimate, X , serves as input to the inverse model and is also compared to the desired state to yield an estimate of the current motor error (E ). The latter signal is used both in feedback control and to drive adaptation of the inverse model (dashed arrow). (See Sabes (2000, p. 742). Figure 1 modified from Sabes (2000))

a distance of 0.25 m from the elbow, stiffness around the elbow = 10 N/rad as we found it to be the average value) will decline from this setpoint by about 15◦ . This severe inaccuracy could be removed by an inverse model, as in the peripheral sensing approach. However, this causes the problem that the feedforward signal provided by the inverse model is given in neural terms, whereas the feedback signal processed by the spring to establish negative feedback control is given in mechanical terms, that is to say, by Hooke’s law. How these incompatible signals could be merged to attain a uniform control signal remains an open question. The revised spring model, which we offer in Fig. 2, solves this problem. In this model, we complete the equilibrium point hypothesis by an inverse model and a neuromechanical interface capable of merging the feedback and feedforward signals mechanically, yet we leave out a forward model. The single-lined arrows in Fig. 2 represent timedependent continuous variables which describe movement execution, whereas double-lined arrows stand for sampled data being constant during a movement and refer to movement planning. The part called ‘execution’, therefore, corresponds to the processes depicted in Fig. 1. In each case, planning of a reaching movement starts by choosing a desired end position, also known as the movement goal. Next, a trajectory has to be selected which serves as a desired pathway between the start point and the goal. Therefore, in our case it is necessary to have a device which changes the angular distance to be passed over into an appropriate and temporally continuous sequence of desired angular positions ϕd . In technical terms, a device accomplishing such a streaming function is called a paral-

lel-to-serial converter. We ascribe this streaming function to a pattern generator. In biology, central pattern generating is viewed as a basic principle for the organisation of rhythmic behaviour, regardless of whether one considers swimming in fish (Holst 1939), walking in mammals (Forssberg et al. 1980), or mastication of food in the lobster’s gastric mill region (Miller and Selverston 1985). Through efferent pathways from other regions of the brain such a pattern-generating neural system can be switched on and off; in the latter case it may also be after exactly one period. At the behavioural level, patterns emitted by a central pattern generator manifest as ‘automatised’ movements distinguishable in terms of intensity, period length, and/or velocity. Nevertheless, from the perspective of the behaviourally oriented biologist these movements appear as if they were of ‘constant form’ (Eibl-Eibesfeld 1987). Here, we define the pattern generator as a functional unit engaged in planning. It obtains the distance intended to be passed over as an input value and, when triggered, emits the complete desired kinematics (= ϕd and its first and second derivative) leading the limb to the goal (Kalveram 1991). However, we do not insist on a particular anatomical location of that unit. In movement execution, the inverse model then receives the stream of the desired kinematics produced by that pattern generator as input and puts out a stream of torques Qff to be fed forward. The neuromechanical interface in the middle of the execution stage in Fig. 2 is the pivot of our concept. Its task is to thread the neurally coded feedforward torque Qff into the mechanical spring and to make the desired angular position ϕd the reference value of the mechanically realised negative feedback loop. Figure 2 depicts how the neuromechanical interface solves this task: at the neural side, first

231

Fig. 2. Revised spring model. In contrast to the peripheral sensing approach sketched in Fig. 1, here the negative feedback loop is realised through a mechanical spring which operates with vanishing time lag and, of course, without noise in the feedback line. X is replaced by the joint angle ϕ, and U by the joint torque Q, while the feedback controller is functionally given by a scalar gain factor K representing the joint stiffness. The neuromechanical interface suggests how merging of feedforward (Qff ) and feedback (Qfb ) can be realised mechanically such that the desired position ϕd – and not the equilibrium position ϕ0 – becomes the reference (setpoint) for negative feedback control. The grey-coloured right-hand part represents the arm’s spring prop-

erty. K  and ϕ0 are transformed into the mechanical values K and ϕ0 by neuromuscular interfaces which are indicated by the two small grey boxes (for details see (3)). The model is completed with a planning section containing a pattern generator which transforms the difference between the planned angular end position and the actual angular start position into a desired temporal kinematic pattern (= trajectory ϕd and its first and second derivative). These trajectories provide the command input (X∗ in Fig. 1) and state inputs into the inverse arm model. The arrangement shown in Fig. 2 allows one to omit entirely a forward model (see text for more information)

the equilibrium position ϕ0 to be transferred to the arm is determined by ϕ0 = Qff /K  + ϕd , where K  denotes a freely selected value for the arm’s stiffness. K  and ϕ0 are then transformed into the respective mechanical values K and ϕ0 by neuromuscular interfaces, as indicated by the two small grey boxes (the transformations are not explicitly shown in Fig. 2, but see (3)). According to Hooke’s law, the muscularly generated torque then becomes Q = K(ϕ0 − ϕ). Taking ϕ0 for ϕ0 and K for K  yields ϕ0 = Qff /K + ϕd , so Q = K(Qff /K + ϕd − ϕ) = Qff + K(ϕd − ϕ). The term K(ϕd − ϕ) describes the output – here called Qfb – of a negative feedback controller which uses ϕd as the setpoint. Therefore, though ϕ0 remains the equilibrium position of the arm, it is not taken as the reference for negative feedback control as is assumed in the equilibrium point hypothesis. Signal processing by the neuromechanical interface also explains the experimental findings that mechanical stiffness and mechanical angular equilibrium position can vary independently of each other (Latash 1992) and even allows one to consider the stream of equilibrium positions as something like a ‘virtual trajectory’ (Hogan 1985), though this trajectory is not centrally determined as originally proposed by Hogan but computed at the very end of the neural processing. In Fig. 2, negative feedback control operates without a delay in the feedback line. Additionally, the state input of the inverse model is generated by the pattern generator – and not, as suggested in Fig. 1, fed back from the

periphery via a forward model. Both facts make a forward model dispensable. Therefore, our revised spring model conforms neither to the equilibrium point hypothesis and its derivatives nor to the control model based on peripheral sensing with forward modelling. The purpose of this paper is to check which data – those created by the peripheral sensing approach or those by the revised spring model – are closer to experimental human movement data. 2 Methods The rationale of our method is to administer disturbing torque impulses to the forearm of real subjects and to compare the kinematic records with simulated data. For these simulations, first the parameters of each subject’s forearm were estimated and fed into the arm model being controlled according to the control model of interest. Then simulated impulses were applied. The peripheral sensing concept sketched in Fig. 1 is comprised of several different aspects which are covered by three control models called model 1a, model 1b, and model 1c. The revised spring approach of Fig. 2 is addressed by control model 2. This section includes a description of the experiment, the formulas, and procedures used to get the necessary parameters for the simulations, while the next section concerns the evaluation of the four control models 1a, 1b, 1c, and 2 with respect to the human controller.

232

Fig. 3. Experimental setup. Subjects sat in an adjustable chair facing a head-centred concave screen 1.5 m in front of them. Their right forearms were inserted into an orthosis which was attached to a lever fixed on the axis of a torque motor. The axis of the motor was located underneath the elbow joint. The size of the orthosis was adjusted according to each subject’s arm anthropometrics to ensure a secure and tight fit. That allowed flexion-extension movements of the forearm, but only in the horizontal plane. Throughout each block of trials, the motor exerted a constant torque QC (Table 1). The angle of the forearm was measured with reference to the upper arm. The direction straight ahead and perpendicular to the upper arm was defined as zero. The joint angle was then transferred to a head-centred flashlamp which projected the feedback marker on the screen. When viewing the feedback marker, therefore, the visual gaze angle was congruent with the joint angle. During movement the feedback marker was darkened. The target marker could be placed either on the left side (at +0.35 rad = +20◦ ) or the right side (at −0.35 rad = −20◦ ) of the screen. While subjects made a reaching movement towards the target marker, a perturbing torque impulse – short enough to prevent adaptation – could be administered

2.1 The experiment 2.1.1 Experimental setup. Nine right-handed subjects (four female, five male) participated in the study. Ages ranged between 19 years and 50 years (m = 32 years). All participants gave written informed consent prior to taking part in the study. The experimental procedures were approved by the local ethics committee. See Fig. 3 for details of the setup. Control software used to drive the torque motor (Mattke MC27P with amplifier MRL150/40) and the marker lights was based on the MATLAB technical computing language, Simulink, and Real-Time Workshop (all by The Mathworks, Natick, MA, USA). Angular position, velocity and acceleration were measured by a potentiometer, a tachometer and an accelerometer, which were fixed on the motor shaft and/or on the lever. The data were sampled at 500 Hz and digitised with a 12-bit analogueto-digital converter (Meilhaus ME300). Digital data were saved on hard disk after completion of a block. 2.1.2 Experimental procedure. The experiment started with three training blocks meant to familiarise the subjects with the hardware and the procedure, followed by five experimental blocks (Table 1). One block included 42 trials, each of which lasted 3 s. Between the blocks a pause of about 60 s was inserted. The complete experiment took

about 30 min. At the beginning of a trial, the target jumped alternatively either from the left (+0.35 rad = +20◦ ) to the right (−0.35 rad =−20◦ ) or vice versa. The laboratory was darkened during the experiment. Subjects were instructed to perform a goal-directed forearm movement – either by flexion or extension – to the requested target position (Fig. 3). Reacting as quickly as possible was not emphasised, but the participants were asked to move accurately and at a quick pace once they had started moving. They were also requested to stay relaxed and to bring the ongoing movement to an end if they experienced a perturbation. An online operating motion detector signalled movement if the absolute velocity exceeded 2◦ /s. The feedback arrow was darkened 0.1 s after the detection of movement onset and brightened again 0.1 s after detection of a standstill, the related position of which we called the ‘first stop position’. The reason for this blacking out was to prevent the subject from receiving visual feedback during the transport phase of the movement, and thus to prevent early corrective movements. However, after the subject had stopped and visual feedback had reappeared, opportunity to make a corrective movement was given. Because the forearm moved horizontally, gravity had no effect on the movements. In order to mimic gravity, the motor could exert a constant torque QC on the forearm (Table 1). This dislocated the forearm’s angular resting position from the equilibrium position (see (1) and the description of the variables). In each block, a small number of trials (4 in training blocks 2 and 3, 12 in the experimental blocks) were singled out in which the motor generated a disturbing torque impulse with an amplitude of 5 N m and a duration of 50 ms in addition to the constant torque, either in or against the actual movement direction. The torque impulse was triggered when the actual position crossed the ‘10◦ criterion’, in other words deviated 10◦ from the start position. On average, this happened about 250 ms after onset of movement. It should be mentioned that the mean target error of the movements at first stop position was independent from grade and direction of the constant torque as well as from application of a disturbing impulse.

2.1.3 Data processing and normalising. Kinematic data were filtered offline by a fourth-order recursive Butterworth low-pass filter with a cut-off frequency of 10 Hz. Due to considerable variability, all curves were first aligned at movement onset as determined by the motion detector mentioned above and then normalised with respect to time and movement amplitude dA (= position at first stop minus start position). The normalised positional trajectory began at the zero position, crossed the 10◦ criterion point 250 ms after movement onset, and attained +1 or −1 at the time point of first stop, while normalised velocity and acceleration curves had values which produced this positional trajectory precisely. When visual feedback reappeared after the first stop, subjects often performed a corrective movement such that the normalised positional trajectory drew away from +1 or −1. We denoted all normalised kinematics of actual disturbed movements again

233 Table 1. Application of constant torques in different experimental blocks

Training blocks

Experimental blocks

Block number

1

2

3

1

2

3

4

5

Constant torque QC [N m]

0

+0.5

−0.5

+0.75

+1.5

0

−0.75

−1.5

by ϕ and its derivatives and those of undisturbed movements by ϕu and its derivatives. 2.1.4 Determination of spring parameters. For each disturbed movement, the spring parameters (inertia J , coefficient of viscous damping B, and stiffness K; see (1) below) were determined (see (4)–(7) below) using the normalised kinematics of a movement. The mean values attained by the subjects were: K = 5.2 to 12.1 [N m/rad], B = 0.32 to 0.80 [N m s/rad], J = 0.067 to 0.21 [kg m2 ] (sum of arm and lever; lever alone: 0.045 kg m2 ). These values lie within the range reported by others as well (Bennett et al. 1992; Gomi and Kawato 1997). The stretch/squeeze factors f used for normalisation ranged from 0.89 to 1.4. Means and standard deviations of J , B, K, and f per subject are given in Table 2. The measurement procedure of J , B, K is described in detail in the following subsection.

2.2 Formulas and procedures The dynamics of an arm with one degree of freedom are governed by the differential equation J · ϕ¨ = K · (ϕ0 − ϕ) − B · ϕ˙ + G · sin (ϕ − ϕg ) −QX ,

 

  QJ

Q

QB

(1)

QG

where ϕ0 , ϕ, ˙ and ϕ¨ denote respectively time-dependent actual angular position, velocity, and acceleration; J is the moment of inertia related to the centre of rotation; K is the coefficient of joint stiffness; B is the coefficient of viscous damping; G is defined as mag, where m denotes the mass, a the distance between the centres of mass and rotation, and g the gravitational constant; QJ , Q, QB , QG , and QX are torques generated respectively by inertial, elastic, viscous, and external forces which include gravitational (QG ) and disturbing (QX ) forces; ϕg is the angle between the gravitational field and the body reference of the limb (here, ϕg = 0 is assumed); ϕ0 is the angular ‘equilibrium position’ the joint angle ϕ will attain after a while if the external torques QG and QX vanish. In contrast, the joint angle finally attained with non-vanishing external torques is called the angular ‘resting position’. Note: In the experiment the forearm moved horizontally, so gravity had no effect on the movements. In order to mimic gravity, the motor exerted a constant torque called QC on the forearm (Table 1), which persistently dislocated the forearm’s angular resting position from the equilibrium position. This feature can simply be taken into account by replacing in (1) the whole term QG with QC .

B and K are allowed to vary with time and/or angular position. Active movements can be executed by shifting the equilibrium position ϕ0 away from its momentary value, which generates a torque Q driving the arm towards the (new) equilibrium position. Equation (1) is arranged to represent the arm’s forward dynamics after division by J . In simulations, (1) can be solved online given that the parameters J , K, B, G and the variables ϕ0 , QX are known. The kinematic state is then yielded by integrating the angular acceleration twice in a manner similar to that shown in Fig. 5 by an analogue representation. A forward model of the arm is obtained likewise if the physical parameters and variables are replaced by their neural representations. An inverse arm model (see (2)) can be deduced from (1) if the actual kinematics are replaced by the desired kinematics (denoted by the subscript ‘d’). The desired command variable (here acceleration) is supplied by the pattern generator. The desired kinematic state variables can be taken either from the pattern generator or from peripheral sensing via the forward model. The neural counterparts of the physical parameters J , B, and G are marked by superscripts ‘  ’. They must be acquired by learning, but this is not an issue of the present paper. Rearrangement of terms then yields the inverse model:   Qff = K  · ϕ0 − ϕd =J  · ϕ¨d + B  · ϕ˙d −G · sin(ϕd − ϕg ) .(2)
  QG

To take account of the present experiment, again the whole term QG must be replaced with QC (see (1) and the note below the description of the variables). A simple neuromuscular interface similar to the alpha model (Bizzi et al. 1992) is given by joint stiffness : K = k(n1 + n2 ) , angular equilibrium position : ϕ0 = r(n1 − n2 )/K , coefficient of viscous damping (proposed) : B = bK ,(3) where n1 , n2 reflect the numbers of motor units recruited in the respective muscles of the antagonistic pair and k, r, and b denote appropriate positive constants. Viscous damping was originally not included in the alpha model but could be concluded from Hill’s model as being roughly proportional to K. More detailed muscle models which also take non-linearities and differences in the muscular tissues into account do exist (Winters 1995; Seyfarth et al. 2000), but this simple model suffices in showing the basic idea. The arm’s spring parameters valid in a single disturbed movement were obtained using (1) as follows: J · ϕ¨ + B · ϕ˙ + K · (ϕ − ϕ0 ) = QC + QX , J · ϕ¨u + B · ϕ˙u + K · (ϕu − ϕ0 ) = QC ,

(4) (5)

234 Table 2. Intra-individual means and standard deviations of stiffness K, viscosity B, inertia J , and stretch/squeeze factor f . Dimensions of K, B and J are N m/rad, N m s/rad and N m s2 /rad

Subject K B J f

Mean STD Mean STD Mean STD Mean STD

1

2

3

4

5

6

7

8

9

Total

12.0291 1.7714 0.4834 0.0468 0.1225 0.0040 1.0837 0.0468

6.1858 0.6056 0.3203 0.0464 0.0717 0.0047 0.9147 0.0220

6.9352 0.2680 0.5080 0.0181 0.0775 0.0123 0.9703 0.0554

8.7832 0.8570 0.4353 0.0320 0.0925 0.0176 1.0278 0.0698

9.9779 0.5972 0.4523 0.0583 0.1078 0.0050 0.9693 0.0168

5.1901 0.8644 0.4111 0.0499 0.0670 0.0065 0.8936 0.0393

5.1568 0.3541 0.3828 0.0497 0.0974 0.0087 1.0770 0.0272

5.4440 0.4428 0.3392 0.0338 0.0864 0.0060 0.9875 0.0321

8.2237 0.5108 0.8018 0.0842 0.2115 0.0190 1.4090 0.0361

7.5473 0.6968 0.4594 0.0466 0.1027 0.0093 1.0370 0.0384

where (4) holds for a perturbed movement and (5) holds for the course which the arm would have taken if it had been controlled identically, but without experiencing the perturbation. ϕ, ϕ, ˙ and ϕ¨ denote the actually recorded kinematics of a movement (see experimental setup). QC denotes the constant torque applied in the experiment, which mimicked the gravitationally induced torque QG introduced in (1). ϕu and its derivatives describe the fictive angular kinematics of the arm if it were unperturbed. We estimated ϕu and its derivatives by averaging, separately with respect to extension and flexion, the normalised unperturbed movements in a block. Presuming that the perturbation leaves the spring parameters unchanged, (5) can be subtracted from (4): J · (ϕ¨ − ϕ¨u ) + B · (ϕ˙ − ϕ˙u ) + K · (ϕ − ϕu ) = QX .

(6)

When turning to the normalised trajectories and applying matrix notation, (6) can then be written as 

{ϕ¨ − ϕ¨u

ϕ˙ − ϕ˙u

 J∗

ϕ − ϕu } • B∗ = Q∗X . K∗

(7)

The values between the left braces in (7) have to be interpreted as an m × 3 matrix, whereby each of the m row vectors represents one sample of differences between the normalised measurements of a disturbed movement and the related assessed undisturbed movement. The lefthand three-element column vector contains the unknowns which are the values for the coefficients of inertia, damping, and stiffness. The m-element column vector on the right-hand side of (7) represents the normalised torque impulse. Due to normalisation (f is the temporal stretch/squeeze factor and dA the movement amplitude; see data processing and normalising), these variables transform to J ∗ =J · f 2 ,

B ∗ =B · f,

K ∗ =K,

Q∗X = QX /(f · dA) . (8)

The sample matrix selected in this investigation contained 300 samples and spanned the normalised interval from t = 200 ms (movement start) to t = 800 ms (whereabouts the real undisturbed prototype movement would have come to an end). Equation (7) is an overdetermined system of linear equations solvable for J ∗ , B ∗ , and K ∗ by the least mean squares (LMS) method if the rank of the sample matrix is three. This was fulfilled in all cases. The procedure

should allow the assessment of the arm’s spring parameters to be valid in a single disturbed movement if they are approximately constant during the time of measurement. If they are not constant, the procedure will acquire temporal means of the spring parameters. We took the LMS method from MATLAB. Whether all these considerations indeed prove to be true will be evident if the computed three parameters suffice to reproduce the related actual movement in the simulation.

2.3 Preparing the control models for numerical simulation For the purpose of numerical simulation, the peripheral sensing approach must be formulated more precisely than in Fig. 1 and must also be tailored to the dynamics of a one-jointed arm. To make the approach comparable to model 2, it is also necessary to add a pattern generator and to take over the denotations of model 2. All this is done in Fig. 4. As mentioned above, the peripheral sensing concept given in Fig. 1 (resp. Fig. 4) can be simulated in different ways, depending on, for instance, how the forward model is realised or where the kinematic state input of the inverse model is coming from. Therefore, to highlight the effects of those different features, the concept shall be covered by three models denoted by 1a, 1b, and 1c. Model 2 then reflects the revised spring model outlined in Fig. 2. Specifically: Model 1a interprets the peripheral measurement schema depicted in Fig. 1 (resp. Fig. 4) to the largest possible extent. The added pattern generator produces the desired angular kinematics, the desired acceleration of which is used as the command input into the inverse model and the desired position as the reference for negative feedback control. The time lag caused by delayed proprioceptive sensing is set to 50 ms (25 ms for the afferent and efferent neural running times, plus at least 25 ms for the neuromuscular-mechanical transfer). A Smith predictor (Smith 1957; Miall et al. 1993) as outlined in Fig. 5 serves to sustain the forward model, an arrangement which is supposed to create a nearly perfect prediction of the arm’s output if the movement is undisturbed. Predicted angular position is then used for negative feedback control, while predicted angular position and velocity supply the inverse model’s state input.

235

Fig. 4. The peripheral sensing approach as described in Fig. 1, but completed by a pattern generator and tailored to the dynamics of a one-jointed arm. The feedback controller is given by the gain K of the negative feedback loop. Learning circuitry has been omitted because learning is not an issue addressed by the paper. To make the peripheral sensing approach (which comprises models 1a, 1b, and 1c) compa-

rable to model 2 with respect to the denotation, the command signal X∗ in Fig. 1 has been replaced with ϕ¨d , the actual arm output X with ϕ, ϕ, ˙ ϕ, ¨ the predicted (estimated) arm output X with ϕp , ϕ˙p , ϕ¨p , and the command signals Uff , Ufb , U with Qff , Qfb , Q. The state feedback is given by ϕp , ϕ˙p . See text for additional information

Fig. 5. Forward model of arm dynamics. The figure shows in analogue representation that the forward model operates through online solving of (1) by integration. In contrast to pure forward dynamics, two loops are added which include the temporal delay units dt. This enables the output of the integrators to readapt to the actual kinematic state of the arm in case of disturbance. For that, the loop delays must be equal to the delays of the state measurements (Kalveram 1998). In the presented simulations, the loop gain g is

limited to 5. This value achieves optimal accuracy without destabilising control. The forward model is embedded into negative feedback control, which is additionally sustained by a Smith prediction loop (Miall et al. 1993). However, the latter can also be omitted at the expense of accuracy. Notice that variables, parameters, and computational units used in the model are neural representations of the variables, parameters, and arithmetic functions physically defined in (1) and must be acquired by learning

236

Model 1b is quite similar to model 1a, except for two features: (1) The supply of the inverse model’s kinematic state input is taken, not from the periphery, but from the pattern generator. Thus, state feedback is replaced by state feedforward. But negative feedback is not altered and continues to be based on predicted angular position. (2) In the forward model the Smith predictor (Fig. 5) is disconnected. Model 1c is identical to model 1b in all features except the forward model: the Smith predictor is reconnected. Model 2 refers to the revised spring concept of Fig. 2: negative feedback control is based on the mechanical spring property of the muscle-limb system, feedback and feedforward torques are merged mechanically using the neuromechanical interface, and the state input of the inverse model is provided by the pattern generator. A forward model of the arm dynamics is not necessary.

2.4 Movement simulation Prior to simulation, the parameters of the arm (J , B, and K) were measured by the algorithm described in (4)–(8) and then inserted into the arm dynamics (1) and its inverse model (2). To create simulated disturbed movements, the arm dynamics were then computed according to (1) and the inverse model according to (2). The averaged undisturbed kinematics (ϕu ,ϕ˙u , ϕ¨u ) per subject, block, and movement direction (see (5)) were taken as the respective desired kinematics thought to be emitted by the pattern generator according to Fig. 2 (resp. Fig. 4). This is justified because the negative feedback signal Qfb vanishes if the inverse model is correct, so that desired and actual positions should coincide under undisturbed conditions. The constant torque mimicking gravity and the impulse torque were applied to the simulated arm dynamics as in the experiment. In simulated movements, final braking was triggered as follows according to a prior finding (Konczak et al. 1999): stiffness K and the coefficient of viscous damping B applied in the arm model underwent a ramp increase between 0.8 s and 0.9 s of movement time with rising rates of 25 K/s (resp. 50 B/s) such that they reached the levels 3.5 K and 6 B at 0.9 s. The neurally represented coefficient of viscous damping in the inverse model (2), however, retained the originally inserted value B  .

3 Results Figure 6 demonstrates the effects of the disturbing torque impulses on the positional trajectories of some ongoing real and simulated movements. The trajectories generated by control model 2, which reflects the spring approach, obviously track the real data the closest, followed by control model 1c, which is based on a forward model sustained by the Smith prediction. Omitting the Smith prediction in control model 1b considerably increases the deviations.

Model 1a, though mirroring the ‘state of the art’ in the literature, performs the worst. In order to achieve a quantitative analysis, the normalised movement time was partitioned into five segments, namely 200–450 ms, 450–800 ms, 800–1000 ms, 1000–1200 ms, and 1200–1400 ms. The first segment covered the undisturbed transport part of movement, the second the disturbed part. The last three segments enclosed the ‘tail’ of the movement in which the limb came to rest. Not included in the analysis was the span between 0 and the beginning of movement at 200 ms. To get quantitative assessments regarding the goodness of fit of the four control models under concern, we defined – with respect to a temporal segment – an ‘index of model fit’ (IMF) for each control model. An IMF value was computed as follows: first, the mean absolute difference between each real and the corresponding simulated trajectory was determined. This yielded 60 values per subject (5 blocks times 12 disturbed movements per block). The average of these values was then taken as a subject’s IMF and the average of all subjects as the (overall) IMF – in the respective temporal segment. The lower such an IMF value is, the closer the fit. But an IMF value also includes the random variation which occurs even if movements are executed under the same conditions. We quantified this random variation in a manner similar to the IMF outlined above and named it the ‘index of movement variability’ (IMV): first, the mean absolute difference between each real disturbed trajectory and the averaged real disturbed trajectories accomplished under the same conditions (same movement direction, torque pulse direction, block) was computed. The average of these values over all conditions was then taken as a subject’s IMV value and the average among the subjects as the (overall) IMV – referring to the considered temporal segment. We regarded such an index of movement variability as the baseline value, below which an index of model fit can hardly fall, even if the data produced by the model approximate the human data as closely as possible. Means and standard deviations of the segmental indices of model fit and the corresponding indices of movement variability are shown in Fig. 7. Here, the main features already visible in Fig. 6 appear even more strikingly: in segment 2 (called pulse), the most critical temporal segment in which the torque impulse is applied, as well as in segment 3 (called postpulse), the model 2 fit is the closest and is barely distinguishable from the movement variability which we consider as the baseline. Notice that in the prepulse segment all models exhibit a very good fit and also perform equally well. To statistically underpin these features, we decided to compute five one-way analyses of variance (ANOVA), one for each temporal segment. In each ANOVA, the independent variable was formally composed of five levels, and the related enlisted values of the dependent variable included the indices of the fit of models 1a, 1b, 1c, and 2 and the index of movement variability (see definitions above). For all five temporal segments (numbered here from 1 to 5), the respective ANOVAs (SPSS) achieved significant effects (F -values: F1 = 28.8, F2 = 155.2, F3 = 129.4, F4 = 244.2,

237

Fig. 6. Normalised kinematics of four selected experimental movements (thick grey curves) and the related simulated trajectories referring to control models 1a, 1b, 1c, and 2 (thin black curves). The tics on the x-axes indicate the points of time used to define the segments for the computation of the segmental model errors. The circle at 0.2 s marks the movement onset, the cross at 0.45 s the impulse onset. Tics and labels of the y-axes refer to normalised angular position. The

thin black rectangle marks the normalised disturbing torque impulse, and the thin grey curve the ‘desired’ positional trajectory which the actual movement would have tracked if it had been left undisturbed. The instances are taken from disturbed trials 4, 5, 6, 7 of experimental block 2 of subject 6; however, the other subjects produced quite similar trajectories

F5 = 294.7, with dfb =5−1=4 and dfw = 45 − 5 = 40 for each analysis, and p1 to p5