Bayesian Plan Recognition and Shared Control ... - Semantic Scholar

pre-print: IEEE/RSJ International Conference on Intelligent RObots and Systems (IROS2007), San Diego, California. pp. 3360-3366

Bayesian Plan Recognition and Shared Control under Uncertainty: Assisting Wheelchair Drivers by Tracking Fine Motion Paths Alexander H¨untemann, Eric Demeester, Gerolf Vanacker, Dirk Vanhooydonck, Johan Philips, Hendrik Van Brussel and Marnix Nuttin Department of Mechanical Engineering, Katholieke Universiteit Leuven Celestijnenlaan 300B, B-3001 Leuven (Belgium) [email protected]

Abstract— The last years have witnessed a significant increase in the percentage of old and disabled people. Members of this population group very often require extensive help for performing daily tasks like moving around or grasping objects. Unfortunately, assistive technology is not always available to people needing it. For instance, steering a wheelchair can represent an extremely fatiguing or simply impossible task to many elderly or disabled users. Most of the existing assistance platforms try to help users without considering their specific needs. However, driving performance may vary considerably across users due to different pathologies or just due to temporary effects like fatigue. Therefore, we propose in this paper a user adapted shared control approach aimed at helping users in driving a power wheelchair. Adaption to the user is achieved by estimating the user’s true intent out of potentially noisy steering signals before assisting him/her. The user’s driving performance is explicitly modeled in order to recognize the user’s intention or plan together with the uncertainty on it. Safe navigation is achieved by merging the potentially noisy input of the user with fine motion trajectories computed online by a 3D planner. Encouraging results on assisting a user who cannot steer to the left are reported on K.U.Leuven’s intelligent wheelchair Sharioto.

I. INTRODUCTION Recent trends in society show that the number of elderly and people with disabilities is increasing at a steady rate. Limited mobility and consequentially high dependence on others potentially affects people’s self-esteem and selfconfidence negatively. Therefore, in the last years, an increasing number of assistive devices have been created to help elderly and disabled people. Predominately among them are electric wheelchairs aimed at enhancing disabled people’s mobility. However, not everybody needing a power wheelchair is able to control it properly. A recent report by Fehr et al. [1] suggests that, in some cases, electric wheelchairs are not prescribed due to the patient’s lack in motor and/or cognitive skills. Even people normally capable of steering a power wheelchair perceive maneuvers such as driving through a door, avoiding obstacles or following a corridor as extremely difficult, time consuming and, therefore, very tiring. The relatively big size of the wheelchair compared to the manoeuvring space in homes and hospitals makes navigation difficult and potentially dangerous. Inaccurate user signals can lead to collisions, block the wheelchair in narrow corridors or make the wheelchair

fall down stairs and ramps. As a consequence of previous findings, Fehr’s survey indicates the existence of a market for wheelchairs equipped with an intelligent controller helping in difficult maneuvers. The purpose of such a device is to combine strengths of both human and robot in a process called shared control. Humans excel in global planning and coarse control, whereas robots are more precise in fine motion control. In accordance to current society trends and the claim for intelligent assistive devices, many smart wheelchairs have been developed in the last decade, for instance: OMNI [2], SmartChair [3], Wheelesley [4], Senario [5] and Navchair [6], just to name a few. An important observation concerning intelligence assistance is that the user’s degree of autonomy may vary in the course of a day. Sometimes, users get tired and require more help. In other occasions, users might desire to take over completely. As a consequence, this requires the assistance mode to be changed, either manually or automatically. Manually changing the degree of assistance continuously might impose a heavy burden on the user. Therefore, different automatic mode selection methods have been developed in the past. Intelligent assistance in most devices consists of predefined assistance behaviors, like avoid-collision, traverse-door or follow-corridor, which get triggered depending on the user input and the environmental state. If the change of assistance mode is hard-coded and independent of the user, the user might not get help appropriately. A user independent approach presumes that the user acts and reacts according to a general pattern. However, especially in the case of elderly and disabled people, the individual driving behavior may vary considerably. Not only do these people have different disorders, their driving performance also depends on the type of wheelchair and user interface they use, and on the experience they have with driving wheelchairs or cars. Furthermore, the user’s driving behavior evolves in time. For example, users may learn to drive better. A user-tailored assistance should estimate the intention of the user prior to helping during driving. Otherwise, the user might get frustrated if the intelligent wheelchair does not behave as expected. It is not straightforward to deduce the correct user intent merely out of the provided user input and environmental information measured with a conventional mobile robotic sensor. The problem is how to relate the present user

pre-print: IEEE/RSJ International Conference on Intelligent RObots and Systems (IROS2007), San Diego, California. pp. 3360-3366

input to his/her true intent if the user’s driving pattern is not considered. Hard-coded decision rules to activate assistance behaviors are very likely to fail, given that different users with distinct disabilities might produce a wide range of inputs for the same intention. The goal of this paper is therefore to present a Bayesian framework for sharing the control between a human and K.U.Leuven’s intelligent wheelchair Sharioto, depicted in Fig. 1. In our Bayesian formulation of shared control, the user’s driving pattern is taken into account explicitly in order not to frustrate the user when trying to help him/her. We divide the process of sharing control with the user in two stages: firstly, the user’s plan is estimated together with the uncertainty on it. Secondly, the intelligent controller decides to which extent to help the user according to an estimated degree of control the user might have for a given situation. Safe navigation is achieved by merging and correcting the input of the user with the velocities needed to track fine motion trajectories computed online by a 3D planner. The remainder of the paper is organized as follows: in section II a general framework for plan recognition is outlined. Section III explains how models of human driving performance can be incorporated in the previously presented plan recognition framework. Section IV details how the plan recognition and user modeling results are employed for taking adaptive decisions under uncertainty. In section V experimental results are reported, which validate the proposed framework in a home-like environment.

the plan recognition framework is generalized to users with arbitrary disabilities. A. Representation of user plans A user plan i can be generically described as a certain goal twist tgoal (vgoal , ωgoal ) that the user wishes to achieve at a certain goal pose pgoal (xgoal , ygoal , θgoal ), where vgoal denotes the desired linear velocity and ωgoal the desired rotational velocity at pgoal , and where θgoal denotes the robot’s orientation at the goal position [xgoal ygoal ]T . The twist t and pose p are represented together as the robot state x. We assume that the user has a trajectory {xstart , . . . , xgoal } in mind to achieve this goal pose and goal twist from his or her current pose xstart . Any user plan can be modeled with this representation in a precise way. The chosen user plan representation provides more flexibility in generating assistance maneuvers than is currently possible with assistance modes that are based on natural language descriptions of tasks. Indeed, it may be tedious to verbally specify complex maneuvers such as “dock under 45◦ to wall F at the position of light switch C” in natural language and to subsequently devise a separate mode for this. B. Bayesian framework for plan recognition In order to capture the user’s plan, a probability function is maintained over the set of possible user plans i. This probability function is updated every time new user signals u are obtained according to Bayes’ rule: pk (ik |uk , H0:k ) = puser (uk |ik , H0:k ) · pk−1 (ik |H0:k ) · η (1)

Fig. 1. K.U.Leuven’s robotic wheelchair Sharioto can be equipped with a hand joystick or a chin joystick. The wheelchair contains 9 infrared sensors, 16 ultrasound sensors with dead zone, 4 ultrasound sensors without dead zone, 2 potentiometers for measuring the orientation of the front castor wheels, 1 lidar (light detection and ranging sensor), and 1 gyroscope for rotational velocity estimation.

II. PLAN RECOGNITION The purpose of our Bayesian plan recognition framework is to recognize all possible user plans without limitation, to estimate the uncertainty upon these plans, and to make the estimation adaptive to the specific user that is interacting with the robot. This framework was already introduced in [7] for able-bodied users along with experimental results. Given the importance of plan recognition for our shared controller, the most important results are summarized. Additionally,

where: 1) H0:k = {u1:k−1 , y 0:k , h} is the history of all user signals u1:k−1 up to time k − 1 (u1:k−1 = {u1 , . . . , uk−1 }), all robot actions up to time k (y 0:k = {y 0 , . . . , y k }) and any form of prior information h (such as maps or model parameters). 2) pk−1 is the a priori distribution over user plans, given the history H0:k . It reflects the belief of the controller in the different possible user plans. 3) puser is the user model: it expresses the likelihood of the current user interface signals uk , given that the user has plan ik , and given the history H0:k . 4) pk is the a posteriori distribution over the user plans, i.e. the probability of the different user plans after user signals uk have been taken into account. 5) η is a scale factor necessary to normalize the probability distribution. The power of Bayes’ rule to estimate stochastic variables stems from the fact that it tackles the estimation problem ‘the other way around’. For example, it may be hard to directly estimate how probable it is that a user wants to dock at a table (i.e. to directly estimate pk ), but estimating which signals a user will give (i.e. puser ) assuming that he/she wants to follow path i to the table seems much easier. In literature, points of comparison to our probabilistic framework can be found in the analysis of motion behaviors.

pre-print: IEEE/RSJ International Conference on Intelligent RObots and Systems (IROS2007), San Diego, California. pp. 3360-3366

Bennewitz et al. [8] propose a probabilistic framework to cluster and predict human motion based on past motion. Walking trajectories are classified into motion behaviors by employing Expectation Maximization (EM). During operation, the system finds the Maximum a Posteriori (MAP) motion pattern given a trajectory in order to predict the motion of people. Glover et al. [9] estimate the activity a person is engaged in when using a walking aid with a hidden Markov model (HMM). Their model integrates metric, topological and temporal information into a probabilistic estimate over activities. In contrast to the framework proposed in this work, neither of these approaches directly includes user signals into predictions of motion behavior or motion plans, and they implicitly assume that users can go everywhere autonomously. The following section discusses the user model puser , which is the cornerstone of this plan recognition framework as it allows to make plan recognition adaptive to the user. The accuracy and usefulness of the framework depends to a large degree on the predictive power of this user model. III. MODELS OF HUMAN DRIVING In order to update the probability distribution using Bayes’ rule, it should be determined how likely the input of the user is, given that he/she has a certain trajectory in mind. In this paper, the wheelchair driver is modeled as a path tracker. The likelihood of the interface signals of the user is then directly related to how well those signals would track possible user trajectories. In the following section the user model first introduced in [7] for able-bodied users is generalized to users with any type of steering disability. A. Generalized user model The presence of steering disabilities introduces additional complexity for estimating the likelihood of the user’s input signals, since not all trajectories can be executed by the user directly. Therefore, we propose to split the user model puser in two components: the reference user model and a performance model relating the steering disability to the reference user model. Mathematically, splitting the user model can be justified as the operation of marginalizing out the joint probability distribution of the ideal reference signals, uref k , and the signals of a disabled user, udis : k Z  
ref dis puser udis p uref k |ik , H0:k = k , uk |ik , H0:k duk Z     ref = pperf udis · puser uref |i , H duref (2) k 0:k k |uk k k where:   ref 1) pperf udis is the performance model relating k |uk the distorted user input in presence of steering disabilities to the reference user model. It is assumed to be independent of the previous history H0:k and only dependent on the current   reference signal. 2) puser uref |i , H is the reference model of an k 0:k k able-bodied user.

B. Reference user model The reference user model is only outlined here. More information and experimental results can be found in [7]. In the ideal case, where any plan can be executed by the user, we hypothesize that perceptual cues lying on a mental path the user is tracking are employed to steer the wheelchair. Path tracking errors are specified in terms of differences between the current robot pose and these subgoals. The parameters of the tracking controller are then estimated through least-squares regression. The regressors ∆θ and ∆lcor are used to predict the linear and rotational velocities the user would produce at his/her interface, upk (vup , ωup ), using the following regression model: ωup = a1 · ∆θ + εω ; vup = b1 · ∆lcor + εv

(3a)

εω ∼ N (0; σω ) ; εv ∼ N (0; σv )

(3b)

uref k

The likelihood of a given user signal = (vu , ωu ) is then calculated from the predicted user signal upk (vup , ωup ) as follows:   puser uref |i , H k 0:k ∼ k ! ! 2 2 (ωu − ωup ) (vu − vup ) · exp − (4) exp − 2σv2 2σω2 C. Performance model for driving disabilities Any form of driving disability can be modeled in relation to a reference user. Such relationship does not have to be stationary nor constant in time allowing to account for varying driving performance. In Fig. 2 one such functional relation between the reference user signal and a user signal with driving disabilities is displayed for a user with a weak left signal. In the wide sense, all deviations from the straight line with unit slope in the performance model can be classified as driving disabilities. Such broad definition allows to model effects like fatigue for able-bodied users as well as any kind of driving behavior of disabled users. Therefore, the performance model can be regarded as the adaptive component in the Bayesian framework for plan recognition and shared control. It has to be adapted to each particular user and his/her steering performance. For the disability depicted in Fig. 2 it is possible to computethe stochastic  component in the performance model, ref i.e. pperf udis |u , analytically. k k Let the steering disability of the user, who can drive to the right but cannot give a strong left signal be: if ω ref > ωth then ω dis = ωth ; ∀v ref : v ref = v dis (5) where ωth ≥ 0 rad/s is a threshold value corresponding to the inflection in the performance model depicted in Fig. 2. It is fairly easy to demonstrate the benefit of our framework for plan recognition and shared control for this driving disability, if the user is able to reach poses in the environment (read intentions) lying at his/her left when he/she intents on reaching them. In addition, this handicap is relatively hard to overcome since it introduces a significant distortion in the user signal when the user wants to go to the left.

pre-print: IEEE/RSJ International Conference on Intelligent RObots and Systems (IROS2007), San Diego, California. pp. 3360-3366

expression of the user model in the angular velocity for a user with a weak left signal (see also Fig. 3):
puser ωkdis |ik , H0:k =      ωth − ωup ωmax − ωup dis Kω · δ(ωk − ωth ) · G −G σω σω
2 ! dis ωk − ωup Kω dis · exp − +Π(ωk [ωmin , ωth ]) · √ 2σω2 2πσω (8) Fig. 2. Example of a personalized performance model for a user with a weak left signal. The angular speed is in hardware specific units (hardware units N V : In SI units turning left implies ω(SI) > 0 rad/s and in N V ’s ω(N V ) < 0 N V ). The performance model relates signals of users with driving disabilities to a reference user signal. Any deviation from the dashed line can be classified as driving disabilities no matter if it arises from actual disabilities or from, for example, fatigue of the user.

In order to calculate the performance model, the linear and angular velocities are assumed conditionally independent:  

ref pperf udis = pperf ω dis |ω ref · pperf v dis |v ref (6) k |uk Determining the performance model for the linear velocity is straightforward, since the linear velocity is
not affected by the driving disability. Thus, pperf v dis |v ref is a Dirac delta function centered on the reference linear velocity. Introducing this performance model in eq. 2 yields a change in variables after marginalizing out over the reference linear velocity by the properties of Dirac’s delta function. The performance model for the angular velocity can be determined with Bayes’ rule as explained in the following equation:

pperf ω dis |ω ref = p ω ref |ω dis · p(ω dis ) · η2 (7) If ω dis = ωth , any reference angular velocity ω ref greater or equal
than ωth is feasible and therefore p ω ref |ω dis = ωth is distributed uniformly in the interval [ωth , ωmax ]. For ω dis < ωth we have the same case as for the linear velocity that is not affected by the driving
disability. Hence p ω ref |ω dis < ωth ∼ δ(ω ref − ω dis ), where δ(·) is Dirac’s delta function. Taking these relations into account and normalizing the distribution leads to the performance model summarized in table I. Please note that TABLE I A NGULAR VELOCITY COMPONENT OF THE PERFORMANCE MODEL FOR A USER WITH A WEAK LEFT STEERING SIGNAL .

` ´ pperf ω dis |ω ref dis ω = ωth ω dis < ωth

ω ref [ωth , ωmax ] δ(ω dis − ωth ) 0

ω ref [ωmin , ωth ] 0 δ(ω dis − ω ref )

for ω ref [ωth , ωmax ] all the probability weight is condensed in one single value, namely ωth , as can be seen when introducing the performance model in eq. 2. Applying the obtained performance model to eq. 2 leads to the following

where: 1) ωkdis is the current noisy input of the user including the steering disability. 2) Kω is h a normalizing constant:   i p p ωmin −ωu ωmax −ωu p − G . Kω (ωu ) = 1/ G σω σω 3) G (·) is the cumulative distribution function of standard normal random variable N (0, 1). 4) Π(ωkdis [ωmin , ωth ]) is the heaviside step function in the interval ωkdis [ωmin , ωth ].

(a) Steering forward or slightly to the left. Fig. 3.

(b) Steering to the right.

User model for a user with a weak left steering signal.

For the linear velocity, the user model is:
puser vkdis |ik , H0:k = v dis − vup K √ v · exp − u 2σv2 2πσv


2 ! (9)

where Kv is again a normalizing constant and vup the linear user velocity predicted by eq. 3 for an intention. D. Fine Motion Planner The trajectory {xstart , . . . , xgoal } of a certain plan i is calculated with a geometric fine motion planner. The planner is more extensively described in [10]. In order to be able to recognize also complex plans such as docking and parking behavior, the fine motion planner should take the robot’s kinematics, its orientation and its geometry explicitly into account. It is imperative to recognize these plans, as elderly and disabled tend to have more difficulties with these maneuvers. The fine motion planner first constructs the free configuration space, which consists T of all discretized 3D robot poses [x, y, θ] that do not collide with an object. Then, a search algorithm is adopted that searches for an optimal path from the current robot pose to each plan’s goal pose pgoal , respecting the robot’s kinematic constraints. The search algorithm constructs a cost function

pre-print: IEEE/RSJ International Conference on Intelligent RObots and Systems (IROS2007), San Diego, California. pp. 3360-3366

over collision-free cells by assigning to all reachable free cells a value that indicates the cost to go to the cell from the current pose. The algorithm finishes when all reachable free cells are visited and have an optimal value. Finding the optimal path from a cell to the current robot pose then simply boils down to following the gradient of the cost function. Since the search algorithm starts from the current robot pose and finishes when all reachable free cells have been visited, the generation of paths to each of the local goal poses is almost instantaneous. IV. SHARED CONTROL The main objective of our shared control algorithm is to assist the user only in situations where assistance is really required. The user should always feel in control of the wheelchair, which in turn should behave as the user expects. Another necessary requirement, related to the fact that a person is transported by the wheelchair, is that the navigation assistance should be safe and comfortable. These user requirements have been translated into a two step process for intelligent assistance: firstly, the intention or plan of the user is estimated online. Secondly, based on the inherently uncertain plan estimate, a decision is made about the extent to which the user should be assisted, i.e. the degree of control granted to the user for a given situation. Based on the estimated degree of control assigned to the user, the user’s input is merged with the output of the shared controller. In order to assist and not to frustrate the user, the decision making process, which incorporates the calculated degree of control, should be made adaptive to each particular user. On the other hand, in order for the user to feel always in control, he/she should be able to override the intelligence assistance, if he/she disagrees with the behavior of the robotic wheelchair. A. Path tracking respecting kynodynamic constraints The shared controller has to generate safe trajectories for the user corresponding to the maximum a posteriori probability (MAP) user plan in eq. 1. This is achieved by tracking the MAP fine motion paths generated by the planner employed for plan recognition (cf. section III-D). The controller converts geometric paths into trajectories, respecting the kinodynamic constraints of the robotic wheelchair using a global dynamic window approach. The implementation is similar to [10], which is an extension of the original dynamic window algorithm by Fox et al. [11]. The global dynamic window algorithm assigns costs to all reachable velocities around the current velocity of the platform. The cost of each velocity depends on the path tracking behavior, on the distance to the actual user input and to the likelihood to cause a collision, if the velocity is executed. costDW (ωi , vi ) = αgrad · cgrad (ωi , vi ) +αuser · cuser (ωi , vi ) + αcollision · ccollision (ωi , vi ) (10) Cost cgrad (ωi , vi ) is calculated by integrating the dynamic window velocity v DW = (ωi , vi ) for a certain time interval ∆TDW . The gradient cost is directly proportional to the

difference in position (∆x, ∆y) and orientation (∆θ) from the path to be tracked according to the following expression: p  β (∆x)2 + (∆y)2 1 − cos(∆θ) 2 + β1 · cgrad (ωi , vi ) = ∆Lmax 2 (11) where ∆Lmax is the maximal linear displacement on the trajectory for ∆TDW . β1 and β2 allow to control the weight and sharpness respectively of the angular displacement in the gradient cost function. The user cost cuser in eq. 10 is proportional to the Euclidean distance between the dynamic window velocity, v DW = (ωi , vi ), and the actual user input, udis k . The collision cost ccollision punishes velocities leading to a collision if executed during a certain time interval. B. Adaptive decision making The global dynamic window approach presented in the preceding section offers the means to merge the input of the user with the velocities needed to track safe trajectories. In order to avoid frustrating the user, the weights αgrad , αuser and αcollision in eq. 10 cannot be held constant. If the user is able to execute a certain maneuver, full control should be granted to him/her. On the contrary, if the user experiences difficulties, he/she should be assisted in a safe manner. Therefore, it is mandatory to vary the weights in eq. 10 in a continuous way according to the estimated degree of control. If the user disagrees with the behavior of the robotic wheelchair, he/she should be able to override the output of the controller. From Fig. 3, where the user model is displayed for user plans requiring a steering signal slightly to the left or center and to the right, it can be observed that the user model distribution tends to be more peaked at the threshold angular velocity, ωth , the further the predicted intentional velocity, ω p , lies
to the left. The conditioned entropy, H ω dis | ω p = ω ˆ p captures precisely this observed phenomenon (cf. Fig. 4) and, therefore, is employed to decide on the degree of control granted to the user.
H ω dis | ω p = ω ˆp = Z


− puser ωkdis |ik · ln puser ωkdis |ik dωkdis (12) In the proposed decision criteria the user has less control when his/her signals are close to the threshold angular velocity, ωth . This fact is justified since the user cannot steer to the left. Thus, if the wheelchair steers to the left, the user could compensate for an incorrect action by steering back to the right. Consequently, steering to the left has a higher informative value for the proposed driving disability. The entropy of the user model could also be employed to estimate some part of the degree of control granted to the user for other types of driving disabilities. The entropy of the user model measures the degree of uncertainty of the user model, i.e. of the confidence in the present user signals. However, the conditioned entropy of the user model is not enough to decide on the level of control granted to the user. It is also necessary to incorporate the confidence of the

pre-print: IEEE/RSJ International Conference on Intelligent RObots and Systems (IROS2007), San Diego, California. pp. 3360-3366

` ´ Fig. 4. Conditioned entropy H ω dis | ω p = ω ˆ p depending on the P position of the predicted intentional velocity ω ˆ .

shared controller in the uncertain plan estimates. Tracking wrong intentions might frustrate the user. Therefore, an additional criterium is introduced related to the percentage of uncertainty on the plan estimates. Such uncertainty is measured as the entropy of the posterior distribution (cf. eq. 1) with respect to the entropy of the uniform distribution, which denotes the case of maximal uncertainty. In order to avoid frequent mode switches, the degree of autonomy is smoothed with a running average of N samples. Should the user disagree with the provided assistance, then he/she can override the output of the shared controller by releasing the joystick. This is detected by the system and full control is granted back to the user in order to disambiguate the situation. As the degree of control is smoothed, granting full control corresponds with initializing all the values of the running average to 100. The adaptive decision making rule for the assigned degree of user control (DC) is summarized in eq. 13: " 
# dis p  ˆp + avg γuser · H ω | ω = ω udis 6= 0 DC = γcont · Hk (ik |uk , H0:k )   initialize avg [100] udis = 0 (13) where, γuser and γcont allow to balance the influence of both terms. A good compromise for the weights γ is γuser = γcont = 0.5, showing good experimental performance. The weights of the dynamic window (cf. eq. 10) are adjusted according to the estimated degree of control at each time step, i.e. αuser = 100 − αgrad = DC. V. EXPERIMENTAL RESULTS The proposed approach for plan recognition and shared control was validated with K.U.Leuven’s robotic wheelchair Sharioto (refer to Fig. 1) in the home-like environment displayed in Fig. 5. Several a priori intents have been placed in locations of potential interest to the user. In this environment, a user with a weak left signal1 was asked to perform maneuvers involving turns to the left. The maximal turning velocity the user could achieve for left turns was set to ωth = 0.0812 rad/s, which is too low to turn on the spot with the motor controller of our electrical wheelchair Sharioto. 1 The weak left signal was simulated by distorting the original input of an able-bodied user according to the mapping specified in eq. 5.

Fig. 5. Maneuver executed in a home-like environment. Possible intentions are displayed in red.

The experiment began at the pose marked as start and finished when the user reached the pose denoted end. With the imposed steering disability the user could not have performed such a maneuver. The shared controller detected the plan of the user and assisted whenever assistance was needed. On Fig. 6 it is possible to observe the evolution of the probability density function (pdf) over user plans for the executed maneuver. The red circles denote the maximum of the pdf, determining which path should be tracked at each time instant, and the solid blue line shows the true user intent. In the first part of the maneuver, the user reaches intention

Fig. 6. Evolution of the probability density function (pdf) during the maneuver. The circles denote the maximum of the pdf, whereas the solid blue line denotes the true user plan. Darker colors indicate a higher value of the pdf.

3 (time steps 1 − 86). This is correctly detected as the pdf converges to a higher likelihood of plan 3. Once the user has reached end pose 3, he will try to get out of that position. Given that he cannot steer to the left and turning to the right is not possible, all the user can do is to drive backwards (time steps 86 − 98). Then, the user probes if the shared controller already detects that he intents on turning (time steps 98−117). The user plan with highest likelihood remains plan 3 and the wheelchair drives again forward. User plans 2 and 4 have a higher likelihood during time steps 113 − 136, which causes a slight turning of the wheelchair to the left. In order to communicate his disagreement with the observed

pre-print: IEEE/RSJ International Conference on Intelligent RObots and Systems (IROS2007), San Diego, California. pp. 3360-3366

action, since the wheelchair is not turning enough, the user releases the joystick (time steps 136 − 153). This is detected and the user is granted a higher degree of control (cf. Fig. 7). During time steps 153−177 the user drives backwards again, this time for a longer distance. Afterwards, the user drives forward with the plan of driving to pose 1. User plans 2 and 4 have the highest likelihood (time steps 168 − 183), matching the user plan of driving forward. Not wanting to be trapped again in the corner of pose 3, the user releases the joystick showing disagreement. Note that plan 3 has the highest likelihood during time steps 183−195. The wheelchair stops and the degree of user control is immediately increased to its maximal value. The controller realizes through a pdf where the maximal values change quickly, that the user wants to reach an intention on the left and turns the wheelchair. For this maneuver, the user is granted no control at the beginning (cf. Fig. 7 situation 203−212). During turning, the user plan with highest likelihood is user plan 5 (time steps 202 − 232). At almost the end of the maneuver (time steps 237 − 254),

a general user model, the user’s plans are estimated before deciding whether and to which extent the user needs assistance. In the second part of the paper, a shared controller is presented, which tracks the maximum probability fine motion paths as computed by the plan recognition framework. The shared controller uses an adaptive decision scheme, which explicitly incorporates the uncertainty on the plan estimates when merging the input of the user with the output of the path tracking algorithm. Encouraging experimental results are presented on a real wheelchair for a user with a weak left steering signal. This user is able to reach poses in the environment, which would not be accessible to him without intelligent assistance. VII. ACKNOWLEDGMENTS The authors gratefully acknowledge the financial help of the Belgian Government under the Inter-University Attraction Poles, Office of the Prime Minister, IAP-AMS, the EU STREP IST MOVEMENT project and the EU STREP IST MAIA project. A. H¨untemann is research assistant with the Research Foundation Flanders FWO. R EFERENCES

Fig. 7. Degree of control granted to the user during the maneuver. The higher the value, the more control is granted to the user. The user can override the decision of the shared controller by releasing the joystick, after which the degree of user control will automatically increase to 100%.

user plan 3 has the highest likelihood, in accordance with the input given by the user. The user is steering backwards in order to brake the wheelchair. Once the wheelchair is driving slowly, the user pushes the joystick slightly forward to reach end pose 1. Observing the pdf evolution in Fig. 6 we can deduce that the plan recognition process does not always converge to the true intent. This will not cause any unpredictable behavior of the wheelchair, as long as the maximum probability path corresponds, for a certain interval, to the path the user has in mind. VI. CONCLUSIONS This paper presented a framework for adaptive plan recognition and shared control. Users are modeled as path tracking controllers steering the wheelchair according to mental trajectories they wish to follow. Since users might not have a constant driving performance, or since they might suffer from various forms of disabilities, a personalized performance model is introduced to account for these effects. The performance model relates the actual driving behavior of the user to a reference, idealized user corresponding to the previously mentioned path tracking controller. With such

[1] L. Fehr, W. E. Langbein, and S. B. Skaar, “Adequacy of power wheelchair control interfaces for persons with severe disabilities: A clinical survey,” Journal of Rehabilitation Research and Development, vol. 37, no. 3, May-June 2000. [2] H. Hoyer, R. Hoelper, U. Borgolte, C. B¨uhler, H. Heck, W. Humann, I. Craig, R. Valleggi, and A. Sabatini, “The omni wheelchair with high manoeuvrability and navigational intelligence,” Rehabilitation Technology, Proc. of the 2nd TIDE Congress, 1995. [3] S. P. Parikh, V. Grassi Jr., V. Kumar, and J. Okamoto Jr., “Incorporating user inputs in motion planning for a smart wheelchair,” in Proceedings of the 2004 IEEE International Conference on Robotics & Automation (ICRA), New Orleans, LA, April 2004, pp. 2043–2048. [4] H. Yanco, “Shared user-computer control of a robotic wheelchair system,” Ph.D. dissertation, Massachusetts Institute of Technology, 2000. [5] N. Katevas, N. Sgouros, S. Tzafestas, G. Papakonstantinou, P. Beattie, J. Bishop, P. Tsanakas, and D. Koutsouris, “The autonomous mobile robot senario: A sensor-aided intelligent navigation system for powered wheelchairs,” IEEE Robotics and Automation, vol. 4(4), pp. 60– 70, 1997. [6] R. C. Simpson, S. P. Levine, D. A. Bell, L. A. Jaros, Y. Koren, and J. Borenstein, “Navchair: An assistive wheelchair navigation system with automatic adaptation,” Lecture Notes in Artificial Intelligence, vol. 1458, pp. 235–255, 1998. [7] E. Demeester, A. Huntemann, D. Vanhooydonck, G. Vanacker, A. Degeest, H. Van Brussel, and M. Nuttin, “Bayesian estimation of wheelchair driver intents: Modeling intents as geometric paths tracked by the driver,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5775 – 5780, 2006. [8] M. Bennewitz, W. Burgard, and S. Thrun, “Using EM to learn motion behaviors of persons with mobile robots,” Proceedings of the International Conference on Intelligent Robots and Systems (IROS), 2002. [9] J. Glover, S. Thrun, and J. Matthews., “Learning user models of mobility-related activities through instrumented walking aids,” In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2004. [10] E. Demeester, M. Nuttin, D. Vanhooydonck, G. Vanacker, and H. Van Brussel, “Global dynamic window approach for holonomic and non-holonomic mobile robots with arbitrary cross-section,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2694–2699, 2005. [11] D. Fox, W. Burgard, and S. Thrun, “The dynamic window approach to collision avoidance,” IEEE Robotics and Automation Magazine, vol. 4, no. 1, pp. 23–33, 1997.