Auton Robot (2006) 20:239–249 DOI 10.1007/s10514-006-7098-8
Smell, think and act: A cognitive robot discriminating odours Amy Loutfi · Silvia Coradeschi
Published online: 8 June 2006 C Springer Science + Business Media, LLC 2006
Abstract In this paper, we explore the integration of an electronic nose and its odour discrimination functionalities into a multi-sensing robotic system which works over an extended period of time. The robot patrols an office environment, collecting odour samples of objects and performing user requested tasks. By considering an experimental platforms which operates over an extended period of time, a number of issues related to odour discrimination arise such as the drift in the sensor data, online learning of new odours, and the correct association of odour properties related to objects. In addition to an electronic nose our robotic system consists of other sensing modalities (vision and sonar), behaviour-based control and a high level symbolic planner. Keywords Electronic noses . Odour discrimination . Mobile olfaction . Anchoring . Planning for perceptual actions
1. Introduction Research in the area of mobile olfaction has mainly focussed on the use of gas sensors in robotic systems for the investigation of odour based navigation strategies. Examples of these platforms include (but are not limited to) trail following (Russell et al., 1994), plume tracking (Kazadi et al., 2000) and odour source localization (Ishida et al., 1996; Lilienthal and Duckett, 2003). In the previously cited work, often single sources and single selectivity gas sensors are used where the chemical composition of the target gas is known a priori. The use of electronic noses per se, that is an array of gas sensors A. Loutfi () · S. Coradeschi ¨ Center for Applied Autonomous Sensor Systems, Orebro ¨ University, 701-82 Orebro, Sweden
with a pattern recognition component for the discrimination of simple and complex odours (Persaud and Dodd, 1982), have only been considered in a few cases. Works which include odour recognition for odour source localization are (Rochel et al., 2002) and (Marques et al., 2002) where the discrimination between two odours aided in order to navigate to specific source in the presence of multiple sources. Today’s electronic noses, however, are capable of distinguishing between several complex compounds. They have been used in the discrimination of a range of different odours sources such as food (Guardarrama et al., 2001; O’connell et al., 2001), biological sources for medical applications (Dutta et al., 2002; Keller et al., 1995), and environmental sources (Delpha et al., 2001). A number of commercial and even hand-held electronic nose devices are now available (Cyranose-Sciences, 2000; AppliedSensor, 2000). The integration of the full abilities of electronic olfaction onto a robotic system would enhance the perceptual competence of the robot and promote a new generation of applications for mobile olfaction. However, given that many of today’s robotic systems are complex, often with multiple sensing modalities and intended for a diversity of tasks, this integration is non-trivial. In previous work, we have shown that the integration of an electronic nose in a robot that can plan and act autonomously, requires a forethought at each level of the robotic architecture. At the sensing level, the data coming from other modalities such as vision need to be combined with olfactory information to maintain a coherent perceptual representation of physical objects (Loutfi et al., 2005b). The sensor characteristic and operation of the e-nose also needs to be taken into account at the control level so that the robot can collect good samples for classification. This in turn requires deliberation at the cognitive level in order to plan for sensing actions and resource allocation (Loutfi et al., 2004). AddiSpringer
240
tionally, information about odour property should be made available for users in an easily accessible form so that the user can interact effectively with the robot (Loutfi and Coradeschi, 2005). We showed that odour recognition in a mobile system is possible and that existing techniques in both AI and robotics can be used to benefit the perceptual competence of the robot as well as to improve the performance of an odour recognition component. The system performed a number of odour related experiments and tested the feasibility of using olfaction in a robotic system under different scenarios (Loutfi et al., 2005a). However, so far each of these scenarios were evaluated individually, and like many experiments done in robotics system, the experimental environment was carefully tailored to minimize disturbances and noise. In this work, we present a crucial extension for our olfactory robotic system. The experimental scale is augmented to a more realistic application where the robot is operating over an extended period time in a partially dynamic environment. Our experimental setup no longer consists of a number of mutually exclusive tasks but rather a large scenario where the robot patrols and inspects a corridor over a number of days and perform a number of user requested tasks. The user requested tasks can be either one of the following: thoroughly inspect a given object, find an object that matches an odour sample, or find an object that matches a symbolic description. By considering a longer experiment new dimensions to the integration issues mentioned above become of interest. At the sensor level, the correct attribution of sensor data to an object must occur both across the different sensor modalities but also over time. At the cognitive level, tasks should be scheduled and resources correctly allocated without sacrificing performance quality. At the odour detection level, odour classification should be able to handle the interleaving of different tasks and the acquisition of new odours online. The paper begins with a description of the system architecture in Section 2. Section 3 describes the general setup of the long term experiment with a brief description of the olfactory-related tasks the robot can perform. In Section 4 the experimental are presented. Finally, we conclude with a discussion of future works and improvements in Section 5.
Auton Robot (2006) 20:239–249
2.1. Olfactory module The olfactory module consists of a commercially available electronic nose, a pattern recognition and classification component, and an electronic odour repository of odours stored in the on-board computer of the robot. The e-nose contains 32 thin-film carbon-black conducting polymer sensors of variable selectivity (Cyranose-Sciences, 2000). Each sensor consists of two electrical leads on an aluminium substrate. Thin films are deposited across the leads creating a chemiresistor. Upon exposure to an analyte, a polymer matrix absorbs the analyte and increases in volume. This increase in volume is reflected in a increase in resistance across the sensor. Each polymer in the array is unique and designed to absorb gases to different degrees, creating a pattern of responses across the array. The array of sensors are contained in a portable unit also consisting of pumps, valves and filters that are required to expose the sensor array to a vapour gas. 2.2. Odour sampling The sampling of an odour occurs in three phases. The first phase is a baseline purge, where the sensors are exposed to a reference gas. The reference gas is filtered air that has been prepared in a ventilation cabinet and is contained in a 50 ml sealed vile and fixed onto the robot. The duration of the purge is 30 seconds and is based on the manufacturer recommendation when a charcoal filer is used. The second phase is a sampling cycle where a valve is switched to allow vapours from a sampling inlet to come into contact with the sensing array. The duration of this cycle is 20 seconds and a substrate temperature is maintained at 42.0◦ C during User goals & tasks
Planner (top-level & recovery) Interface Plan
Plan executor & monitor Request
The system architecture is a sense-plan-act architecture with an added middle layer used for connecting symbols to sensor data, called an anchoring layer (see Fig. 1). This section outlines each of the components in the system beginning with the lower level sensing processes and proceeding to describe the higher-level processes of the system. Springer
Anchors,Status
Destination
Odour Processing Commands
Data
E-Nose
Classes
Status
Navigation Planner
Anchoring Commands
2. System architecture
Problem (state & goal)
Percepts
Vision Processing Images
CCD Camera
Behaviours
Status
Fuzzy Control Commands
Status
Robot Control, Odometry, etc..
Fig. 1 Overview of the complete system. On the left side of the arrows information is flowing downward and on the right side information is flowing upward
Auton Robot (2006) 20:239–249
241
Fig. 2 Sampling setup for the electronic nose
Sampling Inlet (snout) Stainless Steel
Baseline Filter Poly-paraxylene
(Charcoal)
Sensor Array
Pump
Valve Reference gas
Exhaust
sampling. Finally, a sequence of purging cycles is used to remove the sampled odour from the unit and restore the sensor response to the baseline values. A schematic of the sampling setup for the electronic nose is giving in Fig. 2. Figure 3 (Left) illustrates a typical response of all sensors throughout the phases of sampling. 2.3. Odour classification The signals are gathered in a response vector Rk where each sensor’s reaction is represented by: Rk,s =
Rmax,s − Ro,s Ro,s
(1)
for s = 1 to 32. Rmax,s is the maximum signal during the measurement cycle and Ro,s is the baseline response. Each odour sample k is also represented by a linguistic label lk , where lk ∈ D is a user specific discrete set. For example D may contain the collection of odour names. The complete data set is normalized, by dividing each Rk,s by the maximum value for each sensor, in order to set the range of each sensor to [0, 1]. Principal component analysis
(PCA) is performed and the two first principal components are used as features for the classification algorithm. In this work we implement a semi-supervised fuzzyclustering algorithm to perform classification. The use of a fuzzy algorithm provides for the explicit representation of uncertainty in the classification output (this uncertainty can later be used by the planner to initiate further perceptual actions); it also provides for easy integration of symbolic representation without the need to map specific outputs to categorical names.1 The algorithm applied is based on a combination of the fuzzy c-means (FCM) and a fuzzy maximum likelihood estimator (FMLE) (Gath and Geva, 1989). With these algorithms, an initial estimate of the number of clusters is needed so that the data set is split into C fuzzy groups. To determine C for optimum cluster partitioning the algorithm is applied iteratively (each time augmenting C) and subsequently evaluating a number of validity measures. Here, a validity measure called the Compactness and Separation measure is used to evaluate the global attributes of the data space. This measure minimizes for compact, dense, and well-separated clusters (Xie and Beni, 1991). The FCM algorithm normally splits a set of patterns into a number of homogeneous clusters with respect to a similarity measure. The patterns belonging to any one of the clusters are similar and the patterns of different clusters are as dissimilar as possible. However, FCM is not suited for clusters of different shapes, sizes and densities and therefore the FMLE extension to the algorithm is used. To assign each cluster an odour character, a contingency table is used that represents the probability of an odour with label lk occurring within a prototypical region around the cluster center ci . Prototypical regions are defined as a region 1
Fig. 3 (Left) Example of the raw sensor data throughout the three phases of sampling an odour
Although a variety of pattern recognition techniques can be applied to electronic nose data and other array based sensor data analysis, no one method has been identified as superior in all experimental conditions. Rather, certain algorithms are used depending on the success of determining the underlying relationships between one set of independent variables and another set of dependent variables. In our system, the fuzzy algorithm gave comparable performance to other statistical and non-statistical techniques (Loutfi and Wide, 2002). Springer
242
Auton Robot (2006) 20:239–249
on the data space where points have full membership to a cluster. In this algorithm, the membership degree, u i,k , of the kth feature to the ith cluster is calculated using:
u i,k =
c j=1
1 2 d F2 j (xk ,vi ) m−1
(2)
d F2 (xk ,v j ) j
where d F2 j (xk , vi ) is the FMLE distance between the kth feature vector and the ith cluster center and m is a fuzzification factor set to 2. Labels of points which most frequently occur within a cluster’s prototypical centre are then used as the naming convention for that cluster. It is in this way, that the electronic perception of odours is preserved meanwhile, co-ordinating the symbolic representation of odours used by a human. More detail on the odour classification can be found in Loutfi and Coradeschi (2004). Unlabelled data is identified by calculating a matching function βx from linguistic labels to real values as given below: βx lci = u i,x
(3)
Where u Rx given by Eq. (2) above. Both the label of the cluster and the membership value to which βx (lci ) has the highest value is given as the classification result for the odour. 2.4. Vision module In addition to the sonar used to navigate and detect obstacles, the system also uses vision to perceive objects. This is done by using a frame grabber card and taking snapshots from a CCD camera, which are processed on board the robot in order to identify objects and their properties. For the image processing we are using standard techniques. First the captured image is pre-processed using standard filters and segmented using mean shift segmentation in the Luv color space. After segmentation each segment is examined and pattern matched against the contours of a number of predefined objects. For each found object we extract a number of properties such as color, size and relative position using different heuristics. The collection of the properties belonging to an object is called a percept. The system works well for cases where we have single colored, convex objects standing on the floor. Images are received once every second taking into account the total execution speed of the image processing when executed together with all other processes. Springer
The percepts from the vision processing is sent over TCP/IP to a perception process. This perception process is using the percepts in order to build a database of perceived objects, attempting to match percepts to earlier seen objects or creating new objects for completely new percepts. Together with some simple filtering this gives a representation of time persistent objects as opposed to percepts only valid for one frame. In order to maintain and predict sensed objects currently outside the camera’s viewpoint an odometry based localization is used so that objects not currently perceived can be remembered. For limited movements, the system can easily reacquire objects based on their stored position. 2.5. Robot control and navigation architecture The last low-level process to be explained is the control and navigation system. It is organized in a three layer hierarchical structure and consists of sensing modules (odometry, sonar, and vision, etc.), a fuzzy behaviour-based controller, and a navigation planner. The fuzzy behaviour based controller proposed by Saffiotti et al. (1995) uses fuzzy logic to determine activation levels of different behaviours such as obstacle avoidance. Fuzzy behaviours are easy to define and provide a robustness with respect to sensor noise, modelling errors and imprecision (Driankov and Saffiotti, 2001). The main behaviours used in our experiments are: “Go-Near” behaviour, which approaches an object to obtain a good odour sample; “Avoid” which performs obstacle avoidance; “Orient” used to orient the robot towards a particular object or corridor: “Follow-Corridor” which uses the sonar readings to remain within in the corridor. The navigation planner is based on standard search techniques and decides what sequence of behaviours should be active to reach a specific target or goal. Plans are generated in the form of a set of “situation → behaviour” rules. These are called B-plans and are used by the top-level plans to formulate actions described later.
2.6. Anchoring module The purpose of the anchoring module is to maintain the correspondence between symbols used by the planner (when deciding about which actions to execute) and the sensor data referring to specific objects. This correspondence is maintained in an internal data structure called an anchor (Coradeschi and Saffiotti, 2003). Each anchor consists of several fields which are either populated in a bottom-up or top-down fashion. Bottom-up anchors are created when the vision system detects a new and persistent object. Top-down anchors are created by special request from the user. Each anchor can be described by the following components:
Auton Robot (2006) 20:239–249
243
Fig. 4 (Left) Segmented image from the vision module observing two green garbage cans at the center of the screen. (Right) The anchors created in a bottom-up manner for each object. Gar-34 refers to the garbage can on the left of the image and Gar-36 refers to the garbage can on the right
Name — For top-down anchors a name is a symbol denoting the object in the planner (e.g., Cup-22). For anchors that have been created bottom-up the name is initially arbitrary. Symbolic description—In general a symbolic description is given in a top-down fashion, however for bottom-up anchors the symbolic description may also be derived directly from the perceptual information of the object. For example, the odour classification module may populate the symbolic description with the linguistic odour name βx when classifying an odour associated to a particular object. Perceptual description—The perceptual description is a vector which consists of the important properties of the object such as position (relative and global), colour, shape, and when available the odour signature, Rx . To create, maintain and eventually remove anchors a number of functionalities are used. These functionalities have been previously described in Loutfi et al. (2005b) and therefore are only summarized here: Acquire functionality—it creates anchors from the perceptual system when objects of interest are recognized. Recall, that in our system a set of pre-determined objects are given a priori. A perceptual description and the symbolic description are populated with the current information about the object and an arbitrary name is assigned. Track functionality—it is called at every perceptual cycle of the system (200 ms), and updates the anchor when current information is available. The maintenance of anchors is a complex problem, here we use a relatively simple fuzzy matching to attribute current perceptual information to the existing perceptual description in the anchors. More detail on the tracking algorithm can be found in (Coradeschi and Saffiotti (2000). Find functionality—it is used to connect the symbolic name used at the planner level with the anchors created bottom-up. A symbolic name and symbolic description are given and anchors best corresponding to that description are returned. Clearly ambiguous situation may arise, and this problem is considered in Broxvall et al. (2005). To summarize, the anchoring module is used for two reasons. The first is to connect the symbolic information at the
planning level to the perceptual information, eventually facilitating the execution of the actions decided by the planner. Another important function of the anchoring module is to maintain all the perceptual information relating to the same physical information in a single structure. This is especially important since in this system, perceptual information may come from different sensing modalities and at different times. Figure 4 shows an example of two anchors that have been created bottom-up from a segmented image. 2.7. Planning—autonomous generation of high-level tasks Planning is used to autonomously decide which actions the robot should perform. The presence of the planner is especially important to co-ordinate smelling actions with other perceptual and navigation tasks. PTLplan is a planner for partially observable domains with uncertainty (probabilities) (Karlsson, 2001). It searches in a space of epistemic states, or e-states for short, where an e-state represents the agent’s incomplete and uncertain knowledge about the world at some point in time. An estate can be considered to represent a set of hypotheses about the actual state of the world. Each sub-state in the e-state consists of a set of value-assignments to properties or state variables, where a property might represent for instance the shape or smell of a particular object: (shape b1 = cup), (smell b1 = ethanol) .Sub-states are also associated with degrees of probability. The planner can reason about perceptive actions, such as looking at or smelling an object, and these actions have the effect that the agent makes observations that may help it to distinguish between the different hypotheses. Each different observation will result in a separate new and typically smaller e-state, and in each such e-state the agent will know more than before. Figure 5 illustrates the effects on e-states of two actions: one for movement, and one for perception (smell). It would also be possible to represent sensor errors—observations need not always coincide with the actual state. PTLplan is a forward-chaining planner, which basically adds one action at a time, introducing conditional branches Springer
244
0.5 robot–at = r11 smell b1 = ethanol shape b1 = bottle 0.5 robot–at = r11 smell b1 = noteth shape b1 = bottle
Auton Robot (2006) 20:239–249
Action: move–near b1 obs:
0.5 robot–at = b1 smell b1 = ethanol shape b1 = bottle 0.5 robot–at = b1 smell b1 = not–eth shape b1 = bottle
Action: smell b1 obs: smell=ethanol
obs: smell=not–eth
0.5 robot–at = b1 smell b1 = ethanol shape b1 = bottle
0.5 robot–at = b1 smell b1 = not–eth shape b1 = bottle
Fig. 5 Epistemic states (thick borders) contain one or more sub-states with associated probabilities (hypotheses), and are transformed by the applications of actions (arrows). Note how different observations yield different new e-states
whenever several observations are possible (like after the smell action in Fig. 5), and keeping the search space down by the use of strategic knowledge formulated as logical formulae. It searches for a plan with a probability of success above a given threshold, and returns such a plan as soon as it is found. Certain situations may arise where recovery planning may be necessary. This may require the robot to gather more information about the different perceived objects in order to resolve ambiguities or reacquire existing objects. 2.7.1. Formulation of initial situation and goal To initiate a recovery plan for additional perceptual actions, a number of hypotheses relating the different ways the requested object, (b1) , can be anchored is generated. This processe requires (1) the description of the observable properties of the requested object b1 , and (2) the observed properties of each perceived object. Based on the descriptions d for the requested object and di for each perceived object poi , two extra descriptions are formulated for every di : first, a description di+ which completely matches d; and second, a non-matching description di− which contains the different ways in which at least one incompletely specified property in di may not match with d. Each hypothesis then consists of the conjunction of one di+ for one of the poi and d − j for all remaining j = i. To each hypothesis is also added the statement (anchor b1) = poi ) denoting that b1 should be anchored to the object anchored by poi . There is also one hypothesis that no object matches: d − j for all j, and (anchor b1 = none). Finally, if the planner wishes to take a cautious approach and ascertain that no more than one object is matching, it might also add a number of hypotheses consisting of di+ , d + j for two of the poi , poj and dk for all remaining k = i, j, and (anchor b1 = none) . For instance, if b1 is known to be a cup containing ethanol —(smell b1 = ethanol) — and we perceive two cups anchor − 1 and anchor − 2, the (incautious) hypotheses might be: The goal is achieved once a specific action (anchor b1 x has been performed. This action has as a precondition that x is the only remaining anchor for b1: (nec (anchor b1 = Springer
0.4 (smell anchor-1 = ethanol), (smell anchor-2 = not-ethanol), (anchor b1 = anchor-1) 0.4 (smell anchor-1 = not-ethanol), (smell anchor-2 = ethanol), (anchor b1 = anchor-2) 0.2 (smell anchor-1 = not-ethanol), (smell anchor-2 = not-ethanol), (anchor b1 = none) Note that not-ethanol is defined as the complement of ethanol
x)). Thus, all other candidate anchors have to be eliminated before anchor is applied. 2.7.2. Plan generation After the initial situation and the goal have been established, plan generation starts, using the appropriate domain description. The following action, for instance, lets the robot observe the smell of an object. (ptl-action :name(smell ?y) :precond(((?y)(perceived-object ?y)(robot-at = ?y))) :results(obsval(smell ?y)) :execute((go-near ?y)(smell-behavior ?y))) In short, the precond part states that the action requires a perceived object ?y, which the robot must be positioned at. The result part states that the robot will be able to observe the smell of object ?y. The obsval form is a way to encode that the agent makes a specific observation. The right side of Fig. 5 illustrates the effect of the smell action on one specific e-state. It is in the planner’s domain descriptions that information about the capacity, constraints and uncertainty of the e-nose and other sensors are modelled.
3. Experimental setup 3.1. Environment ¨ All experiments were performed in the corridors at Orebro University. Temperature is regulated and maintained at approximately 20◦ C with a 65% RH and no dominant constant airflow was present. The experimental environment is shown in Fig. 6. The workspace and objects to be inspected are located in the three main corridors shown in the map. This rough map is also provided to the system and contains the approximate position of all the doors, rooms, and corridors plus the topological information about their con-
Auton Robot (2006) 20:239–249
245
Fig. 6 The a priori map of the corridor which the robot navigates and where target objects are located. The robot starting position is shown at point X
WC
Room T1203
Corridor 1
Room T1209
Room T1210 Conference
storage
x
Corridor 2
Room T1201
nectivity. It also contains a few metric properties such as the width of the corridors, and doors. Odour sources were dispersed at different locations in the three corridors. All odours sources were contained in green trash bins. The chemical substance was placed on a cloth-like material and placed inside each bin. Each odour source was separated by a 1.5 meter radial distance to minimize the effect of other source interference during sampling. Odour samples were refreshed at regular intervals approx. every 30 minutes.
3.2. Sampling an odour source When the call to inspect an object is made, the robot first navigates to that object using visual and odometric information. A maximum distance of d = 20 cm between the snout of the e-nose is allowed for sampling of an object. This distance was determined experimentally in Loutfi et al.(2005a) where the classification results at different distances for different odours was obtained using a mobile robot. A classification performance threshold of 80% is shown to be maintained provided the robot does not exceed a 20 cm distance from the object. This distance has only been verified for the odours used in the experiments. These results are also valid when no predominant airflow is present. 3.3. Odour-related tasks A number of odour-related tasks can be requested from the human which interupts the current patrol and inspect of the corridor.
Corridor 3
Room T1204
Room T1206
Task 1—Online learning of odours. A human presents to the robot an artefact which contains an odour property, in this case a piece of cloth sprayed in 3-hexanol. The first task of the robot is to sample the artefact and store the odour property. The next task for the robot is to determine the possible candidates with the same odour property, and visit each candidate until a match is found. Task 2—Thorough object inspection. In this experiment, the human requests the robot to thoroughly inspect an object. Thorough inspection may be required to reduce the uncertainty in an odour classification. Sources of uncertainty may be due to incorrect positioning while taking an odour sample, localisation error that places the robot too far from the object, or an uneven distribution of the odour around objects due to awkward shape, or air currents. The planner inspects the objects thoroughly by docking to the object at different angles and performing a smell action. Each sampling attempt has a certain probability p to fail, and therefore the accumulated probability of n consecutive failures (1 ≤ n ≤ 4) is p n . When the accumulated probability reaches a user-defined threshold, the robot stops performing perceptual actions and informs the user of the state of the object. Task 3—Disambiguation of visual similar objects using olfaction. To further test the capability of the planner to reason about sensing resources, an ambiguous situation containing visually similar objects is used. In this experiment, a user requests the robot to find an object given a list of properties. The properties are a symbolic representation of perceptual concepts (e.g. green cup smelling coffee). The objects are first seen using vision but are only distinguishable by their olfactory property. Therefore, a plan is created that determines how each of the objects are visited and when the smell action is executed to disambiguate the objects. Springer
246
Auton Robot (2006) 20:239–249
Fig. 7 Plot of the first two principal components. (Left) Enlarged image of Ethanol samples which show drift in the sensor data. (Right) Enlarged image of the clean air samples
3.4. Corridor monitor The corridor monitor uses the system architecture described in Section 2 with an added pre-compiled scheduler that initiates the task to patrol the corridors, inspect objects or perform user requested tasks. The robot is an iMagellan pro indoor research robot equipped with the Cyranose 320 electronic nose (Cyranose-Sciences, 2000) and the CCD camera described in Section 2. The scheduler was running for a total of 4 days with breaks for charging the batteries and breaks during the nights. Depending on the number of movements and smelling actions, batteries could run the robot for 3–4 hours, charging the batteries took an equal amount of time. Over the four day period an approximate distance of 1.5 km was covered by the robot. Each patrol begins and ends at point X located on the map. Unless a task is requested by the user the robot patrols the corridor and performs a subsequent inspection of each object which involves collecting the object’s odour property. 4. Experimental results 4.1. Classification performance Over the four day period, 87 odour samples were collected. Five different odours were used, Hexanal 98% (C6 H12 O), Linalool 98% (C10 H18 O), Ethanol (C2 H6 O), Ammonia (H3 N) (common cleaning fluid) and Clean Air. The choice of this particular selection of odours was based primarily on availability. Table 1 outlines the classification performance of the algorithm vs. other standard multivariate techniques. The test/unlabelled data set consisted of 50% of the entire 4 day samples. In the first column, only Day 1 training set was used
Springer
to classify the test data. In the second column, an accumulation of Day 1 to Day 3 training data is used to classify the test data. The classification results improve using a larger training set which confirms the need to acquire and update data in an online manner and use that updated information for future tasks. The patrolling task is essential in order to capture the dynamic changes in the sensor data. The mobility and the autonomy of the robot can be exploited to maintain an up-to-date odour repository. When comparing the performance of the fuzzy clustering algorithm against different classification algorithms (linear discriminant analysis and artificial neural network), no significant difference in performance can be noticed. This suggests that the different odours are well separated and that the clusters in the unsupervised algorithm correlates well to the human perception of odours. A further inspection of the odour samples in Fig. 7 of the two first principal components, reveals a drifting effect in the sensor data. This effect is known as short term drift and is a common phenomenon with electronic nose sensors. Optiminum number of clusters C from the validity analysis occurs with seven clusters and not five, this is due to the high variance in the sensor data caused by a drifting effect. Figure 8 shows the results from the validity analysis and the placement of cluster centers using the fuzzy algorithm. Table Table 1 Method
Classification performance with different training data sets Day 1 Training (%)
LDA 82 ANN (Supervised MLFF) 87 FCM-FMLE 87
Day 1–3 Training(%) 95 98 97
Auton Robot (2006) 20:239–249 –3
4
247
Global Validity Analysis for Partition Determination
x 10
Odour Data and Unsupervised Cluster Generation 0.15 odour cluster 0.1
3.5
1 2
0.05
3
validity measure
3
0
7 5
2.5
–0.05 6 –0.1
2
–0.15 4
1.5
–0.2 1
2
3
4
5
6
7
8
prototypes
9
–0.25 –0.7
–0.6
–0.5
–0.4
–0.3
–0.2
–0.1
0
Fig. 8 (Left) The validity analysis yielding optimum partitioning with seven clusters (prototypes). (Right) The unsupervised clustering result of the data set collected over the 4 day run Table 2
Cluster number with respective symbolic description
Cluster number
Symbolic representation
1 2 3 4 5 6 7
Ethanol Ethanol Ethanol Linalool Ammonia Clear Air Hexanal
a success is considered when a matching object to the odour sample is found. For Task 2, a success is considered when a single object is correctly classified after thorough inspection. For Task 3, a success is registered when a correct object is found given a symbolic description in the presence of 5 visually identical objects. The performance of the task executed in the context of a corridor monitor is similar to the performance of the tasks executed individually. There is a correlation between successful performance and the average distance from an object when taking an odour sample. The further away we are from the odour source the less likely the odour is correctly identified. In the first task, the performance results of the individual task vs. the task performed in the corridor monitor are comparable. Meanwhile performing a thorough inspection of objects in the context of the larger experiments give a slightly worse performance compared to the task performed individually. The major source of error is an accumulation of odometric errors which increases in the corridor monitor experiments because of the extra distance required to navigate to the object. When re-approaching an object to take an additional sample, sometime the object is not seen again and its positional information in the anchor is not updated. The result is an accumulating odometric error for each new approach to the object resulting in a higher probability to fail. The same source of error is also responsible for the performance difference result in the third task.
2 shows the symbolic names for each cluster in Fig. 8 (Right) extracted from the contingency analysis. The additional two clusters, however, do not affect the symbolic interpretation of odour classification. Future work will attempt to use drift compensation techniques as part of the odour analysis in the system (Holmberg et al., 1996; Natale et al., 1995). Notice that in these experiments all odour samples were acquired online with the robot during the monitoring of the corridor. 4.2. Task execution during the corridor monitoring Each of the three tasks described in Section 3.3 was performed 20 times throughout different patrolling episodes. Table 3 Column 1, shows the performance success of the task as a singular task, from (Loutfi et al., 2005a). Column 2 of the table shows performance success of the tasks when integrated into the corridor monitoring system. For Task 1, Table 3
Comparison of performance results from single tasks vs. tasks in a long term scenario
Task
Individual task (% of success)
Within the corridor monitor (% of success)
Average distance d from object when smelling (cm)
1 - Finding objects by smell 2 - Thorough inspection 3 - Diambiguate (5 objects)
74 82 76
80 75 70
15.5 19.8 16.2
Springer
248
Both experiments demonstrate the ability to perform multiple tasks on a robotic system without the explicit need to fine tune the system for each task. The olfactory module has been integrated into the system in such a manner as to facilitate the collection of odours online and the execution of a variety of different olfactory tasks. These tasks are independent of one another yet rely on the same classification of odours and representation of objects. We have presented three specific tasks, however, the system has been constructed so that new tasks can be easily included.
5. Discussion and future works In the introduction, we present three challenges that are relevant for a robotic system when considering the extension from performing single tasks to performing over a long term. We show that for an odour classification module to handle the multiple tasks with different goals an updated repository is needed. To accommodate for the drift in the sensor data, a robot can be used to collect odour samples and maintain a fresh representation of odours, thereby improving classification performance. The interleaving different tasks such as patrolling and inspecting objects is required for the maintaince of coherent perceptual information of objects and their properties. The planner is used to correctly activate sensing resources as well as determine which behaviours were necessary to activate. The ability to reason about object properties at both the low-level and the planning level is essential to perform the tasks. Connecting these two levels is the anchoring component, and although a statistical evaluation of the reliability of the anchoring module is yet to be done, the fact that high level tasks were successfully performed during the experiments is an indication that the anchoring module is maintaining a sensor-symbolic connection. As the field of electronic olfaction develops, the integration of electronic noses onto robotic systems will become increasingly popular. It is likely that tomorrow’s robot will be used to accomplish a variety of task, equipped with numerous and heterogeneous sensing modality and many system will have cognitive process that facilitate human-computer interaction. The integration of electronic olfaction onto such systems is non-trivial. Consideration of how to cope with olfaction at each level of the system architecture is required. However, a relationship exists between the olfactory module and the other components of the system and this relationship can be exploited to improve classification performance. A number of open issues still need to be addressed. In the experiments, the patrolling behaviour was instrumental for improving classification results, however, the accumulation of odometric errors which arose had an affect on the performance of odour sampling. A good localization and mapping strategy is needed for better perceptual maintenance and for Springer
Auton Robot (2006) 20:239–249
performing precise movements towards objects. Future work will investigate better techniques for collecting odour samples with a robot. Furthermore, the current technological state of gas sensors still require a long and cumbersome sampling procedure in order to achieve good data samples for discrimination. The robustness and reproducibility of the sensor data is poor. Short-term drift is present and although this can be addressed using a variety of techniques, long term drift over months depends largely on sensing history of each individual e-nose. Experience shows that each electronic nose that is integrated on a mobile robot needs to build its own training repository. These problems will need to be addressed by both the electronic nose community and the robotics communities if electronic noses are to be effectively integrated on robotic systems. Acknowledgments This work has been supported by: Vetenskapsr˚adet and KK-Stiftelse. The authors would like to thank Mathias Broxvall and Lars Karlsson for their contribution to the experimental work and Achim Lilienthal for productive discussions.
References AppliedSensor 2000. AppliedSensor GmbH—Sensor modules. Online at http: //www.appliedsensor.com. Broxvall, M., Coradeschi, S., Karlsson, L., and Saffiotti, A. 2005.Recovery planning for ambiguous cases in perceptual anchoring. In: Proc. of the 20th AAAI Conf. Pittsburgh, PA, AAAI Press. Online at http: //www.aass.oru.se/ ∼ asaffio/. Coradeschi, S. and Saffiotti, A. 2000. Anchoring symbols to sensor data: Preliminary report.’ In Proc. of the 17th AAAI Conf. Menlo Park, CA, AAAI Press pp. 129–135,. Online at http ://www.aass.oru.se/ ∼ asaffio/. Coradeschi, S. and Saffiotti, A. 2003.An introduction to the anchoring problem’. Robotics and Autonomous Systems, 43(2–3):85–96. Cyranose-Sciences 2000.The Cyranose 320 electronic nose’. Online at http: //www.cyranose.com. Delpha, C., Lumbreras, M., and Siadat, M. 2001.Discrimination of Forane 134a and carbon dioxide concentrations in an air conditioned atmosphere with an electronic nose: influence of the relative humidity. Sensors and Actuators B Chemical, B80(1):59–67. Driankov, D. and Saffiotti, A. (eds.): 2001. Fuzzy Logic Techniques for Autonomous Vehicle Navigation, Studies in Fuzziness and Soft Computing. Springer-Phisica Verlag. ISBN 3-7908-1341-9. Dutta, R., Hines, L., Gardner, J., and Boilot, P. 2002. ‘Bacteria classification using Cyranose 320 electronic nose’. Biomedical Engineering Online 1. Gath, I. and Geva, A. 1989.Unsupervised optimal fuzzy clustering. IEEE Trans. on Pattern Analysis and Machine Intelligence, 11(7):773–781. Guardarrama, A., Fernandez, J., and Iniguez, M. 2001.Discrimination of wine aroma using an array of conducting polymer sensors in conjunction with solid-phase micro-extraction (SPME) technique. Sensors and Actuators-B Chemical, B77(1–2):401–408. Holmberg, M., Winquist, F., Lundstr¨om, Y., Davide, F., Natale, C.D., and D’Amico, A. 1996.Drift Counteraction for an electronic nose. Sensors and Actuators B, 35–36:528–535. Ishida, H., Suetsugu, K., Nakamoto, T., and Moriizumi, T. 1996:Study of autonomous mobile sensing system for localization of odor
Auton Robot (2006) 20:239–249 source using gas sensors and anemometric sensors. Sensors and Actuators A, 45:153–157. Karlsson, L. 2001.Conditional progressive planning under uncertainty. In: Proc. of the 17th Int. Joint Conferences on Artificial Intelligence (IJCAI). pp. 431–438. Kazadi, S., Goodman, R., Tsikata, D., Green, D., and Lin, H. 2000.An autonomous water vapor plume tracking robot using passive resistive polymer sensors. Autonomous Robots, 9(2):175– 188. Keller, P., Kouzes, R., Kangas, L., and Hashem, S. 1995.Transmission of Olfactory Information for Telemedicine. Interactive Technology and the New Paradigm for Healthcare, pp. 168– 172. Lilienthal, A.J. and Duckett, T. 2003.Creating gas concentration gridmaps with a mobile robot. In Proceedings of the 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003). pp. 118–123. Loutfi, A. and Coradeschi, S. 2004.Forming odour categories using an electronic nose. In Proc. of European Conference in Artificial Intelligence (ECAI 2004). pp. 119–124. Loutfi, A. and Coradeschi, S. 2005.Improving odour analysis through human robot interaction. In Proc of the IEEE International Conference on Robotics and Automation (ICRA 2005). Barcelona, Spain, pp. 4454–4460. Loutfi, A., Coradeschi, S., Karlsson, L., and Broxvall, M., 2005a. Object recognition: a new application for smelling robots. Robotics and Autonomous Systems, 52:272–289. Loutfi, A., Coradeschi, S., and Saffiotti, A. 2005b.Maintaining coherent perceptual information using anchoring. In The Nineteenth International Joint Conference on Artificial Intelligence (IJCAI ’05). Edinburgh, Scotland, pp. 1477–1482. Loutfi, A., Coradeschi, S., Karlsson, L., and Broxvall, M. 2004. Putting olfaction into action: Using an electronic nose on a multi-sensing mobile robot. In In Proc. IEEE International Conference on Intelligent Robots and Systems (IROS). pp. 337– 342. Loutfi, A. and Wide, P. 2002.Symbolic estimation of food odors using fuzzy techniques.In Proc. Information and Processing and Managment of Uncertainty (IPMU). Annecy, France, pp. 919–926. Marques, L., Nunes, U., and de Almeida, A. 2002. Olfaction-based mobile robot navigation. Thin Solid Films, 418:51–58. Natale, C. D., Davide, F., and D’Amico, A. 1995. A self-organizing system for pattern classificiation: time varying statistics and sensor drift effects. Sensors and Actuators-B, 26–27:237–241.
249 O’connell, M., Valdora, G., and Peltzer, G. 2001.A practical approach for fish freshness determinations using a portable electronic nose. Sensors and Actuators-B Chemical, B80(2):149–154. Persaud, K. and Dodd, G. 1982.Analysis of discrimination mechanisms of the mammalian olfactory system using a model nose. Nature, 299:352–355. Rochel, O., Martinez, D., Hugues, E., and Sarry, F. 2002. Stereoolfaction with a sniffing neuromorphic robot using spiking neurons. In 16th European Conference on Solid-State Transducers EUROSENSORS, Prague, Czech Republic. Russell, R., Thiel, D., and Mackay-Sim, A., 1994.Sensing odour trails for mobile robot navigation. In In IEEE Int. Conf. Robotics and Automation (ICRA 1994). pp. 2672–2677. Saffiotti, A., Konolige, K., and Ruspini, E. H. 1995. A multivaluedlogic approach to integrating planning and control. Artificial Intelligence, 76(1–2):481–526. Xie, X. and Beni, G. 1991. A validity measure for fuzzy clustering. IEEE Trans. on Pattern Analysis and Machine Intelligence, 13(8):841– 847. ¨ Amy Loutfi is a research assistant at Orebro University, Sweden. She has received a BSc. in Electrical Engineering from the University of New Brunswick, Canada and a PhD. in ¨ Computer Science from Orebro University. Her main research interest is in the integration of electronic nose devices into intelligent systems. This pursuit has led to an interest in mobile robotics and artificial intelligence. E-mail:
[email protected]. Silvia Coradeschi is an associate professor ¨ at Orebro University, Sweden. She has received a MSc in Philosophy from the University of Florence, a MSc in Computer Science from the University of Pisa and a Ph.D. in Computer Science at Linkoping University. Her main research interest is in establishing the connection (anchoring) between the symbols used to perform abstract reasoning and the physical entities, which these symbols refer to. She also works in co-operative robotics, mobile robotics and artificial olfaction applied to robotics. E-mail:
[email protected].
Springer
