1160
IEEE TRANSACTIONS ON ROBOTICS, VOL. 23, NO. 6, DECEMBER 2007
Multirobot Formations Based on the Queue-Formation Scheme With Limited Communication Cheng-Heng Fua, Student Member, IEEE, Shuzhi Sam Ge, Fellow, IEEE, Khac Duc Do, and Khiang-Wee Lim, Senior Member, IEEE
Abstract—In this paper, we investigate the operation of the queue-formation structure (or Q-structure) in multirobot teams with limited communication. Information flow can be divided into two time scales: 1) the fast-time scale where the robots’ reactive actions are determined based only on local communication and 2) the slow-time scale, where information required is less demanding, can be collected over a longer time, and intermittent information loss can be afforded. Therefore, there is no need for global information at all times, reducing the overall communication load. In addition, a dynamic target determination algorithm, based on the Q-structure, is used to produce a series of targets that incrementally guide each robot into formation. It provides greater control over the distance between robots on the same queue, instead of relying on interrobot repulsive distance, and, allows better formation scaling. An analysis of the convergence of the system of robots and realistic simulation studies are provided. Index Terms—Convergence, limited communication ranges, multirobot formations, Q-structure.
I. INTRODUCTION OBUST multirobot collaboration in dynamic and uncertain environments has been intensively studied in recent years. Several high-level deliberation approaches such as ALLIANCE [1], MURDOCH [2], and, more recently, COBOS [3], have been proposed. Each robot then utilizes lowerlevel behaviors [4]–[6] to accomplish the allocated tasks. For large multirobot teams in dynamic and uncertain environments (e.g., field applications), where communication can be limited and unreliable, decentralized approaches are generally favored. To guide robots into formations, methods such as
R
Manuscript received September 5, 2006; revised June 26, 2007. This paper was recommended for publication by Associate Editor G. Sukhatme and Editor L. Parker upon evaluation of the reviewers’ comments. This paper was presented in part at the IEEE International Conference on Robotics and Automation, Rome, Italy, May 2007. C.-H. Fua is with the NUS Graduate School for Integrative Sciences and Engineering (NGS), National University of Singapore, Singapore 117576, Singapore, and also with the Institute of Infocomms Research (I2R), Singapore 119613, Singapore (e-mail:
[email protected]). S. S. Ge is with the Social Robotics Laboratory, Interactive Digital Media Institute, National University of Singapore, Singapore 119077, Singapore, and also with the Edutainment Robotics Laboratory, Department of Electrical and Computer Engineering, National University of Singapore, Singapore 117576, Singapore (corresponding author) (e-mail:
[email protected]). K. D. Do is with the School of Mechanical Engineering, University of Western Australia, Crawley, WA 6009, Australia (e-mail:
[email protected]). K.-W. Lim is with the Institute of Materials Research and Engineering, Singapore 117602, Singapore, and also with Singapore’s Science and Engineering Research Council, Agency for Science, Technology, and Research (A*STAR), Singapore 138668, Singapore (e-mail:
[email protected]. Digital Object Identifier 10.1109/TRO.2007.911198
virtual leaders [7], social potentials [8], formation-constrained functions [9] and virtual structures [14] may be used. Individual robots may also be allocated specific identification tags, each associated with a predefined position in the formation [10]–[12]. Agent cohesion and collective behavior, when delays are present in sensing and communication networks, are examined in [13]. General methods for formation controller design have also been proposed in [14] and [15]. A number of studies have also been carried out on the stability and convergence of formation schemes (largely based on the nodes-and-edges representation). In the study presented in [16], specific artificial potential fields for each robot relative to the other robots are calculated to produce stable formations in equilibrium. The concept of leader-toformation stability is introduced in [17], and the paper examined how errors propagate from the leader and influence the stability of formations (those based on leader following). Navigation functions, first proposed in [18], have also been used for centralized control of formations [19]. The problem of ensuring formation stability in a robot team while it moves in formation through an obstacle field between two points is investigated in [20]. Most of the aforementioned seminal works use a formation representation based on connectivity graphs where each robot tracks a specific node as a target (e.g., in virtual structure approaches [21]). When team size changes, the graph representation will change, which can become difficult to track dynamically. Since a large class of problems (e.g., robotic search and surveillance, convoy movements) only require robots to maintain the overall appearance of the formation, the use of queues and artificial potential trenches have been introduced [24]. The Q-structure is independent of reshuffling of robot teams thus facilitating scalability and flexibility. Two drawbacks of the scheme in [22] are the reliance on persistent global communication and that the convergence of robots into formations based on the scheme had only been established via experimentation. The main contributions of this paper are as follows: 1) The Q-structure in [22] is extended by considering finite communication ranges, and separating the decisionmaking process into two time scales—a fast-time scale for reactive decision making based only on local communication, and a slower time scale, which allows less time-critical information to propagate through a weakly connected network. Persistent global communication is not required, reducing overall communication load. It also permits intermittent information losses as information is collected over a longer time.
1552-3098/$25.00 © 2007 IEEE
FUA et al.: MULTIROBOT FORMATIONS BASED ON THE QUEUE-FORMATION SCHEME WITH LIMITED COMMUNICATION
2) A rigorous proof of convergence for a decentralized control law that guides robots into formations, represented by the Q-structure, is presented. The design and proof for navigation controls in [23] are also extended as follows. a) A different set of interagent potentials are designed to conform with a smaller set of properties that reflect limited communication ranges and b) A different, simpler analysis for the controller is presented. 3) A dynamic target determination algorithm is proposed to incrementally guide robots into their queues. This improves upon the original scheme [22] by separating agent decisions regarding positions on queues from reactive interagent repulsive forces, to obtain an even distribution of agents along queues. II. FORMATION REPRESENTATION AND DYNAMIC TARGET DETERMINATION In this paper, a formation FN is a set of constraints on the positions of each agent in relation to others in the team, such that each formation has a specific appearance. In conventional graphical notation, the constraints come in the form of nodes and edges in a connectivity graph (for instance in [24] and [25]), where nodes represent the location of each agent and the edges represent the presence of communication links between agents. Consider an undirected graph GF = (Vg , Eg ) ∈ GN , where GN is the space containing all possible graphs that can be formed from the set of N vertices, given by Vg . Let the set of locations of all the N agents at any time t be represented by LT (t) = {q1 (t), q2 (t), . . . , qN (t)} ∈ CN
(1)
where CN is the configuration space of the team as a whole. Define a function Φc : CN → GN to give Gc (t) = Φc (LT (t)) = ({q1 (t), . . . , qN (t)}, {(qi , qj )|i = j
and
qi − qj ≤ dij })
(2) (3)
with dij being the cutoff distance beyond which there will not be communication between robots ri and rj . In a graphical representation, with converging formation controls/protocols Gc (t) → FN = Gc,d , as t → ∞
(4)
where Gc,d is the connectivity graph of the desired formation. A general collection of agents in different (nonoverlapping) positions does not constitute a formation.1 It is also assumed that Gc,d is weakly connected, although Gc (t) can be disconnected. It has also been shown (e.g., in [26] and [27]) that stability and convergence can be achieved in systems that sometimes become disconnected, if there is a common objective. In this paper, the common objective is the desired formation FN that can either be stationary or moving along a desired path. A. Division of Information Flow Information flow is separated into slow and fast time scales. The control of the formation takes place on these two levels 1 This
distinguishes formations from flocking/swarming.
1161
based on the information available on each time scale. This reduces the amount of information that must be available to each robot for reactive decision making. 1) Fast-time scale: This facilitates time-critical and reactive decision making, such as interrobot collision avoidance and getting into formation. It only involves local communication between robots with limited communication range. Explicit controls governing the actual movements and paths of the agents occur at this level. Such decisions take place at a higher frequency when information is available. 2) Slow-time scale: This refers to the gradual multihop transfer of information, through a weakly connected communication network, between robots that are not within the immediate vicinity of each other. The collection of information over a longer time period allows for intermittent information losses between links. Formation control on this level involves low-frequency decisions regarding the (re)allocation of robots to different parts (either vertices or queues) of a formation. In addition, while information regarding out-of-range agents may be available, these are not taken into consideration while making path decisions other than for (re)allocation. B. Formations and Queues In this paper, the relative position constraints on the agents that characterize a formation is represented in the form of queues and queue vertices. Before proceeding further, for completeness, we briefly state the concepts regarding “queues” and formations, as detailed in [24]. The concept of queues relies on the observation that many formations can be subdivided into a series of smooth line segments, each of which is referred to as a queue. Definition 1 (Formations [24]): A formation is denoted by F = (Q, VF (N )), where Q is the set of queues, and VF (N ) represents the set of formation vertices, Vi (i = 1, . . . , Nv ), where N is the total number of robots.2 Definition 2 (Queues [24]): A queue, Qj ∈ Q, is denoted as Qj = (Vj , Sj , Cj , Ej (N )), with: 1) Vj ⊆ VF (N ) (Queue vertices): A list of either one or two formation vertices through which Qj passes. 2) Sj (Shape): A set of points following an equation in R3 that describes the spatial appearance of Qj , and is specified in the coordinate frame of the first formation vertex in the list Vj . 3) Cj (Capacity): A fraction that refers to the proportion of all N q the robots in the formation it can hold, i.e., j =1 Cj = 1, where Nq is the total number of queues in the formation. 4) Ej (Encapsulating region)3 : The set of all the points within a certain distance, dec , of the queue. The region is dependent on the number of robots that should reside on the queue, and is, hence, related to N . Fig. 1(a) shows a formation consisting of three queues. The formation vertices are represented by circles, labeled V1 –V3 . 2 Each formation vertex is represented by its position relative to the coordinate frame of the target. 3 E is a small region around each queue, and when the desired formation is j reached, all robots should be within the encapsulating region of their queues.
1162
IEEE TRANSACTIONS ON ROBOTICS, VOL. 23, NO. 6, DECEMBER 2007
Fig. 1. Defining the appearance of formations using queues, and vertices (circles). (a) A triangular formation. (b) A three column formation.
Fig. 2. Graphical Representation of Q-structures. Dotted circles represent virtual vertices. (a) Triangular formation. (b) Three column formation.
Formation vertices represent a minimal number of pertinent locations in a formation, and do not yield complete information about what the formation looks like. As such, two different formations may have the same VF (N ) (e.g., in Fig. 1). This may be seen in Fig. 1, which shows two different formations with the same set of formation vertices. Fig. 3. Triangular formation represented using connectivity graphs. (a) Teams with 3 agents. (b) Teams with 6 agents.
C. Properties of the Q-Structure The main difference between the proposed approach and other approaches (such as [11] and [26]) is the use of the Q-structure. 1) Graphical Representation of the Q-Structure: To map the Q-structure into a conventional graph representation, a set of virtual queue vertices, Vv are added to the set VF (N ), to produce Vα (N ) = VF (N ) Vv . (5) The virtual Q-vertices are added to impose a limit on the length of queues with only one queue vertex, which would, otherwise, stretch to infinity. Therefore, each set of queue vertices contains a pair of formation vertices Vj = {Vi , Vj }, where i = j and Vi , Vj ∈ Vα .
(6)
define a function Φα : F → GN , such that Φα (FN ) = GF . Specifically, we have Φα (FN ) = (Vα (N ), {Vj |j = 1, . . . , Nq }).
(7)
Each queue is represented in GF by its two queue vertices, which form an edge of the undirected graph. The graphical representations of the formations in Fig. 1 are shown in Fig. 2. It should be mentioned that these are not connectivity graphs showing sensing/communication links. 2) Consistency in Formation Representation: The Qstructure results in consistent formation representations, independent of the agent team size. It does not ascribe specific positions for individual agents, and relies on a set of decentralized, self-organizing behaviors to determine the final position of each agent. Typical graphical representations of formations rely on exact placement of each agent to achieve the final appearance of the formation. Based on graphs defined in [24], the addition or removal of agents in formation maintenance schemes in [11] and [21] will result in different formations and connectivity graphs. Since the appearance of the formation is the important factor in many applications (such as vehicle convoys
or target encirclement), the Q-structure allows formation specification based on appearance—the reverse of what graph-based approaches adopt. The consistent representation dispenses with the additional computation required for the addition/removal of nodes and the calculation of new interagent relationships. Fig. 3 shows the two different connectivity graph representations of a triangular formation with three and five agents, respectively, while the Q-structure for the same formation [Fig. 1(a)] remains unchanged regardless of team size. 3) Formation Decomposition and Computation: The Qstructure allows a formation to be divided into smaller and simpler formations. Each queue, and its vertices, is a formation, i.e., F = {Fk | k = 1, 2, . . . , Nq }
(8)
where Fk = (Qk , Vk ). After the initial allocation phase of agents to queues, the short-term information required by an agent can be limited to those within the same queue and others in the immediate vicinity. This reduces the communication load in the system especially for more complex formations. In this paper, we measure the computational complexity in terms of the frequency a robot performs a resource-expensive computation (e.g., the “comparison” operation), and, thus, how the complexity order scales with N . The simplest method for (re)allocating agents either to vertices for graph representations, or to queues, is via greedy allocation. Assuming that greedy assignment is made based on shortest distance, and that the graph representation (such as those in [28] and [29] contains the same number of vertices as the number of robots, each robot compares its distance to N vertices. Therefore, with N robots, the computational complexity is O(N 2 ). For queues, each robot makes 2Nq comparisons, comparing its distance to the queue and considering also the current capacity of that queue. This results in complexity of O(N Nq ), where
FUA et al.: MULTIROBOT FORMATIONS BASED ON THE QUEUE-FORMATION SCHEME WITH LIMITED COMMUNICATION
Nq ≤ N . Therefore, the Q-structure would potentially result in a lower computational cost, by implicitly decomposing a formation and lumping groups of vertices together. D. Determination of Target on Queue This section describes an algorithm that each robot ri , associated with a queue Q(i), uses for target determination. The algorithm also governs the distance between robots within the same queue. Compared to the purely reactive scheme in [22], it improves the scaling of formations through an adaptation of the parameter dir (acceptable interrobot distance for robots on the same queue).
1163
the preceding paragraph, the target of the second agent in RQ ∗ , qtg ,q 2 will be constant because qtg ,q 1 is fixed. Therefore, by induction, the target of the nth agent will be fixed and constant, once the agents have converged into a weakly connected net around their respective queues. III. NAVIGATION OF ROBOTS TO POSITIONS IN FORMATIONS Direct communication between agents in each others’ neighborhoods allows target and position information to be transmitted between these agents. At any time instant, each agent will have its targets determined by its position relative to its related queues, as described in Algorithm 1. Upon termination of Algorithm 1, from Lemma 2.1, the targets of each agent will become constant within finite time, and the control laws presented in this section will cause each agent to converge to their desired targets. Consider the following potential function: U = Utg + Uob
(9)
that consists of two parts: 1) Utg describes the attractive potentials between the robots and their targets, and may be written as 1 = qi − qtg ,i 2 2 i=1 N
The algorithm works by considering only the agents within the communication range of the robot ri and which also belong to the same queue as the robot. This group of robots is represented by Rc,i ∈ RN , which is an ordered set based on increasing Euclidean distance from the first vertex of the queue (i.e., VQ(i) (1), and where RN is the set of all robots in the team.The target of ri is set to be a point on Q(i) and at a distance of dir away from the target of rj . If ri is the agent in Rc,i that is closest to the queue vertex VQ(i) (1), its target will be set to be the queue vertex. The common objective (FN , as mentioned in Section II) will result in a weakly connected communication network for each subset of agents within the same queue. Although an agent may not be in direct communication with some others within the same queue, the decisions of preceding agents will be reflected/propagated via the decisions made by others within the communication range. Lemma 2.1: Given a set of agents and considering only direct communication between an agent and those in its neighborhood, Algorithm 1, together with the common objective given in the form of the desired formation FN , will result in constant targets for each agent on each queue. Proof: Let ri and rj be the nth and (n − 1)th furthest agents in Rc,i from the queue vertex VQ(i) (1). According to Algorithm 1, if qtg ,j is constant, qtg ,i will be constant too, and at a distance of dir along the queue from qtg ,j . Consider a queue Q∗ where all agents belonging to this queue have converged into a weakly connected net due to the common objective. Let RQ ∗ = {rq 1 , rq 2 , . . . , rq N q } be this set of Nq agents, ordered in ascending order according to their distance from the queue vertex VQ∗ (1). For the set Rc,q 1 , rq 1 will be the closest to the vertex, and from Algorithm 1, its target will be constant and locked to qtg ,q 1 = VQ∗ (1). From the argument in
Utg
(10)
since it is initially assumed that each robot’s communication range is large enough to cover the team. 2) Uob reflects the collision avoidance behavior of the robots with their neighbors. In real-life scenarios, the communication range of a robot is often limited to a set of robots near it. This can be due to power constraints and the presence of obstacles and noise. Communication signals that could be received from robots outside this range would be heavily attenuated. Each robot treats the other robots within its communication neighborhood as obstacles and constructs an instantaneous path according to a control law ui . The function is given by: Uob =
N −1
N
Uob,ij
(11)
i=1 j =i+1
where Uob,ij is a function of Uij and Utg,ij , which are given by 1 (12) Uij = qi − qj 2 2 1 Utg,ij = qtg,i − qtg,j 2 (13) 2 and Uob,ij is chosen such that it exhibits the following properties: a) Uob,ij = ∞, if Uij = 0; b) Uob,ij > 0, if Uij = 0; ∂U = ∂ Uo bi ,ji j = 0, if Uij = Utg,ij ; c) Uob,ij = d) Uob,ij
∂ 2 Uo b , i j ∂ U i2j
≥ 0, if Uij = Utg,ij ;
e) Uob,ij ≈ 0, if Uij ≥ 0.5d2ij .
1164
IEEE TRANSACTIONS ON ROBOTICS, VOL. 23, NO. 6, DECEMBER 2007
Based on the aforementioned properties, Uob,ij is chosen as Uij 1 Uob,ij = fij (14) + 2 Utg,ij Uij where
1 1 + exp(at (Uij − Utg,ij )3 )
fij =
(15)
where at is a user-defined constant. At each time instant, each robot moves along the negative gradient of the potential function U . In general, the time derivative of the overall potential function U in (9) is given by U˙ =
N
where 3 is a strictly positive number representing the minimum acceptable interrobot distance. Theorem 3.1: Under the conditions stated in (21), the common formation objective given by FN , and Algorithm 1, the control input to each robot, given in (18), will result in the convergence of each robot to its desired target, and such that: 1) The target at qtg is located at an asymptotically stable equilibrium point of (20); and 2) The critical points of the system other than that at qtg are unstable equilibrium points. Proof: Integrating both sides of (19) from t0 to t, we obtain
(qi − qtg,i )T ui
Utg (t) +
N −1
N
Uob,ij (qi − qj )T (ui − uj )
i=1 j =i+1
=
N
(qi − qtg,i )T +
N
N
≤ Utg (t0 ) +
Uob,ij (t)
N −1
N
Uob,ij (t0 )
(23)
i=1 j =i+1
where
T ui Uob,ij qij
1 qi (t) − qtg,i 2 2 i=1 Uij (t) 1 . Uob,ij (t) =fij (t) + 2 Utg,ij Uij (t) N
j = i
i=1
=
N
i=1 j =i+1
i=1
+
N −1
Utg (t) =
ΩTi ui
(16)
i=1
where qij = qi − qj and Ωi is defined as Ωi = (qi − qtg,i ) +
N
Uob,ij qij .
(17)
j = i
This implies that a choice of ui = −CΩi
(18)
Rn+w ×n w
where C ∈ is a symmetric, positive-definite matrix, which is chosen as C = In w ×n w c where c > 0, will result in U˙ = −
N
ΩTi CΩi
(18)
i=1
and the closed-loop dynamics of a single robot ri in the team is, then, given by q˙i = −CΩi .
(20)
If the robots are at different positions (i.e., noncolliding) at an initial time t0 , and the target of each robot is different as well, these conditions may be written as qi (t0 ) − qj (t0 ) ≥ 1
(21)
where 1 is a strictly positive constant, and R is the set of robots comprising the team. In addition, Algorithm 1 guarantees that if the condition in (21) is satisfied, the targets for each cycle do not collide, i.e., qtg ,i − qtg ,j ≥ 2 ∀i, j ∈ R, where 2 is strictly positive. It is, thus, desired that, under such conditions, each robot will converge toward its target, and, at the same time, avoid collisions, i.e., lim (qi (t) − qtg ,i ) =0
t→∞
qi (t) − qj (t) ≥3 ∀i, j ∈ R and ∀t ≥ t0 ≥ 0 (22)
(24)
From the conditions in (21), Uij (t0 ) and Utg,ij are strictly larger than some positive constants. Furthermore, since fij is also bounded (0 < fij < 1), the right-hand side of (23) is bounded by some positive constant (the value of which depends on the initial conditions at t0 ). Hence, the left-hand side is also bounded, which, in turn, implies that Uij (t) must be strictly larger than some positive constant for all t ≥ t0 ≥ 0. From (24), qi (t) − qj (t) will, therefore, always be larger than some strictly positive constant, and there will be no collisions. The boundedness of the left-hand side of (23) also implies that of qi (t) for all t ≥ t0 ≥ 0, and the solutions of the closed-loop system in (20) exist. Let the root sets (critical points) of the system in (20) be represented by qe , and consist of points at q = qtg (due to Property (c) of Uob,ij ) and q = qc (representing the remaining critical points), where the overall system for the N agents is T T ] , Ω = [ΩT1 , . . . , ΩTN ]T , qtg = q˙ = −cΩ, with q = [q1T , . . . , qN T T T T T ]T . The equilibrium [qtg,1 , . . . , qtg,N ] , and qc = [qc,1 , . . . , qc,N points are not separated into stable and unstable points at the outset before the following analysis, but rather, the properties of the points which are desired to be stable are examined versus the rest of the equilibrium points. By construct, qtg is an equilibrium point, and the main objective is for this to be stable and for the remaining critical points qc , wherever they may be, to be unstable. Furthermore, the targets for each agent in the system are determined by Algorithm 1 and are constant for the time period under consideration, and the system, by inspection, is linear-time-invariant (LTI). For the remainder of the proof, the property at q¯tg is examined first, followed by the properties of q¯c . Linearizing the closed loop system about the equilibrium
FUA et al.: MULTIROBOT FORMATIONS BASED ON THE QUEUE-FORMATION SCHEME WITH LIMITED COMMUNICATION
point qe gives
∂Ω
(q − qe ) q˙ = −c ∂q q =q e
where the general gradient of Ω with respect to q is ∂Ω ∂Ω1 ∂Ω1 1 ... ... ∂q2 ∂qN ∂q1 . .. .. .. .. . . . . . . ∂Ωi ∂Ωi ∂Ωi ∂Ω ... ... = ∂qi ∂qN ∂q ∂q1 . .. .. .. .. .. . . . . ∂Ω ∂ΩN N ... ... ... ∂q1 ∂qN with
⇒
N N ∂Ωi T I + = 1+ Uob,ij Uob,ij qij qij ∂qi j = i
(26)
(27)
(28)
(29)
∂Ωi T = −Uob,ij qtg,ij qtg,ij ∂qj
(30)
N
j = i
Considering the Lyapunov candidate
and using (29) and (30), we obtain
∂Ω
V˙ q t g = − c(q − qtg )T (q − qtg ) ∂q q =q t g =−c
N N
(qi − qtg,i )T
i=1 j =1
=−c
N
∂Ωi (qj − qtg,j ) ∂qi
qi − qtg,i 2
1 + N Uob,ij q
ij
=q c , i j
=q c , i j
=0
T qc,ij qc,ij
=
N −1
N
T qc,ij qtg,ij
(38)
i=1 j =i+1
∂Ωi T =I+ Uob,ij qtg,ij qtg,ij ∂qi
1 q − qtg 2 2
N
N −1
ij
T qc,ij qc,ij
i=1 j =i+1
At the equilibrium points at qe = qtg , based on the properties of Uob,ij , and letting qtg,ij = qtg,i − qtg,j , (27) and (28) become
Vq t g =
+ N Uob,ij q ⇒
j = i
∂Ωi T = − Uob,ij I − Uob,ij qij qij . ∂qj
T (qc,ij − qtg,ij ) qc,ij
i=1 j =i+1
(25)
N
N −1
1165
where qc,ij = qc,i − qc,j , Ωij = Ωi − Ωj and T T T T T , q13 , . . . , qij , . . . , qN (39) q¯ = q12 −1N T T T T T q¯tg = qtg,12 , qtg,13 , . . . , qtg,ij , . . . , qtg,N (40) −1N T T T T T , . . . , qc,ij , . . . , qc,(N q¯c = qc,12 , qc,13 −1)(N ) ] (41) T F (¯ q , q¯tg ) = [ΩT12 , ΩT13 , . . . , ΩTij , . . . , ΩTN −1N . (42) T Consider the term qc,ij qtg,ij and the agents i and j. The agent j can be seen as an obstacle situated at qij = 0. Similarly, agent i is an obstacle with respect to j at qj i = 0. At qij = qc,ij , both agents are at their critical points. For this to hold, both critical points must lie along a straight line along the vector qtg,ij and between qtg,i and qtg,j . That is, the point qij = 0 must lie between such that these three the points qij = qtg,ij and qij = qc,ij , and −1 N T points are colinear. Thus, the term N i=1 j =i+1 qc,ij qtg,ij is strictly negative and there exists at least one pair (i, j) denoted by (i∗ , j ∗ ) ∈ R∗ such that
≤ −b (43) 1 + N Uob,i ∗j ∗ q ∗ ∗ =q ∗ ∗ i j
(31)
(32)
(33)
(34)
c,i j
where b is a strictly positive constant. For the system under consideration, the interagent repulsive forces are dependent on the relative distances between individual agents. For an agent i, the other agents can be treated as obstacles, and the equilibrium points are a direct result of the relative positions. Therefore, instead of considering the function Vc = q − qc 2 , the behavior of the equilibrium points in the system are examined first by considering the Lyapunov function based on the relative distances (i.e., q¯ and q¯c ), and the result is then linked to stability of the points qc in the last part of the proof. Consider the Lyapunov function candidate
i=1
−c
N N
Uob,ij
2 T qtg,ij (qij − qtg,ij ) . (35)
i=1 j =1,j = i Since Uob,ij ≥ 0 at q = qtg
V˙ q t g ≤ −2cVq t g
(36)
(37)
(44)
whose derivative along the solution of (44) gives V˙ q¯c = − 2c
indicating the equilibrium points at qtg are asymptotically stable. To show that the remaining critical points of the system, i.e., qc are unstable equilibrium points, consider the following: q¯cT F (¯ qc , q¯tg ) = 0
q − q¯c 2 Vq¯c = ¯
N −1
N
(qij − qc,ij )T
i=1 j =i+1
In w ×n w + N In w ×n w Uob,ij q
ij
=q c , i j
T +N U ”ob,ij |q i j =q c , i j qc,ij qc,ij (qij − qc,ij )
≥ 2cb(qi ∗ j ∗ − qc,i ∗ j ∗ )T (qi ∗ j ∗ − qc,i ∗ j ∗ )
1166
IEEE TRANSACTIONS ON ROBOTICS, VOL. 23, NO. 6, DECEMBER 2007
N −1
− 2c
(qij − qc,ij )T
i=1,i= i ∗ j =i+1,j = j ∗
In w ×n w + N In w ×n w Uob,ij q
− 2c
N
N −1
N
ij
ij
=q c , i j
(qij − qc,ij )
(qij − qc,ij )T
i=1 j =i+1
N Uob,ij q
=q c , i j
T qc,ij qc,ij (qij − qc,ij ).
(45)
Considering a subspace such that qij = qc,ij ∀(i, j) ∈ T (qij − {1, . . . , N }, (i, j) = (i∗ , j ∗ ) and (qij − qc,ij )T qc,ij qc,ij qc,ij ) = 0 ∀(i, j) ∈ {1, . . . , N }. In this subspace, the following holds qij − qc,ij 2 (46) Vq¯c = (i,j )∈R ∗
V˙ q¯c ≥2bcVq¯c which implies that qij (t) − qc,ij ≥ (i,j )∈R ∗
(47)
qij (t0 ) − qc,ij ebc(t−t 0 )
(i,j )∈R ∗
(48) where t ≥ t0 ≥ 0. Assuming that qc is a stable equilibrium with limt→∞ qi (t) − qc,i = ai where ai is a positive constant. This further implies that limt→∞ (i,j )∈R ∗ qij (t) − qc,ij = a∗ ∀(i, i) ∈ R∗ and a∗ being a positive constant, which contradicts the result obtained in (48), and qc is an unstable equilibrium point of the closed-loop system. Remark 1: In the aforementioned proof of qc being unstable equilibrium points, an exception occurs when all the agents start at positions that coincide exactly with their critical points at qc (i.e., (i,j )∈R ∗ qij (t0 ) − qc,ij = 0). In this case, qc will be marginally stable (similar to a linear system, where the real part of one or more eigenvalues equals zero). However, for practical systems that are considered here, noise and other disturbances will cause (i,j )∈R ∗ qij (t∗0 ) − qc,ij = 0 for some finite t∗0 > t0 . Therefore, the instability of qc can be analyzed in the same way as earlier with t0 replaced by t∗0 . For practical implementation, a robot ri may only be able to compute an approximate value of Ωi in (17) since it may not receive any information at all from robots outside the communication radius di . The approximate of Ω is given by ˆ i = (qi − qtg ,i ) + Uob,ij qij (49) Ω j = i,j ∈R c i
and the control law becomes ˆ u ˆ˙ = −C Ω.
(50)
The approximation error for each robot may, thus, be written as ˆ eΩ =Ω − Ω = j = i,j ∈R n i
Uob,ij qij
(51)
where Rn i = R \ Rci is the set of robots that ri cannot communicate with, and “\” denotes the set subtraction operation. From property e) of Uob,ij , we know that for j ∈ Rn i , Uob,ij , Uob,ij ≈ 0. In addition, assuming that qij (0) is bounded, since the robots converge to their targets on the queues and qtg ,ij is also bounded, the value of eΩ is bounded by some small positive real value, and the error that arises due to incomplete information from robots out of communication range can be kept small through the use of fij to weight the importance of repulsive forces between agents. Therefore, for the control law, described in (17) and (18), for an agent i, the use of fij heavily attenuates the contribution of any agent j that is out of range to approximate the control when global communication is present, to facilitate practical implementation with limited communication ranges and scaling. IV. SIMULATION STUDIES Realistic simulations were conducted using Player and Stage [30]. The system consists of five circular omnidirectional robots, each of diameter 0.3 m. Each of these robots act based on commands to their speed, which, in this case, is determined ˆ i in (49). by the control input ui in (50) with the estimated Ω The parameters at , dir , and C are chosen to be 10, 2, and the identity matrix, respectively. It is assumed that each robot is able to localize itself in the global frame. Furthermore, each robot is equipped with a laser scanner (180◦ ) and 16 sonar range sensors arranged in a ring around the circular robots for obstacle avoidance. The sensor noise introduced into the range sensing has a normal distribution of 0.2 variance. The communication range of the robots is set to 3 m, and there will be connectivity between each robot on each queue as they converge into position as long as dir is set to be less than the communication range. A. Formation Convergence and Scaling The first part of the simulations consists of examining the convergence of the robots to a given wedge formation, and how it scales when two robots are removed (deactivated) at t = 10 s. In the final formation, robots are to be a minimum of 2 m from others. The robots are initialized at random (noncolliding) positions in a 20 m × 20 m square around the point (10 m,10 m) in the workspace. Fig. 4 shows how the distance of the robots from their targets varies over time. The targets evolve according to Algorithm 1. From the graphs, we can see that the robots are able to converge to the formation in a relatively short time of 6–8 s, and in approximately 3 s after scaling. Fig. 5 shows the minimum center-to-center distance that exists between any two robots in the team at each time. It can be seen that the minimum distance between any two robots is always greater than 0.5 m at all times, and, hence, no collisions occur. It should be noted that Fig. 4 shows the distance of each robot from their target at each time instant. The spikes in the graphs are the result of changes in the targets for each robot (according to Algorithm 1) as they interact with others within communication range during the transient period. It can also be noted that these spikes, however, cease to appear when the robots get within communication range of each
FUA et al.: MULTIROBOT FORMATIONS BASED ON THE QUEUE-FORMATION SCHEME WITH LIMITED COMMUNICATION
Fig. 4. Robot convergence to formation with robot deactivation/removal at t = 10s.
Fig. 6.
1167
Distance of robots from related queue’s encapsulating area.
Fig. 7. Formation convergence with a moving target. (a) Moving Target, the queues (b) Robot Convergence to Formation and virtual vertex.
Fig. 5. Robot separation and control forces. (a) Minimum Inter-Robot Separation.
other and their targets reach a constant state. This is further evidenced by the absence of spikes when scaling occurs at t = 10 s, and the robots converge to their new targets. This observation applies also to the subsequent sections. For comparison, the convergence of robots using the purely reactive technique presented in [22], with limited communication ranges between robots and for a wedge formation, is shown in Fig. 6. It can be seen that convergence is adversely affected by limited communication ranges, since the robots frequently reallocate themselves to different queues depending on the robots within their own neighborhoods, which causes constant shuttling between queues. This is an effect that has been removed by the current proposed scheme.
simulations are run on the same team of robots with a separate sixth robot acting as a moving target. The formation vertex of the wedge formation is set to be at a distance of 2 m along the negative x-axis of the target robot, as shown in Fig. 7(a). The moving target is programmed to start moving at time t ≈ 3 s from its initial point at (11.4 m, 11.4 m) (such that the initial formation vertex is approximately at (10 m, 10 m)) at a constant velocity of [0.2 0.2]T m/s. The robots’ task is to form a straight line formation (2 m apart from each other) behind the target when it is not moving and to follow it in a wedge formation when it starts to move. The convergence of the robots to the formation is shown in Fig. 7(b). It can be observed that when the target begins to move, the line formation that the robots are originally in is disrupted as they attempt to form a wedge. Convergence into the wedge formation is subsequently observed after approximately 4 s. The minimal interrobot distance is shown in Fig. 8.
B. Moving Formations
C. Changing Formations
In order to verify the effectiveness of the proposed method in enforcing formation maintenance relative to a moving target,
To further investigate the proposed method when formation changes, we conduct similar experiments for the case when
1168
Fig. 8.
IEEE TRANSACTIONS ON ROBOTICS, VOL. 23, NO. 6, DECEMBER 2007
Robot separation and control forces. Minimum Inter-Robot Separation.
Fig. 10. Robot separation and control forces. Minimum Inter-Robot Separation.
V. CONCLUSION In this paper, the original scheme using the Q-structure has been extended to improve the performance of the scheme when only local communication is present, and results in a weakly connected network. Each robot is restricted to communication with other robots that are within a certain radius of its current position. This is a more realistic environment compared to approaches that require persistent global communication, which is seldom achievable in real-world applications. In addition, a dynamic target assignment strategy has been proposed, based on Q-structures, that aims to guide robots into appropriate positions in the required formations. Furthermore, we examine the convergence properties of the proposed approach, and further verify the effectiveness of our approach with realistic simulations.
Fig. 9. Robot convergence to formation with formation switching. Wedge: t = [0 s, 15 s), column: t = [15 s, 30 s), line: t = [30 s, 45 s).
the formation changes at predefined times from a wedge, to a column (perpendicular to the orientation of the target), and finally to a line (parallel to the target’s orientation). The results are shown in Figs. 9 and 10. We can observe spikes in the graphs at the times when formation changes are initiated, occurring due to the abrupt change in targets. Furthermore, comparing the second and third clusters of spikes, it can be seen that, as expected, the transition from a column to a line is more disruptive compared to the transition from a wedge to a column, due to the further distances to the new targets. On the whole, the team requires an average of 4–6 s to transition between formations and settle stably into the new formation.
ACKNOWLEDGMENT The authors would like to thank the Editor L. Parker, the Associate Editor G. Sukhatme, and the anonymous reviewers for their valuable comments and suggestions for improving earlier versions of this paper. REFERENCES [1] L. E. Parker, “ALLIANCE: An architecture for fault tolerant multirobot cooperation,” IEEE Trans. Robot. Autom., vol. 14, no. 2, pp. 220–240, Apr. 1998. [2] B. P. Gerkey and M. J. Matari´c, “Sold!: Auction methods for multirobot coordination,” IEEE Trans. Robot. Autom., vol. 18, no. 5, pp. 758–768, Oct. 2002. [3] C. Fua and S. S. Ge, “COBOS: Cooperative back-off adaptive scheme for multi-robot task allocation,” IEEE Trans. Robot., vol. 21, no. 6, pp. 1168– 1178, Dec. 2005. [4] S. S. Ge and Y. J. Cui, “Dynamic motion planning for mobile robots using potential field method,” Auton. Robots, vol. 13, pp. 207–222, 2002. [5] S. S. Ge and Y. J. Cui, “New potential functions for mobile robot path planning,” IEEE Trans. Robot. Autom., vol. 16, no. 5, pp. 615–620, Oct. 2000.
FUA et al.: MULTIROBOT FORMATIONS BASED ON THE QUEUE-FORMATION SCHEME WITH LIMITED COMMUNICATION
[6] S. S. Ge, X. Lai, and A. A. Mamun, “Boundary following and globally convergent path planning using instant goals,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 35, no. 2, pp. 240–254, Apr. 2005. [7] N. E. Leonard and E. Fiorelli, “Virtual leaders, artificial potentials and coordinated control of groups,” in Proc. IEEE Conf. Decis. Control, Dec. 2001, vol. 3, pp. 2968–2973. [8] T. Balch and M. Hybinette, “Social potentials for scalable multi robot formations,” in Proc. IEEE Int. Conf. Robot. Autom., Apr. 2000, pp. 73– 80. [9] M. Egerstedt and X. Hu, “Formation constrained multi−agent control,” IEEE Trans. Robot. Autom., vol. 17, no. 6, pp. 947–951, Dec. 2001. [10] T. Balch and R. C. Arkin, “Behavior-based formation control for multirobot teams,” IEEE Trans. Robot. Autom., vol. 14, no. 6, pp. 926–939, Dec. 1998. [11] A. K. Das, R. Fierro, V. Kumar, J. P. Ostrowski, J. Spletzer, and C. J. Taylor, “A vision based formation control framework,” IEEE Trans. Robot. Autom., vol. 18, no. 5, pp. 813–825, Oct. 2002. [12] J. P. Desai, J. P. Ostrowski, and V. Kumar, “Modeling and control of formations of nonholonomic mobile robots,” IEEE Trans. Robot. Autom., vol. 17, no. 6, pp. 905–908, Dec. 2001. [13] Y. Liu, K. M. Passino, and M. Polycarpou, “Stability analysis of onedimensional asynchronous swarms,” IEEE Trans. Autom. Control, vol. 48, no. 10, pp. 1848–1854, Oct. 2003. [14] W. Kang and N. Xi, “Control and adaptation of multi vehicle formations,” in Proc. Int. Conf. Intell. Robots Syst., 1999, pp. 1155–1160. [15] W. Kang, N. Xi, Y. Zhao, J. Tan, and Y. Wang, “Formation control of multiple autonomous vehicles: Theory and experimentation,” in Proc. IFAC 15th Triennial World Congr., 2002, pp. 1155–1160. [16] P. Song and V. Kumar, “A potential field based approach to multi robot manipulation,” in Proc. IEEE Int. Conf. Robot. Autom., May 2002, pp. 1217– 1222. [17] H. G. Tanner, G. J. Pappas, and V. Kumar, “Leader-to-formation stability,” IEEE Trans. Robot. Autom., vol. 20, no. 3, pp. 443–455, Jun. 2004. [18] E. Rimon and D. E. Koditschek, “Exact robot navigation using artificial potential functions,” IEEE Trans. Robot. Autom., vol. 8, no. 5, pp. 501– 518, Oct. 1992. [19] H. Tanner and A. Kumar, “Towards decentralization of multi-robot navigation functions,” in Proc. IEEE Int. Conf. Robot. Autom., Apr. 2005, pp. 4143–4148. ¨ [20] P. Ogren and N. E. Leonard, “Obstacle avoidance in formation,” in Proc. IEEE Int. Conf. Robot. Autom., Sep. 2003, pp. 2492–2497. [21] W. Ren and R. W. Beard, “A decentralized scheme for spacecraft formation flying via the virtual structure approach,” AIAA J. Guid. Control Dyn., vol. 27, no. 1, pp. 73–82, Jan. 2004. [22] S. S. Ge and C. Fua, “Queues and artificial potential trenches for multirobot formations,” IEEE Trans. Robot., vol. 21, no. 3, pp. 646–656, Aug. 2005. [23] K. D. Do, “Bounded controllers for formation stabilization of mobile agents with limited sensing ranges,” IEEE Trans. Autom. Control, vol. 52, no. 3, pp. 569–576, Mar. 2007. [24] A. Muhammad and M. Egerstedt, “Connectivity graphs as models of local interactions,” J. Appl. Math. Comput., vol. 168, no. 1, pp. 243–269, Sep. 2005. [25] A. Muhammad, M. Ji, and M. Egerstedt, “Applications of connectivity graph processes in networked sensing and control,” in Proc. Workshop Netw. Embedded Sens. Control, Notre Dame, IN, Oct. 2005. [26] M. Egerstedt, A. Muhammad, and X. Hu, “Formation control under limited sensory range constraints,” in Proc. 10th Mediterranean Conf. Control Autom., Lisbon, Portugal, Jul. 2002. [27] H. G. Tanner, “Flocking with obstacle avoidance in switching networks of interconnected vehicles,” in Proc. IEEE Int. Conf. Robot. Autom., New Orleans, LA, Apr. 26–May 1, 2004, pp. 3006–3011. [28] R. Olfati-Saber and R. M. Murray, “Distributed structural stabilization and tracking for formations of dynamic multi agents,” in Proc. IEEE Conf. Decis. Control, Dec. 2002, pp. 209–215. [29] H. Axelsson, A. Muhammad, and M. Egerstedt, “Autonomous formation switching for multiple mobile robots,” in Proc. IFAC Conf. Anal. Des. Hybrid Syst., Sant-Malo, Brittany, France, Jun. 2003. [30] The Player and Stage Project. [Online]. Available: http://playerstage. sourceforge.net/.
1169
Cheng-Heng Fua (S’03) received the B.Eng. degree in electrical and computer engineering from the National University of Singapore (NUS), Singapore, in 2003. He is working toward the Ph.D. degree at the NUS Graduate School for Integrative Sciences and Engineering (NGS), NUS. Currently, he is a Research Engineer at the Institute of Infocomms Research (I2R), Singapore. His current research interests include the area of autonomous multirobot collaboration and planning. Mr. Fua is the recipient of the A*STAR Graduate Scholarship, awarded by the Agency for Science, Technology, and Research (A*STAR), Singapore. Shuzhi Sam Ge (S’90–M’92–SM’00–F’06) received the B.Sc. degree from the Department of Automatic Control, Beijing University of Aeronautics and Astronautics, Beijing, China, in July 1986, and the Ph.D. degree and the Diploma of Imperial College from the Departments of Mechanical Engineering and Electrical Engineering, Imperial College of Science, Technology, and Medicine, University of London, London, U.K., in January 1993. Currently, he is the Director of Social Robotics Laboratory, Interactive Digital Media Institute, National University of Singapore (NUS), Singapore, and Supervisor of Edutainment Robotics Laboratory, Department of Electrical and Computer Engineering, NUS. He is the author or coauthor of more than 300 papers published in international journals, is the coauthor Adaptive Neural Network Control of Robotic Manipulators (World Scientific, 1998), Stable Adaptive Neural Network Control (Kluwer, 2001), and Switched Linear Systems: Control and Design (SpringerVerlag, 2005), and is the Editor of Autonomous Mobile Robots: Sensing, Control, Decision Making, and Applications (Taylor and Francis, 2006). His current research interests include social robotics, multimedia fusion, adaptive control, and intelligent systems development. Dr. Ge is an Elected Member of the Board of Governors, IEEE Control Systems Society. Currently, he serves as an Editor of the Taylor & Francis Automation and Control Engineering Series. He has served as an Associate Editor for a number of flagship journals including the IEEE TRANSACTIONS ON AUTOMATIC CONTROL, the IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, the IEEE TRANSACTIONS ON NEURAL NETWORKS, and the Automatica. Khac Duc Do received the M.E. degree from the University of Wollongong, Wollongong, New South Wales, Australia, and the Ph.D. degree (with distinction) from the University of Western Australia, Perth, Australia, in 1999 and 2003, respectively, both in Mechanical Engineering. Currently, he is a Senior Lecturer in the School of Mechanical Engineering, University of Western Australia. His current research interests include control of nonlinear systems, control of multiple agents, control of land, air, and ocean vehicles, and control of systems governed by partial differential equations. Khiang-Wee Lim (M’83–SM’92) received the Graduate degree in electrical engineering from the University of Malaya, Kuala Lumpur, Malaysia, and the D.Phil. degree in engineering science from the University of Oxford, Oxford, U.K., in 1978 and 1982, respectively. He has been with the National University of Singapore, Singapore, and the University of New South Wales, Sydney, New South Wales, Australia. Currently, he is an Executive Director at the Institute of Materials Research and Engineering, Singapore. Concurrently, he is the Programme Director for Research at Singapore’s Science and Engineering Research Council, Agency for Science, Technology, and Research (A*STAR), Singapore, and the Programme Director for the Singapore International Graduate Award (SInGA). He is currently convening the Working Group for Standardization, Risk Assessment, and Safety of the Asian Nano Forum as well as leading the Singapore National International Standards Organization (ISO)/TC 229 Working Group that is part of the ISO effort to develop standards in nanotechnology.
