Multi-AUV Hunting Algorithm Based on Bio-inspired ... - SAGE Journals

International Journal of Advanced Robotic Systems

ARTICLE

Multi-AUV Hunting Algorithm Based on Bio-inspired Neural Network in Unknown Environments Regular Paper

Daqi Zhu1, Ruofan Lv1*, Xiang Cao1 and Simon X. Yang2 1 Laboratory of Underwater Vehicles and Intelligent Systems, Shanghai Maritime University, Shanghai, China 2 The Advanced Robotics and Intelligent Systems Laboratory, School of Engineering, University of Guelph, Guelph, Canada *Corresponding author(s) E-mail: [email protected] Received 19 May 2014; Accepted 17 September 2015 DOI: 10.5772/61555 © 2015 Author(s). Licensee InTech. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

1. Introduction

The multi-AUV hunting problem is one of the key issues in multi-robot system research. In order to hunt the target efficiently, a new hunting algorithm based on a bioinspired neural network has been proposed in this paper. Firstly, the AUV’s working environment can be represent‐ ed, based on the biological-inspired neural network model. There is one-to-one correspondence between each neuron in the neural network and the position of the grid map in the underwater environment. The activity values of biological neurons then guide the AUV’s sailing path and finally the target is surrounded by AUVs. In addition, a method called negotiation is used to solve the AUV’s allocation of hunting points. The simulation results show that the algorithm used in the paper can provide rapid and highly efficient path planning in the unknown environment with obstacles and non-obstacles.

An autonomous underwater vehicle (AUV) is a type of intelligent robot that can travel in the underwater environ‐ ment without requiring input from an operator [1-2]. AUVs have been studied by many scientists and applied in a variety of tasks such as underwater rescue, detection, location, etc. Many achievements in single AUV research have been made. However, many complicated tasks nowadays go beyond the single AUV’s capability. MultiAUV systems, in recent years, have been studied in areas such as formation [3-4], localization [5], cooperative hunting [6-8], cooperation searching [9], path planning [10-11], task assignment and cooperation [12-15], due to their outstanding robustness and high efficiency of coordi‐ nation and collaboration. Among the areas mentioned above, the multi-AUV hunting problem has attracted much attention, because it can be applied in military tasks and is a good verification of cooperation and coordination of a multi-AUV system.

Keywords Multi-AUV (Autonomous Underwater Vehi‐ cle), Bio-Inspired Neural Network Algorithm, Hunting, Path Planning

Much research has been carried out recently on the multirobot hunting issue and some approaches are proposed in Int J Adv Robot Syst, 2015, 12:166 | doi: 10.5772/61555

1

this paper. The hunting algorithm can essentially be classified into two categories: centralized control method‐ ology and distributed control methodology. The difference is whether or not there is a supervisor. Various methods have been proposed, which include behaviour based, virtual structure based, leader-follower, artificial potential based and graph theory based methods. Grinton [16] presented a mechanism of commitments and conventions to guide the multi–robots’ cooperation in a hunting task. Sauter [17] used a reinforcement learning method with animal behaviour to conduct research on the hunting problem. Cai [18] proposed an improved auction algorithm for multi-robot hunting cooperative behaviour. However, all of the above articles concentrate on the known calculate the for velocity of hunting robots in order to environment a cooperative robot hunting task. In execute a hunting task. Sheng [21] has proposed reality, the working environment for robots is oftena method based on diffusion adaptation over hunting networks to unknown. In order to deal with a multi-robot task conduct research environment, on intelligentNighot predators in the unknown [19] hunting proposedfor a schools of fish. However, the previous fourfor papers did hunting strategy with swarm intelligence hunting not consider map-building and robots to encircle the target. comprehensively Feng [20] presented anobstacle input– avoidance was notlinearization usually considered in the literature.the output feedback algorithm to calculate velocity of hunting robots in order to execute a hunting Recently, some researchers approached thediffusion hunting task. Sheng [21] has proposedhave a method based on process with simple obstacles. Yamaguchi [22]on proposed adaptation over networks to conduct research intelli‐ a method based on for making for gent predators hunting schoolstroop of fish.formations However, the enclosing the and did presented a smoothmap-building time-varying previous fourtarget papers not consider feedback control and lawobstacle for coordinating the not motions of comprehensively avoidance was usually considered in thePan literature. multi-robots. [23] applied the improved reinforcement algorithm to the multi-robot hunting Recently, some researchers have approached the hunting problem. However, in these studies the hunting target is process with simple obstacles. Yamaguchi [22] proposed a often static and it is not fully consistent with the real method based on making troop formations for enclosing environment. the target and presented a smooth time-varying feedback control law for coordinating the motions of multi-robots. To tackle the shortcomings above, Ma [24] Pan [23] applied the improved discussed reinforcement algorithm to proposed a cooperative strategy with in dynamic the multi-robot huntinghunting problem. However, these alliancethe to hunting chase atarget moving target. can studies is often staticThis and itmethod is not fully shorten the completion time to some extent. Wang [25] consistent with the real environment. proposed a new hunting method with new definition To tackle ofthe shortcomings discussedangle above, Mafinally [24] concepts occupy and overlapping and proposed a cooperative hunting strategy with dynamic calculated an optimized path for multi-robot hunting, but alliance to chaseis atoomoving target. This location methodofcan the environment open and the initial the shorten the completion time to some extent. Wang [25] hunting robots is too close to the moving target. Next, Ni proposed a new hunting method with new definition and Yang [26] proposed an algorithm based on a bioconcepts of occupy and overlapping angle and finally inspired neural network model with formation strategy calculated an optimized path for multi-robot hunting, but that was applied in a hunting task with good the environment is too open and the initial location of the communication among several neurons; good hunting robots is too close to the moving target. Next, Ni coordination can be viewed during the whole hunting and Yang [26] proposed an algorithm based on a biotask. However, in the catching stage, the robots depend inspired neural network model with formation strategy on the formation strategy and do not need the guidance that was applied in a hunting task with good communi‐ of the neural network. Therefore, although there have cation among several neurons; good coordination can be been many approaches applied to the multi-robot viewed during the whole hunting task. However, in the hunting problem, the limitations in terms of coordination, catching stage, the robots depend on the formation robustness and effectiveness of a robot team mean that strategy and do not need the guidance of the neural these methods cannot be fully applicable for a multi-AUV network. Therefore, although there have been many cooperative applied hunting problem in underwater approaches to the multi-robot hunting problem, circumstances. the limitations in terms of coordination, robustness and

2

effectiveness of a robot team mean that these methods cannot be fully applicable for a multi-AUV cooperative hunting problem in underwater circumstances. This paper focuses on the situation in which the environ‐ ment is unknown and the target is intelligent, with unpre‐ dictable and irregular motions. The multi-AUV hunting algorithm based on the bio-inspired neural network is presented. The hunting AUVs’ paths are guided through the bio-inspired neural network and the results show that it can achieve the desired hunting result efficiency. This paper is organized as follows: Section 2 describes the map of the hunting process and four kinds of hunting final states are given. In Section 3, the bio-inspired neural network is designed. The strategy of pathin planning algorithm and the whole hunting process are described planning and the whole hunting process are 4described in detail. Simulations are conducted in Section and Section detail. Simulations are conducted in Section 4 and Section 5 concludes the whole paper. 5 concludes the whole paper.

2. Problem Statement 2. Statement InProblem this paper, a cooperative hunting task of multi-AUV in an unknown environment is studied. The multi-AUV In this paper, a cooperative hunting task of multi-AUV in system has no information about the underwater an unknown environment is studied. The multi-AUV environment. The hunting task will be accomplished system has no information about the underwater environ‐ when the target is encircled by hunting AUVs. The ment. The hunting task will be accomplished when the underwater environment model is presented by using a target is encircled by hunting AUVs. The underwater discrete grid map. The grid map divides the working environment model is presented by using a discrete grid condition into cells of the same size, while every cell has map. The grid map divides the working condition into cells two states - obstacles and free space, as shown in Figure of the same size, while every cell has two states - obstacles 1. free space, as shown in Figure 1. and The The two-dimension two-dimension grid grid map map isislabelled labelledas asV. V.The Thespace space and of the hunting region is alsoisdiscretized. Thus, andthe thetime time of the hunting region also discretized. the hunting area canarea be defined a set of as grid Thus, the hunting can beasdefined a maps. set of The grid number of AUVs is denoted as P = P , P , ...P and in { } maps. The number of AUVs as C C1 isC2 denoted Cr this paper research work on the that in focuses this paper the condition research work PC  {PC1 ,the PC2 ,... PCr } and only one target is hunted by multi-AUVs. Hence the target focuses on the condition that only one target is hunted by is labelled as Ev and the obstacles are denoted as multi-AUVs. Hence the target is labelled as Ev and the Ob = {Ob1, Ob2, ...Obs }. The target has the same intelligent obstacles are denoted as Ob  {Ob1 , Ob2 ,...Obs } . The target abilities as the hunting AUVs. Each AUV has 360 degree has the same intelligent abilities asofthe hunting AUVs. visual capability. The detection angle each AUV and the Each AUV has 360 degree visual capability. The detection 0 0 target is 360 / 8 = 45 respectively. angle of each AUV and the target is 3600 / 8  450 respectively.

Figure 1. Two dimension map

Figure 1. Two dimension map

This paper focuses the| doi: situation in which the Int J Adv Robot Syst, 2015, on 12:166 10.5772/61555 environment is unknown and the target is intelligent, with unpredictable and irregular motions. The multiAUV hunting algorithm based on the bio-inspired neural network is presented. The hunting AUVs’ paths are

1 2

3

1 2

bots depend he guidance there have multi-robot oordination, m mean that a multi-AUV underwater

Figure 1. Two dimension map

which the s intelligent, The multipired neural ’ paths are ork and the unting result

describes the of hunting pired neural egy of path

1

3

1

2

2

3

1 2

2 3

4 1

Figure 2. Target is hunted by AUVs in four conditions (a) Hunted state in corner (b) Hunted state in boundary (c) Hunted state with help of obstacles (d) Hunted state by four hunting AUVs

When the hunting process begins, the hunting AUVs will move towards the moving target. During the process, the hunting AUVs can avoid the obstacles and find a short path to catch the target. The target will judge whether there are any AUVs lying in the neighbouring cells. If so, then the target will try to modify its moving direction and run to the free space. Figure 2 shows the conditions where the target is successfully surrounded by hunting AUVs. The final hunting state can be divided into four situations, which are, respectively: the target surrounded by AUVs at a corner, at a boundary, with help of obstacles and without any help.

proposed bio-inspired neural network algorithm and the negotiation method, without a further synchronization method. The synchronization strategy will be considered in the further multi-AUV hunting research, in order to improve the hunting efficiency. The hunting problem for AUVs and mobile robots is theoretically the same, so this work is a preliminary study for the multi-AUV hunting problem. In this paper, our starting point is to try to apply this method to the AUV system, which has not been considered in previous work. Unlike the mobile robot or an Unmanned Aerial Vehi‐ cle (UAV), due to the complicated underwater environ‐ ment, the obstacles are assumed to be unknown and will be detected by underwater sensors, especially sonar, which is very different from mobile robots or UAVs. The work of underwater map building has been examined in the author’s former work [31]; therefore in this paper only a simple conclusion is given as a fundamental part of the AUV hunting problem. Here, the target is assumed to be moving on a set path; when it detects the risk of hunting AUVs, it will move to avoid the hunting. In this paper, since studies have already been carried out on map building and localization, we have concentrated on the hunting process. 3.1 Bio-inspired Neural Network Algorithm The “shunting model” proposed by Grossberg is shown in the following formulation:

3. Hunting Algorithm based on Bio-inspired Neural Network The neural network model, as a highly parallel distributed system, has shown its superiority in the mobile robot path planning and trajectory tracking research. On the whole, the study process is the essential part when a neural network is applied, but timeliness and efficiency cannot be guaranteed. The bio-inspired neural network model was proposed by Hodgkin and Huxley in 1952, by using a circuit element to describe the electric current of membrane [27]. Grossberg [28] summarized and improved this model into a “shunting model”, which is based on the HodgkinHuxley model. The bio-inspired neural network model was applied to complete coverage path planning by Yang and Luo [29]. Pichevar and Rouat applied the approach to solve the sound source segregation problem [30]. The bioinspired network model applied in the multi-robot coop‐ erative hunting area does not need any learning process and the external excitation and inhibition will lead the robot to select every step to reach the goal. In his 2011 paper [26], Ni used the bio-inspired neural network model with a formation and dynamic alliance algorithm to chase targets. Unusually, in this paper, the bioinspired neural network is directly used in an AUV hunting task without the assistance of any other algorithm. This means that the hunting process can be completed with the

dxi = - Axi + ( B - xi )Si+ - ( D + xi )Sidt

(1)

This function is called the shunting equation. In this equation, xi is the neural activity of the i-th neuron; A, B

and D represent the passive decay rate and the upper and lower bounds of the neural activity respectively, which are nonnegative constants; Si+ and Si− are the excitatory and

inhibitory inputs to the neuron. In the hunting process, the hunting AUVs’ motions are guided by the dynamic landscape of the neural network. The excitatory input Si+ results from the target and its neighbouring AUVs and the input Si− only results from the obstacles. In this context, the dynamic of the i-th neuron in the neural network can be characterized by a shunting equation as k æ ö dxi = - Axi + ( B - xi ) ç [ I i ]+ + å wij [ x j ]+ ÷ - ( D + xi )[ I i ]ç ÷ dt 1 j = è ø

(2)

where k is the number of neural connections of the i-th neuron to its neighbouring neurons. The terms k

I ie + + ∑ wij xj j=1

+

and I io − are the Si+ and Si− in equation (1),

respectively.

Daqi Zhu, Ruofan Lv, Xiang Cao and Simon X. Yang: Multi-AUV Hunting Algorithm Based on Bio-inspired Neural Network in Unknown Environments

3

+

The term a

is a linear-above-threshold function defined

as a = max {a, 0} ; similarly the term a − = min { − a, 0}. I i +

and I i

−

+

are the variables that represent the input to the i-

th neuron from the target and obstacle, respectively. They are defined as ìE ï I i = í- E ï0 î

if it is aneighboring celltotarget if it is anobstacle otherwise( free space )

(3)

(4)

where E q Bare , which is a very large constant. In qk and − ql | is the where two vectors and Euclidean | qk positive l equation (2), the term ij is defined distance between them.wThe functionas f (a) is a monotoni‐ cally decreasing function, as wij which  f (| qkisdefined ql |) (4) where qk and ql are two vectors and | qk  ql | is the Euclidean distance The /a 0 > B , which is a very large positive constant. In equation (2), the term wij is defined as wij = f (| qk - ql |)

the AUV to move. Under the neural network guidance, the AUV’s moving strategy can be followed as

(5)

  / a if 0  a  Rn f (a)   (5) ifconstants. a  R n Obviously, the 0 where μ and Rn are positive positive constants. Obviously, where connection  and Rn are weight coefficients are symmetrical, thattheis, connection coefficients are symmetrical, is, wijweight = w ji . Figure 3 shows the neural network model that in a 2-D . Figure 3 shows the neural network model in a w  w environment [32]. ij ji 2-D environment [32].

for the AUV’s current position. m is the number of neigh‐ bouring cells. Pc , Pn and Pp represent the AUV’s current

position, next position and previous position respectively. In the AUV’s moving process, the neighbouring cell of maximum activity value will be chosen as the next position. Simultaneously, the activity value of the whole neural network will be refreshed. 3.3 Strategy of Intelligent Target On the basis of the mechanism of the bio-inspired neural

3.3 Strategy of Intelligent Target network model in the multi-AUV hunting process, the On the basis of the mechanism of the bio-inspired neural target‘smodel motioninis the alsomulti-AUV limited by the control of the neuron’s network hunting process, the value. As mentioned above, the target is intelligent and the target‘s motion is also limited by the control of the escape runaway choice for the target is random. To neuron’s value. As mentioned above, the target isput it simply, the target will run to an space intelligent andmoving the escape runaway choice for open the target is with few obstacles or hunting AUVs. When the hunting random. To put it simply, the moving target will run toAUVs impede route, the escape direction will be anblock openorspace withitsfew obstacles or hunting AUVs. immediately affected. If the hunting AUV occupies When the hunting AUVs block or impede its route, theone of escape direction be immediately affected. If the the grids aroundwill the target, the movement direction will be hunting AUV occupies one of the grids around theoccupied target, by limited. When the surrounding grids are all the movement the the hunting AUVs,direction the targetwill will be stoplimited. moving.When Generally, surrounding hunting AUVs, maximum grids speedare of all theoccupied movingby target is less thanthe that of target will stopAUV. moving. Generally, the maximum speed the hunting of the moving target is less than that of the hunting AUV.

3.4 Hunting Process

3.4 Hunting Process When the hunting process begins, all the AUVs will

When the hunting process begins, all the AUVs will pursue

pursue the moving target together. In order to show clear the moving target together. In order to show clear results

× 2) is to defined to the results of hunting, a matrix of hunting, a matrix Trace (kTrace × 2) is( kdefined memorize

k represents position the of the AUV for path planning. memorize position of the AUV for pathThe planning. The the

of hunting steps for each AUV. The hunting task represents the number of hunting steps for each AUV. k number of the AUVs is to move towards encircling the target in a The hunting task of the AUVs is to move towards few steps. The whole process of the hunting behaviour can encircling the target in athe fewfollowing steps. Theprocedures: whole process of be summarized with the hunting behaviour can be summarized with the

Step 1: Initialize the whole activity values of cells to zero.

following procedures:

Step1:2:Initialize Set the initial position of values each AUV to to the current Step the whole activity of cells zero.

Figure 3. 2-D model of neural network

Figure 3. 2-D model of neural network

In this structure, each neuron is connected by adjacent

In this structure, each neuron is connected by adjacent neurons, which form the whole network for transmission neurons, which form the whole network for transmission of activity. of activity. 3.2 Strategy of Path Selection AUV’sofmoving path is guided by the activity of the 3.2An Strategy Path Selection neural network. Let us assume that an AUV’s current

An AUV’s moving path is guided by the activity of the theus constant represents the number position is k andLet neural network. assumem that an AUV’s current of neighbouring neural cells. Thus, there arethe m selections position is k and the m represents constant number of m selections neighbouring neural cells. Thus, there for for the AUV to move. Under theareneural network 4

guidance, the AUV’s moving strategy can be followed as

Int J Adv Robot Syst, 2015, 12:166 | doi: 10.5772/61555

Path  {Pn | x pn  max{xkl , l  1, 2,...m}, Pp  Pc , Pc  Pn }

where xkl

(6) represents the activity value of neighbouring

location.

Step 2: Set the initial position of each AUV to the current

Step 3: AUV will find the next step by choosing the location. maximum activity value eightstep neighbouring cells. Step 3: AUV will find theofnext by choosing the

maximum activity of eight cells. Trace. Step 4: Store thevalue current AUVneighbouring position to matrix Step 4: Store the current AUV position to matrix Trace.

Step 5: Set the activity value of the cells that the AUV has

Step 5: Setthrough the activity value of the cells that the AUV travelled to zero. has travelled through to zero.

Step 6: Judge whether the current position is the neigh‐ bouring cell of the target. If it is, set the activity value of the neighbouring cell to of −the is, set other the activity E intarget. order If to it prevent AUVs from current position value of the current to  E inotherwise order to prevent moving to the sameposition cell by mistake; go to Step 3. Step 6: Judge whether the current position is the

other AUVs from moving to the same cell by mistake; otherwise go to Step 3. Step 7: If the target continues to move, the AUV will follow the target until the up, down, left and right positions of the target are occupied by hunting AUVs.

Figure 4. Method of negotiation Figure 4. Method of negotiation

Step 7: If the target continues to move, the AUV will follow 3.5 Negotiation Method the target until the up, down, left and right positions of the In the hunting process, due to the attraction of the target are occupied by hunting AUVs. Then it will stop Figure 4. Method of negotiation maximum moving. neuron value, the hunting AUVs will occupy

in [33]. It will be a separate work, which needs further research.

the hunting points randomly. However, without a The hunting process will show that the AUV can encircle 3.5 Method mechanism forNegotiation allocating hunting point for will each move AUV the target until it cannotthe escape. The AUV In the hunting process, due to the attraction of the towards thethe target directly and will avoid task the various in balance, effectiveness of the hunting will be maximum neuron value, the hunting AUVs will occupy obstacles. weakened. Thus, a method called negotiation is presented the hunting points randomly. However, without a

in this paper to solve the problem that is mentioned mechanism 3.5 Negotiation Method for allocating the hunting point for each AUV in balance, the effectiveness of the hunting task will above. The whole method can be summarized as Figure 4. be Figure 5. The schematic diagram of the negotiation method In the hunting process, due to the attraction of the ismaxi‐ weakened. Thus, a method called negotiation presented mum neuron this value, thetohunting AUVs will that occupy the paper solve the problem is mentioned Under the inguidance of the negotiation method, the 4. Simulation Studies hunting points randomly. However, without a mechanism above. The whole method can be summarized as Figure 4. To Figure demonstrate the feasibility and effectiveness hunting points will be allocated by AUVs automatically; 5. The schematic diagram of the negotiation method of the for allocating the hunting point for each AUV in balance, Figure 5. The algorithm, schematic diagram of the negotiation experiments method proposed some simulation have the effectiveness of the task will weakened. hence it can make the hunting AUV finish the be hunting task Under the guidance of the negotiation method, the been 4. Simulation Studies conducted. In this section, the simulation can be Thus, a method called negotiation is presented in this paper quickly and shorten anwill unnecessary As To demonstrate the feasibility and effectiveness ofwithout the hunting points be allocated sailing by AUVspath. automatically; divided into two parts. Hunting with and 4. Simulation Studies to solve the problem that is mentioned above. The whole proposed algorithm, some simulation experiments have shown themake four AUVs run to the hence it5,can AUV4. finish the moving hunting task obstacles in an unknown environment is simulated and methodin canFigure be summarized as the Figure To been demonstrate theIn feasibility conducted. this section,and the effectiveness simulation can of be the compared with another approach. In addition, the growth quickly and and the shorten unnecessary target respectively four an hunting points sailing markedpath. as As proposed dividedalgorithm, into two parts. Hunting with and without some simulation experiments have Under the guidance of the negotiation method, the hunting in activity value each AUV will be displayed in aand chart. in Figure four AUVs method. run to theThe moving been obstacles in anofunknown environment is simulated 1 to 4 are shown allocated with 5, thethenegotiation conducted. In this section, the simulation can be points will be allocated by AUVs automatically; hence it Thecompared simulation environment used is Windows 7, withparts. another approach. In addition, the growth target respectively and the four hunting points marked as divided into two Hunting with and without obstacles AUV can then hunting task successfully. can make the finish AUV the finish the hunting task quickly and Intel(R)Core(TM)2 4Ga chart. memory. in activity value ofDuo eachCPU AUVE8400 will be 3.00GHz, displayed in to 4 are allocated with the negotiation method. The in an unknown environment is simulated and compared shorten an 1unnecessary sailing path. As shown in Figure simulation TheThe compilation toolenvironment is MATLABused 2011a.is Windows 7, withIntel(R)Core(TM)2 another approach. In addition, the growth in activity AUV can finish the hunting successfully. 5, the four AUVs runthen to the moving targettask respectively and Duo CPU E8400 3.00GHz, 4G memory. If the hunting AUVs are more than needed (4 AUVs value each AUV be displayed in a chart. The Theof compilation tool will is MATLAB 2011a. the four hunting points marked as 1 to 4 are allocated with 4.1 Simulation Design shown in Figure 5), the task assignment can be conducted simulation environment used is Windows In‐ the negotiation The AUV can than thenneeded finish (4theAUVs In these experiments, a task is given for a team of7,AUVs If the method. hunting AUVs are more 4.1 Simulation Design tel(R)Core(TM)2 Duo CPU E8400 3.00GHz, 4G memory. first. Thetask task allocation given according the hunting successfully. shown in Figure can 5), thebetask assignment can betoconducted one Ev. The PC  {these PC1 , Pexperiments, In aMATLAB task only is given for atarget, team of AUVs C2 ,...P Cr } iswith The compilation tool 2011a. first. The task that allocation canassigned be given a according distance and the AUVs are task willto the If the hunting AUVs are more thannot needed (4 AUVs shown PC  {PC1 , PC2 ,...PCr } with only one target, Ev. The of the hunting area is a distance and thetask AUVs thatbeare not a The task will environment stand still.5),Some work on allocation hasassigned beenfirst. carried in Figure the task assignment can conducted 4.1 Simulation Design environment of the hunting area is a still. Some work task has been carried NS  NS  20  20 grid map. The hunting task can be task by allocation can bein given according toathe distance and out the stand authors [33]. It on will beallocation separate work, NS  experiments, NS  20  20 grid map.is The hunting task can be these a task given forwithout a team of AUVs out are by not the assigned authors ina [33]. will be astill. separate the AUVs that task It will stand Somework, In divided into two conditions: hunting obstacles which needs further research. P = P , P , ...P with only one target, Ev. The environ‐ divided into two conditions: hunting without obstacles { } C C1 C2 Cr work on task allocation has been carried out by the authors which needs further research. and with different shapes of obstacles. The boundaries of

and with different shapes of obstacles. The boundaries of Daqi Zhu, Ruofan Lv, Xiang Cao and Simon X. Yang: Multi-AUV Hunting Algorithm Based on Bio-inspired Neural Network in Unknown Environments

5

ment of the hunting area is a NS × NS = 20 × 20 grid map. The hunting task can be divided into two conditions: hunting without obstacles and with different shapes of obstacles. The boundaries of the area are known to the AUVs as well as to the target, while the environmental information of the whole area is unknown to both. The number of AUVs is set at four and their movement is based on the bio-inspired neural network model in the sections above. The target is intelligent and moves ran‐ domly until it is surrounded by hunting AUVs. The speed of the hunting AUVs is set at 1 second / grid and the target speed is 4 second / grid . The parameters are set at A =2, B =1, D =1, μ =0.6, E =100, Rn =2. 4.2 Hunting Simulation Experiment without Obstacles

with obstacles. The red data show the value corresponding to the next position that the AUV chooses. Obviously, the AUV selects the cell with the biggest activity value from the neighbouring eight cells. In Table 1, in the initial stage of the hunting process, PC3 is

located in the position (20,1), which is adjacent to the corner and boundary, so the number of neighbouring neurons does not equal eight. PC3 then chooses Pc3(x,y+1) to be the

next position, corresponding to (20,2), because the activity value of the neuron in this position is 6.945e-13, which is the largest of the neighbouring cells.

After the AUV runs one step, the whole system of the neural network will be refreshed immediately; then PC3 will judge whether the next position is the neighbouring cell of the target or not. Obviously, the answer is no. Therefore it chooses Pc3(x-1, y+1), which corresponds to (19,3). The activity value in that position is 4.396e-12, which is the largest of five neighbouring values. Similarly, PC3 chooses the next position (18,4) by the same mechanism of path Int J Adv Robot Syst, 2015, 12:166 | doi: 10.5772/61555

PC4

Figure 6(c

process, t

16 14

AUVs in (

12

see that th

10

do not coll

8 6

Table 1 lis

4

in

2 5

10

15

PC1

PC2

PC3

the

h

circumstan

20

(a)

correspond

PC4

Obviously

  20

4(a) shows the initial locations and state of the hunting condition. Figure 6(b) shows the hunting process for the first seven steps. The target has already found the hunting AUVs moving towards it and it starts to escape from its initial location.

Table 1 lists the activity value of the neuron at each step in the hunting process of AUV PC3 under the circumstance

PC3

18

18

Figure 6(c) shows that in the final state of the hunting process, the moving target is surrounded by hunting AUVs in (11,9) and, through the hunting task, it is easy to see that the AUVs are moving directly to the target and do not collide with each other.

PC2

20

The first simulation is conducted to test the cooperative hunting process without obstacles. For easy discussion, it is assumed that there are four hunting AUVs with only one target. The initial location of the target is (11,6). The hunting AUVs are PC1, PC2, PC3, PC4 and the initial positions of them are PC = {(1, 20), (1, 0), (20, 1), (20, 20)}, respectively. Figure

6

PC1

and the sta seven steps

activity va

16 14

In Table 1

12 10

is located

8

corner an

6

neurons

4

Pc3(x,y+1)

2 5

10

15

because th

20

(b) PC1

PC2

PC3

6.945e-13, PC4

 20 18 16 14 12 10 8 6 4 2 5

10

15

20

(c) Figure 6. Hunting process of the simulation (a) initial locations

Figure 6. Hunting process of the simulation (a) initial locations and the state of hunting condition (b) hunting process - first seven steps (c) final locations position (x,y) withCurrent trajectories of AUVs

Neighbouring cells Pc3(x+1,y)

(20,1)

(20,2)

(19,3)

(18,4)

-

-

1.530e-13

1.627e-09

planning. Now the number of neurons is eight, because the to the boundary. the same positionPc3(x-1,y) of PC3 is not close3.848e-14 1.493e-12 With 1.833e-10 7.500e-07 methodPc3(x,y-1) of choosing the maximum activity value of 3.848e-14 2.643e-12 1.547e-08 neighbouring neurons, when PC3 sails to (11,8), it finds that Pc3(x,y+1)

6.945e-13

1.530e-13

9.944e-12

7.156e-08

the position is next to the moving target and the other positions of neighbouring cells are occupied Pc3(x+1,y+1) - of the target 3.065e-13 2.796e-09 by other AUVs. It then stops hunting and finishes its Pc3(x-1,y+1) 6.373e-14 4.396e-12 4.782e-10 1.561e-06 hunting task. The results shown in Table 1 correspond to the hunting process in Figure 6 and confirm that it is effective to apply the bio-inspired neural network algo‐ rithm to the multi-AUV hunting task.

(17,5

8.603e

0.000

6.226e

1.873e

1.221e

0.000

Current position (x,y) (20,1)

(20,2)

(19,3)

(18,4)

(17,5)

(16,6)

(15,7)

(14,7)

(13,7)

(12,8)

Neighbouring cells Pc3(x+1,y)

-

-

1.530e-13

1.627e-09

8.603e-07

7.879e-05

0.0021

0.0234

0.1291

0.4016

Pc3(x-1,y)

3.848e-14

1.493e-12

1.833e-10

7.500e-07

0.0001

0.0060

0.0719

0.3138

0.6023

0.9809

Pc3(x,y-1)

-

3.848e-14

2.643e-12

1.547e-08

6.226e-06

0.0005

0.0116

0.0793

0.2566

0.5678

Pc3(x,y+1)

6.945e-13

1.530e-13

9.944e-12

7.156e-08

1.873e-05

0.0009

0.0119

0.0881

0.3306

0.9807

Pc3(x+1,y+1)

-

-

3.065e-13

2.796e-09

1.221e-06

8.953e-05

0.0019

0.0206

0.1173

0.3237

Pc3(x-1,y+1)

6.373e-14

4.396e-12

4.782e-10

1.561e-06

0.0002

0.0072

0.0611

0.2935

0.9807

-0.9195

Pc3(x+1,y-1)

-

-

6.654e-14

7.446e-10

4.736e-07

5.346e-05

0.0018

0.0197

0.3055

0.3865

Pc3(x-1,y-1)

-

5.276e-14

4.878e-11

2.529e-07

6.502e-05

0.0035

0.0565

0.2286

0.4206

0.7320

Next position

(20,2)

(19,3)

(18,4)

(17,5)

(16,6)

(15,7)

(14,7)

(13,7)

(12,8)

(11,8)

Table 1. The changing activity values of the neurons of PC in the hunting process of Figure 6(c) 3

4.3 Hunting Simulation Experiment with Obstacles To prove the robustness of the proposed approach, some obstacles are added to this part of the simulation. The shapes of the obstacles are varied, comprising U-shape, polygon-shape, square-shape and rectangle-shape, in order to increase the difficulty of the hunting task. In Figure 5, the yellow pentagram represents the target and the black blocks are the static obstacles in the simulation. The hunting AUVs are PC1, PC2, PC3, PC4,which still start moving from the

location of PC = {(1, 20), (1, 0), (20, 1), (20, 20)} respectively. With the guidance of the neural network, the AUVs will move directly to the target and avoid the obstacles. Figure 7(a) shows the initial state of the hunting process. Figure 7(b) shows the hunting process of each AUV and moving target and Figure 7(c) shows the final state and the whole trajectories of the target and AUVs. Figure 7(d) shows that the AUVs can complete the hunting task with different shapes of obstacles. Table 2 reflects the whole hunting process of one of the hunting AUVs (PC3). The dynamic changing values of the

neurons also show the mechanism of path planning for each hunting AUV. The sign “-”represents a cell that is out of boundary and whose activity value does not exist. The data that are marked in a red colour represent the maxi‐ mum value of the neighbouring eight cells of the current position.

In order to further prove the robustness of the proposed approach, the hunting process with a wider U-shaped obstacle has been simulated in Figure 8. It can be clearly seen that when the AUV is inside a U-shaped obstacle and the target is on the other side, the hunting AUV can navigate back and move around the obstacle to reach the target successfully.

with the artificial potential field method [34-35] applied in the hunting process. The potential fieldwork was proposed 15 years ago and has been applied in many areas. However, the application in a multi-agent system is still a new area, especially for the multi-AUV system, and a number of research papers on this topic are being published every year. In the artificial potential field method, a gravitational field to a target and a repulsive field to obstacles are built to work together, to lead the AUVs to move towards the target step by step. The direction of the hunting AUV is decided by a composition of forces, which include the gravitational pull from the target and the repulsion from the other hunting AUVs. A brief description of the artificial potential field method can be summarized as follows: first, construct a distance function between the AUV and the target:

r (r , g) = r( x1 , y1 ) - g( x2 , y2 ) The generated gravitational field can then be given as: U att 1 ( r (r , g )) = x r (r , g )

m

where m is a positive constant. The attractive force of the target is: Fatt1 = -ÑU att 1 ( r (r , g )) = mx r( x1 , y1 ) - g( x2 , y2 )

m -1

nRG

The distance function between the AUVs can be given as:

4.4 Comparison with Different Method To further test the priority of the proposed method applied to the hunting process, this paper conducts a comparison

N

U reps = å U reps ( r i (r , o)) i =1

Daqi Zhu, Ruofan Lv, Xiang Cao and Simon X. Yang: Multi-AUV Hunting Algorithm Based on Bio-inspired Neural Network in Unknown Environments

7

PC1

 

PC2

PC3

trajectories of AUVs (d) hunting with different types of obstacles

PC4

20

PC1

18 16

 

PC2

PC3

trajectories of AUVs (d) hunting with different types of obstacles

PC4

PC1

20

20

PC1

18

14 16 14

10

12

8

10

16

6

8

14 12

10

10

5

10

5

PCP2 C

1

P PCC2

3

15

10

(a)

15

20

8

8

20

(a) PCP 3 C

66

PC4

4

44

20

2

2

18 16

5

16 14

14

10

10

8

8

6

5

5

PC1

10

10

PC2

(b) PC3

15

15

20

20

PC4

(b)

 20 18P

C2

PC3

PC4

16

 20

14

18

12

16

10

14

8

12

6

10

4

8

2 5

6

10

4

  20 18 16

15

20

(c)

2

PC1

  20

PC1

PC2

PC3

185

10

16

(c)

14 P

C2

PC3

PC4 15

20

12 10

15

20

20

each hunting “-”represents cell that is neurons alsoAUV. showThe thesign mechanism of apath planning for out of boundaryFand( rwhose activity value does not exist. (r , o)) = -ÑU reps ( r i (r , o)) each hunting AUV. a cell that is reps i The sign “-”represents The data that are marked in a red colour represent the out of boundary activity value not exist. maximum value ofand the whose neighbouring eight cellsdoes of the The data that are marked in a red colour represent current position. The hunting AUVs will then be guided by the total forces the maximum value of the neighbouring eight cells of the of attraction and repulsion. In order to further prove the robustness of the proposed current position.

Figure 9 the shows the process simulation of U-shaped the artificial approach, hunting withresult a wider potential field method in a hunting experiment. AUVs obstacle has been simulated in Figure 8. It can beFour clearly Inlabelled order to further prove the robustness of the proposed as {1,2,3,4} start from locations {(25,25), (1,25), seen that when the AUV is inside a U-shaped obstacle approach, the hunting process with a wider U-shaped (25,1),(1,1)} respectively and hunt the red moving target, and the target is on the other side, the hunting AUV can obstacle has been simulated in cantarget be clearly which starts from (13,13) simultaneously. The is navigate back and move around theFigure obstacle8.toIt reach the finally caught in (13,24). Figure 10 shows the simulation seen that when the AUV is inside a U-shaped obstacle target successfully. result proposed in this paper.AUV Four can and thebased targetonisthe onmethod the other side, the hunting hunting AUVs start from locations {(25,25), (1,25),(25,1), 4.4 Comparison with Different Method navigate back and move around the obstacle to reach the To further test the and priority the escapes proposedfrom method (1,1)} respectively the of target (13,13), target successfully. which is finally hunted in (13,25). The number of step applied to the hunting process, this paper conducts a for each AUVwith in the process under the proposed comparison the hunting artificial potential 4.4 Comparison with Different Methodfield method [34method is shown in Figure 11. The result shows that the 35] further applied test in the hunting process. The potentialmethod To the priority of theprocess proposed average number of steps for the hunting is reduced fieldwork was proposed 15 years ago and has been by 45%.toTherefore the method proposed in this paper a applied the hunting process, this paper conducts applied areas. process However, the application in a appliedintomany the hunting is much more efficient.

comparison with the artificial potential field method [34multi-agent system is still a new area, especially for the

Theapplied reason forin the the superior performance can be explained 35] hunting process. The potential

PC4

multi-AUV and a field number of research papers on as follows:system, the potential method is basically designed

12

fieldwork 15 years ago has been on topic the modelling of a gravitational field and a repulsive this arewas beingproposed published every year. In the and artificial

10

field. Different designs the gravitational field and thein a applied in many areas. However, the to application potential field method, a of gravitational field a target

8 6

14

15

hunting AUVs ( PCthe ). mechanism The dynamic changing values themselves: neurons also show of path planning for of the 3

2

PC1

10

Figure 8. Hunting task with U-shaped obstacle

2

4

10

Figure 8. task with U-shaped obstacle Table 2 reflects theHunting whole hunting process of one of the Table 2 AUVs reflects whole hunting process of of the hunting (force ). The dynamic changing values ofone the PCthe The repulsive is generated between the hunting AUVs 3

4

6

5

Figure 8. Hunting task with U-shaped obstacle

12

12

target

12

6

2

18

PC4

target

14

2

20

PC3

PC4

18 16

4

PC1

PC2

PC3

18 20

12

4

PC2

repulsive field will affect the hunting performance, but cannot directly move to the target like the proposed biotogether, lead thenetwork AUVs move towards the target multi-AUV system, and atonumber ofFurthermore, research papers inspiredto neural method. one on step by step. The direction of the hunting AUV is decided important issue that has not been discussed is that the this topic are being published every year. In the artificial field method a shortage of local minimization bypotential a composition ofhasforces, which include the potential field method, a gravitational field toit amay target (called deadlock); hence, theand U-shaped obstacle, gravitational pull from the for target the repulsion from fall ainto the obstacle inside without any strategy, and repulsive field to obstacles areother built to work while the bio-inspired neural network method can solve it together, to lead the AUVs to move towards the target very well, as shown in Figure 9. and a repulsive fieldistostill obstacles built to work for the multi-agent system a new are area, especially

4 2 5

10

8

15

20

(d)

Figure 7. Hunting task with obstacles (a) initial location (b) Figure6 7. Hunting task with obstacles (a) initial location (b) hunting process hunting process of first six steps (c) final locations with of first4six steps (c) final locations with trajectories of AUVs (d) hunting with different types of obstacles 2

step by step. The direction of the hunting AUV is decided 8

20 Int J Adv Robot 5Syst, 2015,1012:166 | 15 doi: 10.5772/61555

(d) Figure 7. Hunting task with obstacles (a) initial location (b) hunting process of first six steps (c) final locations with

by

a

composition

of

forces,

which

include

the

gravitational pull from the target and the repulsion from

Current position

(20,1)

(20,2)

(19,3)

(18,4)

(17,4)

(16,4)

(15,5)

(14,6)

(13,7)

(12,8)

Pc3(x+1,y)

-

-

2.782e-14

1.203e-10

4.124e-08

2.427e-06

-0.9389

0.0031

0.0236

0.0937

Pc3(x-1,y)

1.947e-14

2.543e-14

6.987e-11

1.330e-07

2.059e-05

0.0004

0.0070

0.0559

0.2136

0.5257

Pc3(x,y-1)

-

9.876e-13

6.905e-13

4.028e-09

6.658e-07

1.672e-05

0.0005

0.0077

0.0496

0.1043

Pc3(x,y+1)

4.549e-13

6.905e-13

1.056e-12

-0.9389

4.582e-07

-0.9389

0.0024

0.0225

0.1170

0.5287

Pc3(x+1,y+1)

-

-

2.586e-14

5.404e-11

-0.9389

1.410e-06

-0.9389

0.0039

0.0241

0.1271

Pc3(x-1,y+1)

1.054e-13

3.193e-12

8.593e-11

7.994e-08

-0.9389

0.0009

0.0147

0.1085

0.9396

0.9401

Pc3(x+1,y-1)

-

-

2.432e-14

1.379e-10

3.5652e-08

2.130e-06

8.799e-05

0.0018

0.0162

0.0446

Pc3(x-1,y-1)

-

2.226e-14

3.193e-11

1.111e-07

6.3491e-06

8.531e-05

0.0021

0.0224

0.1208

0.1410

Next position

(20,2)

(19,3)

(18,4)

(17,4)

(16,4)

(15,5)

(14,6)

(13,7)

(12,8)

(11,8)

(x,y) Neighbouring cells

Table 2. The changing activity values of the neurons of PC in the hunting process of Figure 6(c) 3

In order to further show the priority of the proposed method used in the hunting process, a chart describing the comparison of the two methods is shown in Table 3. AUV

PC

1

PC

2

PC

3

PC

4

Average steps

Algorithm Steps for

42

42

42

42

42

Artificial potential method Steps for

26

22

25

19

23

In this simulation experiment, static obstacles are added to the 3-D map, as shown in Figure 12(a), where a blue cube hunting path From thissix point of view, is easy to represents anlength. obstacle. The AUVs are it labelled as conclude the bio-inspired neural network method is and they start from the initial PC1, PC2, PCthat , P , P , P C4 C5 C6 3 superior to potential fieldwork. position {(1,1,0),(10,0,2),(10,10,10),(3,10,10),(1,10,0),

45 40

(10,1,10)}. The AUVs approach the target according to the In order toneuron’s further show the selection priority of the proposed maximum activity mechanism and method used in the hunting process, a chart perform obstacle avoidance. The hunting taskdescribing is finally the comparison the two methods is shown in Table 3. completed at theofpoint (5,5,5). It should be noted that when AUV Average the target is surrounded PC by six PC2hunting PC3 AUVs, PC the success‐ Algorithm ful hunting state can be1 accomplished. Figure4 12(b) steps shows the final state of successful hunting and the moving Steps for Artificial 42 The42 42 up by 42 six trajectory of each AUV. target 42 is rounded potential method AUVs, which are all in the red cube. In order to show the Steps for Biofinal state clearly, an enlarged view of the hunting AUVs inspired 26 of being 22 25 23 and the neural target at the point caught is19 demonstrated network algorithm in Figure 12(b).

Number of steps for hunting AUVs

For the power consumption problem, since this work is based on the design of path planning, the power consump‐ tion can simply be in linear correlation with the hunting path length. From this point of view, it is easy to conclude that the bio-inspired neural network method is superior to potential fieldwork.

35 30 25 20 15 10 5 0

P

Figure 11. methods

Table 3. The comparison of step numbers between the two

4.5 Huntin In this se

methods

hunting i

Bioinspired

introduce

neural

the bio-in

network

case, whil

algorithm

D simula

Table 3. The comparison of step numbers between the two methods

4.5 Hunting Simulation Extended to 3-D Environment

selected a

In this section, some preliminary work on multi-AUV hunting in the three-dimensional (3-D) environment is introduced. The proposed hunting algorithm based on the bio-inspired neural network is extended to the 3-D case, while the basic idea is essentially the same. In the 3-D simulation experiment, the hunting map is also selected as the discretization grid map. Six AUVs are selected for the dynamic hunting of an escapee. It should be noted that the complex current situation in the actual three-dimensional environment is not considered.

selected f be noted

three-dim

In this sim

to the 3-D Figure 9. Hunting process in artificial potential field method

cube repr

Figure25 9. Hunting process in artificial potential field method

Daqi Zhu, Ruofan Lv, Xiang Cao and Simon X. Yang: Multi-AUV Hunting Algorithm Based 20 on Bio-inspired Neural Network in Unknown Environments

9

PC1 , PC 2 , position

{(1,1,0),(10 15

AUVs ap

be noted that the complex current situation in the actual three-dimensional environment is not considered.

and target experiment, at the point of being In this the simulation static obstacles are caught added

is

to the 3-D map,inasFigure shown12(b). in Figure 12(a), where a blue demonstrated

Figure 9. Hunting process in artificial potential field method

cube represents an obstacle. The six AUVs are labelled as 25

PC1 , PC 2 , P10C 3 , PC 4 , PC 5 , PC 6 and they start from the initial 8

position

20

6 4

{(1,1,0),(10,0,2),(10,10,10),(3,10,10),(1,10,0),(10,1,10)}. 2 15

The

0 AUVs approach the target according to the maximum 10 8 6

8

10

4 neuron’s activity selection mechanism4 and perform 2 6

0

10

2

0

obstacle avoidance. The hunting task is finally completed

(a) point Initial (5,5,5). huntingItstate underbeobstacles environment task at the should noted that when the 5

target is surrounded by six hunting AUVs, the successful 10

hunting state can be accomplished. Figure 12(b) shows 8

5

10

15

20

6

25

the final state of successful hunting and the moving 4 2

Figure 10. Hunting process in method proposed in this paper

Figure 10. Hunting process in method proposed in this paper

0 10

trajectory of each AUV. The target is rounded up by six 8

6

8

6

4 2

2

10

4

AUVs, which are all in the red Pcube. In order to show the 0

0

C4

final state clearly, an enlarged view of the hunting AUVs

is easy to method is

Artificial potential method Bio-inspired neural network method

45

PC3

proposed describing Table 3. Average steps

Number of steps for hunting AUVs

40

PC2

35

PC5

30

PC1

20

(b) Final hunting state under obstacles environment task and partial enlarged detail

15 10 5 0

42

Pc1

Pc2

Pc3

Pc4

Figure 12. The multi-AUV hunting simulation experiment under static Figure 12. The multi-AUV hunting simulation experiment obstacles

average

Figure 11. Hunting efficiency comparison between the two Figure 11. Hunting efficiency comparison between the two methods methods

23

n the two

ethod

10

PC6

25

4.5Conclusion Hunting Simulation Extended to 3-D Environment 5. In this section, some preliminary work on multi-AUV Cooperative hunting by multi-AUVs in an unknown hunting in the environment is environment is three-dimensional investigated and a(3-D) bio-inspired neural network is proposed for application the wholebased hunting introduced. The proposed hunting to algorithm on process. By choosing the maximum activity value of the the bio-inspired neural network cells, is extended to theAUV 3-D neural network of neighbouring the hunting will direct movingthe target and finish case,select while athe basicpath ideato is the essentially same. In the 3the hunting task. The proposed approach can deal with D simulation experiment, the and hunting is also various situations automatically catchmap the moving target effectively. In addition, grid it canmap. dealSix with hunting selected as the discretization AUVs are tasks in the environment with different shapes of selected for theparameters dynamic hunting of an escapee. It should obstacles. The in the hunting experiment are decided by real-world However, from be noted that the complexapplications. current situation in the actual simulation results, it can be seen that sometimes there three-dimensional environment is not considered. will be a collision between AUVs. This indicates that the cooperative and collaborative mechanism among AUV team members is not built properly. Thus, further study In this simulation experiment, static obstacles are added will continue to focus on how to avoid collision be‐ tween hunting team inmembers and how to the 3-D map,AUV as shown Figure 12(a), wheretoa com‐ blue plete the hunting task in a 3-D environment under the cube represents an obstacle. The six AUVs are labelled as proposed method. In addition, a further important problem thefrom ocean PC1 , PC 2 ,that PC 3 ,needs PC 4 , PCto5 , be PC 6discussed and they is start thecurrent initial effect in the underwater environment. position

Int J Adv Robot Syst, 2015, 12:166 | doi: 10.5772/61555

{(1,1,0),(10,0,2),(10,10,10),(3,10,10),(1,10,0),(10,1,10)}.

The

AUVs approach the target according to the maximum neuron’s activity selection mechanism and perform

under static obstacles 6. Acknowledgements 5. Conclusion This project is supported by the National Natural Science Cooperative hunting by multi-AUVs in an unknown Foundation of China (51279098, 51575336, 61503239), environment is investigated and a bio-inspired neural Creative Activity Plan for Science and Technology Com‐ network is proposed for application to the whole hunting mission of Shanghai (14JC1402800). process. By choosing the maximum activity value of the neural network of neighbouring cells, the hunting AUV 7. References will select a direct path to the moving target and finish the [1] hunting TheJ.A. proposed can V. deal with P. A.task. Miller, Farrell, approach Y. Zhao and Djapic, various “Autonomous situations automatically catch Navigation,” the moving Underwaterand Vehicle target effectively. it can deal withNo.3, hunting IEEE JournalInof addition, Oceanic Engineering, Vol.35, pp. tasks in663-678, the environment with different shapes of 2010. obstacles. parameters in the P. hunting are Fiorelli, N.E. Leonard, Bhatta,experiment D. Paley, D.R. [2] E. The decidedBachmayer by real-world applications. However, from and D.M. Fratantoni, “Multi-AUV simulation results, it can besampling seen thatinsometimes there control and adaptive Monterey Bay,” will be aAutonomous collision between AUVs. This indicates that pp. the Underwater Vehicles, Vol.31, No.4, 935-948, cooperative and2004. collaborative mechanism among AUV team Thus, and further study R. Cui, is S. not Sambuilt Ge, properly. B. V. E. How Y. Choo, [3]members will continue to focus on how to avoidcontrol collisionofbetween “Leader–follower formation under autonomous vehicles,” Ocean huntingactuated AUV team membersunderwater and how to complete the Vol.37, No.17–18, under pp. 1491-1502, 2010. huntingEngineering, task in a 3-D environment the proposed method. addition, a further problem that S. Wang, Z. Wuimportant and Y. Wang, “Motion [4] Y.InYang, planning for multi-HUG formation in an environ‐ ment with obstacles,” Ocean Engineering, Vol.38, No. 17-18, pp. 2262-2269, 2011.

needs to underwa

6. Acknow This proj Foundati Creative Commiss

7. Referen [1] P.A “Au IEEE 663[2] E. F Bach cont Aut 935[3] R. C “Lea actu Eng [4] Y. Y plan env Vol. [5] M. Alg Aut Sini [6] J. L Coo Min Syst [7] M. Dist Alg Con Robo [8] R. R UAV Proc 2013 [9] S. Y Surv (AU Syst [10] M. P veh algo App [11] G. Z base Con [12] D. Z

[5] M. Liu, W. Li and P. Xuan, “Convex Optimization Algorithms for Cooperative Localization in Auton‐ omous Underwater Vehicles,” Acta Automatica Sinica, Vol.36, No.5, pp.704-710, 2010. [6] J. Li, Q. Pan, B. Hong and M. Li, “Multi-robot Cooperative Pursuit Based on Association Rule Data Mining,” Sixth International Conference on Fuzzy Systems and Knowledge Discovery, pp. 14-16, 2009. [7] M. Wu, F. Huang, L. Wang and J. Sun, “A Distrib‐ uted Multi-Robot Cooperative Hunting Algorithm Based on Limit-cycle,” International Asia Conference on Informatics in Control, Automation and Robotics, pp. 156-160, 2009. [8] R. R. McCune and G. R. Madey, “Swarm Control of UAVs for cooperative hunting with DDDAS,” Procedia Computer Science, Vol. 18, pp. 2537-2544, 2013. [9] S. Yoon and C. Qiao, “Cooperative Search and Survey Using Autonomous Underwater Vehicles (AUVs),” IEEE Transactions on Parallel and Distrib‐ uted Systems, Vol.22, No.3, pp. 364-379, 2011. [10] M. P. Aghababa, “3D path planning for underwater vehicles using five evolutionary optimization algorithms avoiding static and energetic obstacles,” Applied Ocean Research, Vol. 38, pp. 48-62, 2012. [11] G. Zhang and H. Jia, “3D path planning of AUV based on improved ant colony optimization,” China Control Conference (CCC), pp. 5017-5022, 2013. [12] D. Zhang, G. Xie, J. Yu and L. Wang, “An Adaptive task assignment for multiple mobile robots via swarm intelligence approach,” IEEE International Symposium on Computational Intelligence in Robotics and Automation, pp. 415-420, 2007. [13] Z. Liu, Q. Luo, D. Wen, S. Qiao, J. Shi and W. Zhang, “A Task Assignment Algorithm for Multiple Aerial Vehicles to Attack Targets With Dynamic Values,” IEEE Transactions on Intelligent Transportation Systems, Vol.14, No.1, pp. 236-248, 2013. [14] A. T. Tolmidis and L. Petrou, “Multi-objective optimization for dynamic task allocation in a multirobot system,” Engineering Applications of Artificial Intelligence, Vol.26, No. 5–6, pp. 1458-1468, 2013. [15] S. Hunt, Q. Meng, C. Hinde and T. Huang, “A consensus-based grouping algorithm for multiagent cooperative task allocation with complex requirements,” Cognitive Computation, Vol. 6, No. 3, pp. 338-350, 2014. [16] C. Grinton, “A Testbed for Investigation Agent Effectiveness in a Multi-agent Pursuit Game,” The University of Melbourne, 1996. [17] M. Z. Sauter, D. Shi and J. D. Kralik, “Multi-agent reinforcement learning and chimpanzee hunting,” IEEE International Conference on Robotics and Biomi‐ metics, pp. 622-626, 2009. [18] Z. Cai, L. Sun, H. Gao and P. Zhou, “Multi-robot Cooperative Pursuit Based on Combinatorial

Auction Mechanism Under Dynamic Environ‐ ment,” 2nd International Symposium on Systems and Control in Aerospace and Astronautics, pp.1-6, 2008. [19] M. K. Nighot, V. H. Patil and G. S. Mani, “Multirobot hunting based on Swarm Intelligence,” International Conference on Hybrid Intelligent Sys‐ tems, pp. 203-206, 2012. [20] S. Feng and T. Wang, “Implementation of control algorithm for interception and hunting by Amigo‐ Bot robots,” International Conference on Mechatronic Science, Electric Engineering and Computer (MEC), pp. 2429-2433, 2011. [21] Y. Sheng, A. H. Sayed, “Cooperative prey herding based on diffusion adaptation,” IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3752-3755, 2011. [22] H. Yamaguchi, “A distributed motion coordination strategy for multiple nonholonomic mobile robots in cooperative hunting operations,” Robotics and Autonomous Systems, Vol. 43, No. 4, pp. 257-282, 2003. [23] Y. Pan and D. Li, “Improvement with Joint Rewards on Multi-agent Cooperative Reinforcement Learn‐ ing,” International Conference on Computer Science and Software Engineering, pp. 536-539, 2008. [24] Y. Ma, Z. Cao, X. Dong, C. Zhou and M. Tan, “A multi-robot coordinated hunting strategy with dynamic alliance,” Control and Decision Conference, pp. 2338-2342, 2009. [25] C. Wang, T. Zhang, K. Wang, S. Lv and H. Ma, “A new approach of multi-robot cooperative pursuit,” China Control Conference (CCC), pp. 7252-7256, 2013. [26] J. Ni and S. X. Yang, “Bio-inspired Neural Network for Real-Time Cooperative Hunting by Multi-robots in Unknown Environments,” IEEE Transactions on Neural Networks, Vol.22, No.12, 2011. [27] A. L. Hodgkin and A. F. Huxley, “A quantitative description of membrane current and its application to conduction and excitation in nerve,” Journal of Physiology, Vol.117, No.4, pp. 500-544, 1952. [28] S. Grossberg, “Nonlinear neural networks: Princi‐ ples, mechanisms, and architecture,” Neural Networks, Vol. 1, No.1, pp. 17-61, 1988. [29] S. X. Yang and C. Luo, “A Neural Network Ap‐ proach to Complete Coverage Path Planning,” IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics, Vol.34, No.1, pp. 718-725, 2004. [30] R. Pichevar and J. Rouat, “Binding of audio ele‐ ments in the sound source segregation problem via a two-layered bio-inspired neural network,” IEEE Canadian Conference on Electrical and Computer Engineering, pp. 1151-1154, 2003. [31] M. Yan, D. Zhu, S. X. Yang, “Complete coverage path planning in an unknown underwater environ‐ ment based on D-S data fusion real-time map

Daqi Zhu, Ruofan Lv, Xiang Cao and Simon X. Yang: Multi-AUV Hunting Algorithm Based on Bio-inspired Neural Network in Unknown Environments

11

building,” International Journal of Distributed Sensor Networks, Vol.9, No.2, pp. 1-11, 2013.

Workspace,” IEEE Transactions on Cybernetics, Vol. 43, No.2, pp. 504-514, 2013.

[32] C. Luo and S. X. Yang, “A Bioinspired Neural Network for Real-Time Concurrent Map Building and Complete Coverage Robot Navigation in Unknown Environments,” IEEE Transactions on Neural Networks, Vol.19, No.7, pp. 1279-1298, 2008.

[34] N. V. Dounskaia, “Artificial potential method for control of constrained robot motion,” IEEE Transac‐ tions on Systems, Man, and Cybernetics—Part B: Cybernetics, Vol.28, No.3, pp. 447-453, 1998.

[33] D. Zhu, H. Huang, S. X. Yang, “Dynamic Task Assignment and path planning of Multi-AUV System Based on an Improved Self-organizing Map and Velocity Synthesis Method in 3D Underwater

12

Int J Adv Robot Syst, 2015, 12:166 | doi: 10.5772/61555

[35] E. P. Lopes, E. P. L. Aude, J. T. C. Silveira and H. Serdeira, “Obstacle avoidance strategy based on adaptive potential fields generated by an electronic stick,” IEEE/RSJ International Conference on Intelli‐ gent Robots and Systems, pp. 2626-2631, 2005.