THE GAME STRATEGIES OF SOCCERTEAM* Bora I. Kumova Dokuz Eylul University; Department of Computer. Engineering 35100 Izmir; Turkey
[email protected]; http://cs.deu.edu.tr/~kumova
Abstract Soccer game rules are necessary and sufficient to play correct Soccer and to win a match randomly, if no further strategies are applied. A soccer game consists of combinations of various type of re−occurring simple object movements, which must comply to the game rules. In order for a team to win a game, these allowed movements are combined to different types of strategies. When and how to apply a strategy depends on the related field constellations. In this work we present the game strategies that have been considered in the plan construction process within the CEVOP methodology, which includes a co−evolutionary algorithm that generates plans for offensive and defensive team strategies.
Keywords Distributed Artificial Intelligence; Knowledge Representation; Planning; Machine Learning; Game Theory 1. INTRODUCTION Since midd of the 1990s one focusing area for agent research and application is the Soccer domain. It involves a broad variety of techniques borrowed from artificial intelligence and related disciplines, ranging from software agents to robotics, sensing to behaviour, software architecture to electronics and mechanisms, etc. The simulation of a Soccer game and the success of a team requires a suitable combination of these techniques within a multi−agent system. One international forum for testing Soccer playing agents is the RoboCup simulation league [2], in which eleven team members co−operate whilst competing against another team of eleven members. The success of the teams is constantly increasing in each year’s competition. The goal set by the RoboCup initiative is: "By mid−21st century, a team of fully autonomous humanoid robot soccer players shall win the soccer game, comply with the official rule of the FIFA (Fédération Internationale de Football Association), against the winner of the most recent World Cup." The current state of the art however does not allow any comparison with this goal, yet. The capabilities of current simulation teams are still far behind of any human junior Soccer team. Out of the analysis of recent RoboCup competitions *
This paper was published at TAINN’02.
[1], [12] one can conclude that still much progress is required in all areas of technology used in the multi−agent teams. From the perspective of the architecture of the Soccer teams, various approaches are proposed [3], [9], [10], [11], [13]. With respect to the movements of the game objects, different abstraction levels can be constructed (Figure 1). Physical rules, such as location, gravity, momentum, and acceleration and most of the official FIFA game rules are implemented in the RoboCup Soccer simulation server [2]. Individual rules are applied by each agent, without considering the other team members. Whereas, team rules include all individual rules plus some further rules for co−operation within the team.
Team Rules Individual Rules Game Rules Physical Rules
... ...
... ...
Figure 1 Figure 1: Object Movements Performed At Different Abstraction Levels
In earlier work, we have designed the AgentTeam framework for multi−agent systems and prototype implemented some of its features in the distributed database management domain [4], [5], [6], [7]. In a recent work, we have refined the framework by the Co−evolutionary Planning (CEVOP) methodology and implemented related features for the Soccer game domain in the prototype SoccerTeam [8]. Principally, the CEVOP methodology maps from the multi−agent domain, co−operation/competition structures and homogeneity/ heterogeneity, onto the co− evolutionary domain and produces for a few given primitive game strategies, combined game strategies to be stored in the knowledge base of each SoccerTeam agent. In this paper, we explore all game strategies that were considered for and in the planning phase through the CEVOP methodology and the supporting strategies, which are additionally designed for SoccerTeam, but which were not considered in CEVOP. In other words, the CEVOP methodology seeks for team rules, for given game rules and individual rules. Physical rules are not considered at these abstraction levels. On the other hand, SoccerTeam agents apply these team rules together with some supporting individual rules that where not considered in CEVOP. Additionally, all physical rules are taken in account by a SoccerTeam agents, while applying game rules, individual rules, and team rules. In the following we will discuss first the static game strategies thereafter the dynamic once. Where, the latter is separated into primitive game strategies, combined game strategies, and coache’s game strategies. Finally, we will summarise the results. In this text we use the terms rule, strategy, and plan interchangeably depending on the perspective of the related discussion.
2. STATIC GAME STRATEGIES Based on the official Soccer field lines and areas, we have identified some further lines and areas on the field, which appear to have strategic significance for a match. We will refer to this geometry as the static game strategies, since it never changes over time. Strategic Areas In human soccer games one can observe that the teams usually follow some common strategies that depend on specific field locations, such as focusing area, major acting areas, major attack direction, and goal shoot areas (Figure 2). Focusing area: For a team to implement its strategies successfully, its members should be located strategically around the ball. To represent this information, we calculate the location of each agent relative to the location of the ball: (∆x, ∆y). Major acting areas: Back field (Fb), middle field (Fm), and front field (Ff) are the major acting areas on the Soccer field. Before each match the coach decides which players and how many players to locate in these areas. Major attack direction: This direction is clearly towards the opponent’s goal (G). It serves the team as orientation to which an attack should be planned.
SA
SA
G
G
Back/Front Field (Fb) Fb/Fm/Ff
Middle Field (Fm)
Front/Back Field (Ff)
Shoot Area (SA) Goal (G) Figure 2 Figure 2: Strategic Lines And Areas Learned By SoccerTeam
Goal shoot areas: These are the areas (S) in which statistically most of the goals are shot. They incapsulate the official goal areas. Each team’s shoot area is located immediately in front of the opponents goal and each team’s defence area is at the home side.
Learned Distance Granularity The selection of the grid line granularity of the field requires a compromise to be made between faster learning of rules that are more location insensitive and slower learning of rules that are more location sensitive. As result of our experiments, we have defined one Meter grid lines, which divide the field into some 60x100 squares. Since the planning process within CEVOP is affected by the grid lines, the found game strategies’ sensitivity is of this granularity. 3. PRIMITIVE GAME STRATEGIES Besides the static game strategies we define now some further game strategies, which we call dynamic, since they can be applied, depending on the constellation, over variable distances and at any location on the field. A primitive game strategy consists of the strategy type S and the relative distance ∆x, ∆y, to be applied on a specific game constellation (Table 1). Table 1 Table 1. Structure Of A Primitive Game Strategy S
∆x
∆y
The following strategies are designed for individual application, which may be decided by an agent autonomously. Below they will be combined with each other to construct combined strategies and team strategies. Primitive Defensive Game Strategies For All Players Applying only defensive strategies always successfully could at least prevent the opponent from shooting goals. Therefore, following strategies are essential. (1) Cover goal: This strategy positions the agent somewhere in between the ball and the own goal, so that the player becomes a handicap for attacking opponents. Goal: Move the agent in between the ball and the goal and let it cover the nearest opponent. Rule: IF the opponent has the ball Ob AND is in the shoat area SG THEN cover the closest opponent Oi, IF Oi is not already covering by another agent. (2) Attack: This strategy enables the defensive team to try to get the ball from the opponent. Goal: Move the closest agent to the opponent with the ball and try to get the ball. Rule: IF the agent Di is the closest to Ob THEN move Di to the location of Ob.
Primitive Offensive Game Strategies For All Players The topmost goal of a team is to win the game. For this purpose, offensive strategies enable it to shoot goals. The intention of each following strategy is to move the ball from its current position to a more advantages position. (3) Move: This strategy moves the agent from its current location to another location on the field by the distance (∆x, ∆y), where ∆x, ∆y = [0..20] Meter. It has following second semantic: if the agent has the ball, then both, agent and ball, are moved. Goal: Move the agent/ball from its current location to a more strategical location, relative to the ball. Rule: IF the desired direction is clear THEN move the agent by the distance (∆x, ∆y). (4) Pass: Passing the ball to another team member is the basic co−operative strategy. This strategy is relevant for the offensive agent who has the ball (Ob) in order to decide which team member (Oi) to select as next Ob, to which the ball will be passed. Goal: Pass the ball to the closest team member or to a strategically more advantages positioned team member. Rule: IF Ob AND Oi is at the most strategical position THEN kick the ball to Oi. Primitive Defensive Game Strategy For Players (5) Cover: Covering opponent players can prevent the attacking player from co− operating with them and therefore can establish only a weaker attack. This rule is similar to covering the goal, except that in this case the player is not in the shoat area, but must cover one opponent, and does not need to be located between the ball and the home goal. Primitive Offensive Game Strategy For Players (6) Shoot into goal: A team can win a match only, if it shoots at least one more goal than the opponent team. This strategy enables to shoot a goal. Goal: Shoot into the opponent’s goal. Rule: IF Ob is in the shoot area SG THEN shoot into the goal. In a more detailed analysis of these rules, one may find various interesting special cases, which we cannot discuss here. For instance the move strategy for (∆x, ∆y) = (0, 0). To fill−up some time by not moving can be a tricky strategy. It has important advantages, such as saving the player’s stamina, waiting for a better match constellation, or initiating the covering opponent agent to disassociate. Each of these primitive game strategies can be applied in the context of different specific game strategies for different purposes, some of which are introduced below. Initial Primitive Game Strategies According the Soccer game rules, following special game situations are defined: kick−off, free−kick, corner−kick, penalty, and touch. We do not need to design special strategies for them, since they can be covered by the Move strategy, however with an appropriate game constellation that complies with the game rules for the specific game situation.
Goalkeeper Strategies Additionally to the capabilities of the other players, the goalkeeper can catch the ball with the hands. Since this capability is restricted to the penalty area, a goalkeeper can play more effectively inside this area. Usually, a goalkeeper does not shoot a goal or is not directly involved in attack or defence strategies at the front. The goalkeeper becomes a major figure in those strategies, which are applied near the home goal. Following strategy can be applied as defensive as well as offensive action: (7) Catch the ball: Successfully catching the ball within the penalty area stops the attack of the opponent and prevents the ball from moving into the goal. Goal: Within the penalty area, catch the ball with the hands. Rule: IF the ball is in the penalty area THEN catch the ball. Supporting Primitive Game Strategies These strategies were not considered in the planning process with CEVOP, since they aim at changing the position and strategic situation of an agent, in order to enable the application of a suitable co−operative strategy. Sometimes they are called low level rules, since they can be applied in conjunction with any primitive strategy. General rules that can be applied in any situation: Kick the ball, if the agent is sufficiently near to the ball to hit it. Pass the ball in front of an agent, but in the direction of its move, so that both will meet at a certain point. In case of defensive situations and not valid for the goal keeper: Find the closest opponent, which is not covered, yet. Get ball from an opponent player. In case of offensive situations: Get rid of the opponent’s covering, to enable a pass. Move to a more advantages position, to enable a pass or a shoot to the goal. Only for the goal keeper allowed strategies: Carry the ball for a short distance. Since the current Soccer simulation server [2] supports only two−dimensional movements, no three−dimensional supporting strategies are designed for this version, such as header, shoulder, jumping, kicking the ball high, etc. 4. COMBINED GAME STRATEGIES The CEVOP methodology is a co−evolutionary planning approach [8], in which the algorithm, for a given game constellation, seeks for a team strategy that promises the greatest success. A team strategy consists of one combined game strategy for each of the 22 agents. Each combined game strategy consists further of three successive primitive game strategies (Table 2). Table 2 Table 2. Structure Of A Combined Game Strategy S1,1
∆x1,1
∆y1,1
S1,2
∆x1,2
∆y1,2
S1,3
∆x1,3
∆y1,3
Candidate Primitive Game Strategies It is an important design issue for the co−evolutionary algorithm of CEVOP that each primitive game strategy is independent from the others. This property enables the algorithm to explore any possible combination of the strategies, in order to find the best plan. In total there are 7 different primitive game strategies, which are used to construct a combined game strategy. Another constraint is that it can be constructed of either defensive or offensive primitive game strategies either player or goal keeper related primitive game strategies. Interpretation Of The Strategy Sequence The reason why we have chosen three primitives for a combined game strategy is explained as follows: 1. Primitive game strategy: the filed constellation is chosen randomly therefore, the agents are not necessarily at strategic positions relative to the ball therefore, this strategy is less realistic 2./3. Primitive game strategy: the field constellation is calculated according each agent’s first/second primitive strategy therefore, the agents are definitely at strategic positions relative to the ball therefore, this strategy is more realistic Looking ahead further steps would become less realistic for each next strategy, since the field constellation can change dramatically with each step. In this sense, even the third strategy is less realistic than the second one. On the other hand, exploring the third strategy brings the advantage that more complicated solutions can be found. For instance, depending on the current field constellation, it may be sometimes suitable to pass the ball backward, in order to continue the attack from a more advantages position. Compared for instance with Chess, Soccer has by far a larger number of possible combinations for each step, since all eleven agents can move simultaneously and with continues variable distances, and therefore the number of apriori planned steps is significant only for the first few once. Knowledge Base Structure As already mentioned above, a team strategy is a team plan proposed by CEVOP for a specific game constellation, consists of eleven combined game strategies, one for each agent. The selection criteria are based on the fitness of a strategy for the current situation [8]. The constellation, for which a team strategy was calculated is stored in each agent’s knowledge base. The related team strategy however, is not stored as a team strategy in the knowledge base. Instead, all of its primitive game strategies are stored independent from each other. Thus, each agent can decide autonomously which strategy to apply in the current constellation. With this approach, even more possible field constellations can be covered than it was calculated by CEVOP. A clear advantage is that, for the purpose of strategy selection, no explicit communication within the team is necessary and thus the decision process of each agent becomes more flexible.
In terms of knowledge representation, the team’s primitive and combined game strategies represent meta knowledge over Soccer game rules. Which however does not explicitly need to be stored in the knowledge bases, as just mentioned. 5. COACH STRATEGIES In human Soccer games, prior to a match, in brakes, or dynamically throughout a match, meta strategies are determined by the coach, in order to adapt his team to the opponent’s capabilities and the current match situation to be considered in all individual and team strategies. These strategies remain relative stable and can be disregarded only, when temporarily another strategy appears to be more promising, which may be determined by each player. Meta strategies are dependent on the overall match situation, such as the current score, the team’s general or current strength or weakness against the opponent, or any other situation that requires a global solution for the team. To represent this knowledge, we have design team distribution strategies for SoccerTeam. Team Distribution Strategies Depending on the location of the ball and whether the team is currently defending or offending, the distribution of the players over the field is a general approach for co−operation. In SoccerTeam the distribution of the team over the major acting areas is represented in form of the triple (Fb, Fm, Ff). The distributions are currently implemented in the range F = {(8, 2, 0), ..., (1, 2, 7)}, depending on defensive or offensive game, respectively. Defensive team distribution: (Fb, Fm, Ff) = (8, 2, 0) Offensive team distribution: (Fb, Fm, Ff) = (1, 2, 7) Sample scenarios that demonstrate the significance of coach strategies are given below. Back field attack scenario: If the ball is in the back field and one of the own team members gets the ball, then most of the team members should move towards the opponent goal. Goal: Most of the team members should be distributed over the middle and front fields, in order to support the attacking team member. Rule: IF ball is in Fb AND Ob THEN (Fb, Fm, Ff) = (Fb−−, Fm++, Ff++) Where, −− indicates a decrease of the value and ++ an increase. Front field retreating scenario: If the ball is in the front field and the opponent team gets the ball, then most of the team members should move towards the own goal. Goal:Most of the team members should be distributed over the back and middle fields, in order to clear an attack. Rule: IF ball is in Ff AND D THEN (Fb, Fm, Ff) = (Fb++, Fm+−, Ff−−) Where, +− indicates that the value first may increase than decrease, depending on whether the team members will stay in the related field whilst this strategy is applied. Duration Of The Strategies Each coach strategy is valid for a specific situation or a group of situations. After the situation has changed, the coach re−capitulates it and modifies the
distribution range. This range is modified at strategic times, in order to adapt the team to the current overall match situation. Such strategic times are: Before a match. Before the second half time of a match. Before an initial game constellation. At a time, when any other change of the global match situation is encountered; e.g. a clear superiority or inferiority of a team. These meta strategies are not fixed by the coach at each step of a match. For that purpose individual strategies are more effective, since they adapt the team to the current situation in a microscopic sense. Whereas, meta strategies act in a macroscopic sense. It suffices that the team acts mostly within the given range of distribution. Besides the strategic value of such coach rules, another major advantage is the saving of stamina of each player. This is guaranteed by assigning each player to one of the major acting areas. 6. CONCLUSIONS We have introduced all important game strategies of SoccerTeam. Because of place limitation, we could only sketch them. For more details, the reader is referred to [8], [9]. The strategies are sensitive to specific field constellations, but are independent from absolute field co−ordinates, since the distances are measured relative to the ball location. Furthermore, the strategies are independent form the third dimension, which could be implemented by further supporting strategies, if desired. We are still experimenting with sample CEVOP results, in order to improve the co−evolutionary algorithm and to get more significant team rules. For instance, by testing the significance of a fourth step within a combined game strategy. At the side of the SoccerTeam prototype, a more sophisticated implementation of the physical rules is the next development step. Some test results of CEVOP are available at the authors Web address. Final results on SoccerTeam will be available by end of 2002. Acknowledgement I am gratefully to Akira Imada for contributing to this work indirectly. While he was co−advising my PhD thesis by helping me to design and improve the co− evolutionary algorithm, his suggestions implied some of the dynamic strategies. References Referenced URLs were valid by 08. May, 2002. 1. Asada, Minoru; Kitano, Hiroaki; Noda, Itsuki; Veloso, Manuela; 1999; "RoboCup: today and tomorrow − what we have learned"; Artificial Intelligence; Elsevier 2. Chen, Mao; Foroughi, Ehsan; Heintz, Fredrik; Zahn, Xiangh Huang; Kapetanakis, Spiros; Kostiadis, Kostas; Kummeneje, Johan; Noda, Itsuki; Obst, Oliver; Riley, Pat; Steffens, Timo; Wang, Yi; Yin; Xiang; 2001; "Robo Cup Soccer Server"; R oboCup Federa ti on ; http:// www.robocup.org/
3. Jung, Christoph G.; 1999; "Layered and Resource−Adapting Agents in the Ro boCup Simulation"; LNCS ; Springer 4. Kumova, Bora I; Kut, Alp; 1998; "A Communication Model for Distributed Agents"; ISCIS’98; IOS Press 5. Kumova, Bora I.; 1999; "The Agent Model of AgentTeam"; TAINN’99; Bogazici University 6. Kumova, Bora I.; Kut, Alp; 1999; "Agent−based System Co−operation"; PDP’99; IEEE Computer Society 7. Kumova, Bora I.; 2000; "Software Design Patterns for Intelligent Agents"; SSGRR’00, lAquila 8. Kumova, Bora I.; 2002; "Co−evolutionary Planning"; PhD thesis; Dokuz Eylul University 9. Lubbers, Jan; Spaans, Rogier R.; 1999; "The Priority/Confidence Model as a Framework for Soccer Agents"; L NCS; Springer 10. Shen, Wei−Min; Adibi, Jafar; Adobbati, Rogelio; Lanksham, Srini; Moradi, Hadi; Salemi, Behnam; Tejada, Sheila; 1999; "Integrated Reactive Soccer Agents"; L NCS; Springer 11. Tambe, Milind; Adibi, Jafar; Al−Onaizan, Yaser;Erdem, Ali; Kaminka, Gal A.; Marsella, Stacy C.; Muslea, Ion; "Building agent teams using an explicit teamwork model and learning"; Artificial Intelligence; Elsevier 12. Tanaka−Ishii, Kumiko; Frank, Ian; Arai, Katsuto; 2000; "Trying to Understand RoboCup"; AI Magazine; AAAI 13. Werger, Barry Brian; 1999; "Cooperation without deliberation: a minimal behavior−based approach to multi−robot teams"; Artificial Intelligence; Elsevier
