A Microcell Oriented Load Balancing Model for ... - Semantic Scholar

A Microcell Oriented Load Balancing Model for Collaborative Virtual Environments Dewan Tanvir Ahmed, Shervin Shirmohammadi Distributed and Collaborative Virtual Environments Research Laboratory School of Information Technology and Engineering University of Ottawa, Canada E–mail: {dahmed, shervin}@discover.uottawa.ca.

Abstract – In this paper, we present a microcell oriented load balancing mechanism in the context of a hybrid MMOG (massively multiplayer online game) architecture. The objective of the proposed approach is to define a fair load balancing model for peer-to-peer MMOGs and virtual environments that minimizes peer-to-peer distortions in the overlays. In the case of an overloaded server, the proposed approach identifies appropriate microcell(s) that reduces overlay maintenance cost and minimizes inter-server communications. The load is relieved by devolving these microcells to the other less loaded servers. The model is justified through extensive simulation. Keywords – Area of interest, hotspot, load Balancing, MMOG

I. INTRODUCTION Since the experience of Networked Virtual Environments (NVE) in 1980s for military simulation, many other similar applications have been introduced in this domain over the decades. Massively Multiplayer Online Game (MMOG) is a kind of NVE where many players interact simultaneously. Currently, Second Life and World of Warcraft are the two most popular NVEs. Traditionally, commercial NVEs practice the client-server architecture which is expensive to deploy and maintain. For example, Second Life has approximately 5000 servers to support its virtual world (i.e. game world). Such huge deployment cost as well as regular maintenance demand suggests a need for an alternative solution. The P2P architecture can be a good solution in terms of scalability, but considering the business prospect and the quality of gaming experience, it reveals that the pure P2P architecture is an infeasible solution. Recently, the hybrid P2P architecture is considered as an ideal solution both for the vendors and end-users [1][2][3]. Crowding, a large group of people gathered in one place, is a natural phenomenon in MMOGs. The load-balancing algorithms partition the players’ data and workload among the servers, i.e. the masters. The effective load division greatly influences the performance of the game. The decompositions that balance the masters’ loads and have a low inter-server communication costs are preferred. Although a wide variety of load-balancing algorithms have been developed, their effectiveness depends on the characteristics of the application using

them. We discuss current efforts toward improving partitioning algorithms for network game applications and architectures. Numerous partitioning strategies have been developed. The strategies are distinguished by different trade-offs like partition quality, amount of data movement, etc. The characteristics of an application (e.g., computation-to-communication ratio, cost of data movement, and frequency of repartitioning) determine which strategies are most appropriate. For example, the geometric algorithms like recursive bisection partitioning provide high-speed, medium-quality decompositions that depend only on geometric information. The graph-based algorithms provide higher quality decompositions based on communication cost among the partitions, but at a higher cost. At its simplest, a load-balancing algorithm attempts to distribute equal numbers of players to partitions while minimizing inter-partition communication costs. A partition’s sub-zone consists of the data exclusively assigned to the sub-zone. The union of sub-zones is equal to the entire problem domain. The workloads in a continuously changing system evolve in time, so a partitioning policy that works well for a static system or for a slowly-changing environment may not be effective in a highly dynamic application. In finite element methods, for example, with adaptive mesh refinement, the process workloads can vary singnificantly as elements are added and/or removed from the mesh. The dynamic repartitioning of mesh data often called dynamic load balancing becomes necessary. The zoning decomposes the game world into disjoint regions, i.e. zones (here we call microcell), that gives a provision of easier state management. Each zone is coordinated by a server that performs the statement management with the active participation of the players; hence, it is a hybrid peer-to-peer system: there is a server that communicates to clients, and also clients communicate among each other. A zonal MMOG architecture must cover many aspects: zone organization, zonal communication strategy, interest management, seamless zone crossing, load balancing and many others [2]-[4]. In this research, we are only covering load balancing for collaborative virtual environments. This paper presents a microcell oriented load balancing mechanism for the servers while minimizing peer-to-peer distortions in the overlays. The road map of the paper is as fol-

lows. We study the nature of load balancing and related works in Section II. The microcell oriented load balancing approach is presented in Section III. The simulation results and corresponding analysis are covered in Section IV. Finally, we conclude the paper in Section V with future research directions. II. LITERATURE REVIEW Generally, each user in a NVE is only interested in a small portion of the world where most of the interactions take place. This interaction space is called Area of Interest (AOI). In MMOG, it is possible that many players move into a zone causing the degradation of the game’s quality and affecting players’ gaming experience. This problem is known as the hotspot problem. Chen et al. mention that flocking (i.e. hotspot or too many players moving into a zone administered by a server) is a natural phenomenon in MMOGs that cannot be avoided [5]. This is logical as some areas are more interesting than others in the game world. Moreover, this hotspot is unpredictable as it depends on game events like players chasing each other or performing a mission. Chen et al. also mention that in most cases, the static partitioning handles it poorly [5]. A preventive mechanism for migration is to impose boundaries between zones: cities are separated by forests or wilderness, where parties do not tend to stay for long [6]. Thus, a migrating player does not experience others’ interactions at the boundaries. But such restrictions cannot always be imposed, and even if applied, they do not avoid hotpots in ”battle” situations where everyone must be in the same zone facing the enemy. Takuji et al. propose a zoned federation model to adjust multiplayer online game to peer-to-peer network [10]. This approach inserts a zoning layer between the game and the peerto-peer network. A zone is a small space of a game world, and a zone owner is an authoritative server of a specific zone. Here each node changes its role to a zone owner and works like a centralized authoritative server. The workloads of the authoritative centralized game server are distributed to a federation of nodes by dividing the whole game world into zones. A player keeps connection to the authoritative node in order to receive latest updates directly and to request for modified data. The authoritative node for a particular global state is discovered by DHT (distributed hash table). In other words, a zoning layer confines the update request and notification within a single zone, and the zoning layer depends on DHT layer for data storage and rendezvous. The big question is the feasibility of carrying out complete server functionalities through a peer node. One of the important design issues is that subspaces overlapped with each player’s view must be switched quickly and seamlessly as the view moves around the game space. The zone federation approach does not propose techniques for achieving this in detail, and the concept of a data holder causes high latencies. It also does not handle the situation if the zone owner dies. Overall, the complexity of the proposed architecture makes it difficult to deploy.

Lu et al. present some techniques required for load balancing [3]. An ideal solution ensures equal load distribution to prevent exhausting one server while keeping spare resources at other servers. Theoretically, it is a good solution as it is ideal and does not disrupt players’ gaming experience. But assessing the load of the servers, frequently migrating players from one sever to another, and also tracking the area of interest seem to be a big problem in this regard. This would be even more critical for hybrid systems where the players have routing responsibility. According to Duong et al., the dynamic load distribution methods can be categorized into load sharing algorithms and load balancing algorithms [7]. The load sharing algorithms prevent unbalanced load among servers. The load balancing algorithms address the situation by equalizing the workload among all servers in a distributed manner. The computational overhead associated with load balancing reduces algorithms’ effectiveness [8], and subsequently the gaming experiences. In this paper, we present a load balancing model for peerto-peer MMOGs. In the case of an overloaded server, i.e. the master, the proposed approach determines one or more suitable microcells (a small region in the virtual space) of that loaded server. The microcells are chosen to reduce overlay maintenance cost and to minimize inter-server communications. The load is reduced by devolving those microcells to the other less loaded servers. III. THE MICROCELL ORIENTED LOAD BALANCING MODEL FOR HYBRID MMOG In this model, a set of microcells defines the virtual game world that is maintained and coordinated by a set of servers called server pool. The server pool having n servers is represented by a set S = {S1 , S2 , ..., Sn }. Each server, e.g. Si , serves one or more microcells depending on load distribution and avatars’ population in the corresponding microcells. The microcells are given unique identification numbers: left to right, top to bottom in order. In zonal MMOGs, the game world is usually divided into several manageable regions like zones. The zone concept is not entirely new and has been considered over the last few years. The hexagonal shape has a couple of advantages over others like circular [4], hence considered in this design to define the shape of a microcell. Each microcell is run by a server called master: a special node that coordinates the interactions in a collaborative manner with the active participation of the microcell members, i.e. avatars. So, a set of master nodes, i.e. the server pool, regulates the operation of the MMOG and provides overlay services. In that sense, the system is hybrid as it combines the benefits of both centralized and distributed systems. The layout of microcells and a random microcell distribution to servers are shown in Figure 1.

Fig. 1. The layout of microcells and a random microcell distribution to servers

A. Load definition In this paper, we are dealing with hotspots. So, the implied resolution in this regard is that the server pool can handle the total system load. As described before, a hotspot refers to a crowded region in a zonal MMOG, leading to the need for load balancing. But, the key question is how to figure out a hotspot. Most of the approaches like [2] consider the total number of players in a zone/microcell as an identifying attribute. If the total number of players in a microcell exceeds the maximum affordable limit, it is a hotspot. Logically, it is correct. But, here we look at it in a different way. In MMOG, there are many players like soldiers, robots, aircrafts, etc with various features. In addition, the actions that actually take place in a game have an influence on message generation rate. For example, in spite of the regular updates from a player, there may be an immediate message due to a trigger in a pistol by an avatar. Every message requires reaching to every other interested player in the interested region which eventually increases the load of the master. So, the load is not completely related to the number of players, actually, it is purely linked with the message generation rate. Thus, the message generation rate, Rm , is used to estimate the load of the masters. B. Marking of a loaded server Say, the server, Si , is in charge of a set of microcells, Mi = {mi1 , mi2 , ..., mik }. Let Rim = {ri1 , ri2 , ..., rik } are the message generation rate of the microcells {mi1 , mi2 , ..., mik } respectively where all of them are governed by server Si . Thus, the load of the server Si according to message generation rate is: Pk Rim = j=1 rij Let us define a message threshold term T m . Thus, a server is loaded or there is a hotspot if Rim > T m . C. An approach to load balance When there is a hotspot, the load of the concerning server is relieved by moving one or more microcells to other less loaded

Fig. 2. The buffer region and microcell covering

servers. As the movement of microcell introduces complexities, some intelligence is required to identify the best possible set of microcells that need to be moved. The reduced deployment and maintenance overhead of the hybrid MMOGs (peers or avatars perform some of the message relaying tasks) come at the cost of performance due to peer dynamics. Many techniques have been proposed in literatures such as entity typing and the smart interest driven zone crossing with buffer region [9] to keep the desired level of performance. As it is difficult to predict avatars’ movement around a cell boundary (possible frequent in and out movements), it could be effective to share the relevant state information between two adjacent microcells. Instead of sharing the complete state information, a buffer region is defined between two adjacent microcells, as shown in Figure 2. This greatly reduces exchanging of unimportant messages. But, if two adjacent microcells are handled by a single master, there is no buffer region in between, as shown in Figure 2. According to that figure, there is a buffer region between Microcell 2-4 and Microcell 3-3 as they are administered by two different servers but Microcell 2-4 and Microcell 2-3 have a common master and hence no buffer region in between. The objective of changing a microcell ownership from one master to another is to approach hotspot. But, the key point that must be taken care of is to minimize inter-server communication cost while carrying out such microcell ownership alteration. Let, Ii is the number of interested links completely

internal to the Microcell i among the avatars and Bi,j is the number of interested links involved between Microcell i and Microcell j in the buffer region. Let, the Server Si is overloaded and currently serving k microcells: Mi = {mi1 , mi2 , ..., mik }. Our target is to determine an ordered list of microcells so that the microcell ownership alteration according to that order will introduce less performance penalty in the overlays of hybrid MMOGs. The algorithm initially forms group of microcells based on the microcell’s neighborhood properties. A microcell can be a member of a nonempty group if and only if it has a common edge with any microcells of that group. Let, after the application of this grouping policy, Server Si has l groups Gi1 , Gi2 , ..., Gil in order of their cardinality, i.e. |Gi1 | ≤ |Gi2 | ≤ ... ≤ |Gil |. The general strategy is to find a microcell within a group with a minimum number of interested links among the microcells of that group. Due to the change of ownership, these links become foreign links that will cause inter-server communication. Thus, a group with more microcells may have more common edges than a small group, at least heuristically. Thus, it can be effective to keep those groups intact. So, a group with a fewer microcells is chosen for load balancing and that is why we start with the group Gi1 , then Gi2 and so on, if necessary. Finally within a group, a microcell with a minimum number of potential foreign links is considered for ownership alteration.

Fig. 3. MMORPG client traffic from ShenZhu: exponential distribution, mean packet inter-arrival time 550ms

IV. SIMULATION AND ANALYSIS We have carried out detailed simulation to verify the proposed model and to evaluate the performance. The focused MMOG genres are MMORPG (Massively Multi-player Online Role Playing Game) and FPS (First Person Shooter). We have chosen the realistic traffic for these two types of game due to their varying characteristics, as explained below. In some sense, MMORPG and FPS games are similar because of their low bandwidth requirement. Compared to FPS, the bandwidth requirement of MMORPG is even lower due to its strategic nature. In addition, the game traffic of an MMORPG has strong periodicity but usually follows temporal locality. The traffic pattern of a client is a composite function of different factors for the chosen game. Most of the cases, the pattern follows the exponential distribution, e.g. ShenZhu online [10]. The packet interarrival time is in between 0ms to 600ms. In our simulation, each player’s traffic is modeled using exponential distribution and the packet interarrival time was set to 550ms as measured in other works like in [10]. It should be noted that, the client packet interarrival time is independent on the number of gamers [11]. The most computationally expensive part of the cycle in FPS like Quake is rendering. It causes the slower hosts to have significantly higher and more variable interarrival times, while the fastest hosts transmit most packets at 14ms intervals [12][13]. Thus, the client traffic can be modeled as one extreme [13] or two extreme distributions [14]. We considered one extreme distribution function. The client traffic

Fig. 4. Quake client traffic: extreme distribution, a = 50, b = 10

of ShenZhu and Quake are shown in Figure 3 and Figure 4, respectively, for an hour long simulation in two scales: w.r.t. second (the top one) and w.r.t. minute (the bottom one). From this setup it reveals that on an average each client generates more traffic in FPS compared to MMORPG. The inbound traffic of a server depends on the number of microcells administered and the number of players in each microcell. The summarized simulation parameters for MMORPG are given in Table I. As explained earlier, we have considered packet counts to define load. The traffic volume of a microcell is shown in Figure 5 for 20 players according the distribution mentioned earlier. The aggregate server traffic from eight microcells are shown in Figure 6 which has an average of 293.52 packet/sec with a variance of 329.37. The effectiveness of the presented microcell oriented load

TABLE I. The simulation setup parameters

Parameter

Values

Simulation platform

Microsoft Windows XP Professional (SP2)

Tools used

Matlab

Run time

3600 sec

Number of servers

100

Number of microcells

1000

Averger number of players in each microcell

15 to 30

Server’s average capacity

8 microcells

Fig. 5. The traffic volume of a microcell in MMORPG (The size of a microcell: 20)

Fig. 6. The Aggregate server traffic of eight microcells in MMORPG

managing model is justified thorough Figure 7 and Figure 8. For an hour long simulation with identical setup, we observed system load at every minute. Figure 7 shows the standard deviation of packets being processed by all servers of an MMORPG system. It clearly shows the load imbalance of the servers. The significance of the microcell based MMOG model is the balanced load of the deployed servers. From Figure 8, it reveals that the standard deviation of packets being handled by the servers has dropped to a large extend. V. CONCLUSION In this paper, we have demonstrated a microcell oriented load balancing mechanism for peer-to-peer MMOGs. The object of the proposed approach is to discover one or more microcells of a loaded server in terms of low movement cost. In the case of an overloaded server, the proposed approach identifies appropriate microcell(s) that reduces overlay maintenance cost and minimizes inter-server communications. The load is reduced by devolving these microcells to the other less loaded

Fig. 7. The standard deviation of the processed packets by the servers overtime

[10] K. Chen, P. Huang, C. Huang, and C. Lei, “Game traffic analysis: an mmorpg perspective,” in NOSSDAV ’05: Proceedings of the international workshop on Network and operating systems support for digital audio and video, New York, NY, USA, 2005, pp. 19–24, ACM. [11] R.A. Bangun, E. Dutkiewicz, and G.J. Anido, “An analysis of multiplayer network games traffic,” Multimedia Signal Processing, 1999 IEEE 3rd Workshop on, pp. 3–8, 1999. [12] T. Lang, P. Branch, and G. Armitage, “A synthetic traffic model for quake3,” in ACE ’04: Proceedings of the 2004 ACM SIGCHI International Conference on Advances in computer entertainment technology, New York, NY, USA, 2004, pp. 233–238, ACM. [13] M.S. Borella, “Source models of network game traffic,” 2000. [14] C. Chambers, W. Feng, S. Sahu, and D. Saha, “Measurement-based characterization of a collection of on-line games,” in IMC ’05: Proceedings of the 5th ACM SIGCOMM conference on Internet measurement, New York, NY, USA, 2005, pp. 1–14, ACM.

Fig. 8. The standard deviation of the processed packets by the servers overtime using microcell model

servers. VI. REFERENCES [1] A. Yu and S. T. Vuong, “MOPAR: a mobile peer-to-peer overlay rrchitecture for interest management of massively multiplayer online games,” in NOSSDAV ’05: Proceedings of the international workshop on Network and operating systems support for digital audio and video, New York, NY, USA, 2005, pp. 99–104, ACM Press. [2] I. Kazem, D.T. Ahmed, and S. Shirmohammadi, “A visibility-driven approach to managing interest in collaborative virtual environments with dynamic load balancing,” in Proceedings of IEEE International Symposium on Distributed Simulation and Real Time Applications (DS-RT, 2007. [3] T. Iimura, H. Hazeyama, and Y. Kadobayashi, “Zoned federation of game servers: a peer-to-peer approach to scalable multi-player online games,” in NetGames ’04: Proceedings of 3rd ACM SIGCOMM workshop on Network and system support for games, New York, NY, USA, 2004, pp. 116–120, ACM. [4] D.T. Ahmed, S. Shirmohammadi, and I. Kazem, “Zone based messaging in collaborative virtual environments,” in IEEE International Workshop on Haptic Audio Visual Environments and their Applications (HAVE), 2006, pp. 165–170. [5] J. Chen, B. Wu, M. Delap, B. Knutsson, H. Lu, and C. Amza, “Locality aware dynamic load management for massively multiplayer games,” in PPoPP ’05: Proceedings of the tenth ACM SIGPLAN symposium on Principles and practice of parallel programming, New York, NY, USA, 2005, pp. 289–300, ACM Press. [6] B. BDe Vleeschauwer, B. Van Den Bossche, T. Verdickt, F. De Turck, B. Dhoedt, and P. Demeester, “Dynamic microcell assignment for massively multiplayer online gaming,” in NetGames ’05: Proceedings of 4th ACM SIGCOMM workshop on Network and system support for games, New York, NY, USA, 2005, pp. 1–7, ACM. [7] T.N. Binh Duong and S. Zhou, “A dynamic load sharing algorithm for massively multiplayer online games,” in ICON ’03: Proceedings of 11th IEEE International Conference on Networking, 2003, pp. 131–136. [8] F. Lu, S. Parkin, and G. Morgan, “Load balancing for massively multiplayer online games,” in NetGames ’06: Proceedings of 5th ACM SIGCOMM workshop on Network and system support for games, New York, NY, USA, 2006, p. 1, ACM Press. [9] D.T. Ahmed, S. Shirmohammadi, and J. Oliveira, “Improving gaming experience in zonal MMOGs,” in MULTIMEDIA ’07: Proceedings of the 15th international conference on Multimedia, New York, NY, USA, 2007, pp. 581–584, ACM.