Two New Approaches for Orphan Detection - IEEE Computer Society

Two New Approaches for Orphan Detection

a

M. Jahanshahia, K. Mostafavia, M. S. Kordafsharia, M. Gholipoura , A.T. Haghighata,b Department of Electrical, Computer & IT, Islamic Azad University, Qazvin Branch, Qazvin, Iran b Atomic Energy Organization of Iran (AEOI), NPPD, Tehran, Iran E-mail: [email protected]

Abstract In distributed systems which use RPC 1, If a failed process sent a request before failing, the receiver of this request becomes an orphan process and must roll back to undo the effects of receiving the message. There are two types of orphan: on of them mostly called “crash- orphan” in witch client crashes. Another which causes orphan process is called abort–orphan in which parent’s process is aborted [1].Orphans are undesirable because they waste system resources and because may make inconsistent data [2]. In this paper initially we present two novel methods for orphan detection. Finally we compare our new methods with the older ones.

Keywords Distributed systems, Orphan, RPC, DSG, Reincarnation, Extermination, dedicated server group, load balancing.

1. Introduction In the simplest case a remote procedure call is implemented by first sending a massage to a server and then waiting for a reply from the server. In RPC systems if client requests something from server, and immediately before response to come back it crashes, a process shall be performed that no parent is waiting for response. The mentioned process which has no parent is called “Orphan”. This is mostly called “Crash- orphan“. But other one which causes orphan is called abort–orphan in which parent’s process is aborted [1]. Orphans cause some problems including wasting the processor cycle or lock the resources forever, even client may request for the same previous RPC and cause re-execution of it which shall have special consequences [5]. Initially RPC systems only provided peer-to-peer communication involving the interaction of each client with only one server[3].But nowadays distributed applications run on a set of 1

independent nodes interconnected by a communication network that nodes can crash independently [2,5,6]. A basic goal of all rollback-recovery protocol is to bring the system into a consistent state when inconsistencies occur due to failure. Rollback-recovery protocols classified into checkpoint-based and log-based [15]. These protocols require that each process periodically record its local state. The state of a process is a mapping of program variables and implicit variables such as program counter to their current values [6]. Checkpointing have three types: coordinated, uncoordinated, or communication-induced. Log-based protocols combine Checkpointing with logging of nondeterministic events called determinants. Log based protocols can be pessimistic, optimistic, or causal. Theoretically message logging alone is sufficient but checks pointing speeds up the recovery [7].Message logging cause some overhead: First, each message must in general be copied to the local memory of the process that directly affects communication throughput and latency. Second, the volatile log is regularly flushed to stable storage to free up space. Third, message logging nearly doubles the communication bandwidth required to run the application for systems that implement stable storage via a highly available file system accessible through the network. In pessimistic log-based protocols orphans are never created due to a failure. Optimistic log-based methods allow orphans to be created due to failures. Also there exist pessimistic receiver based logging protocols, optimistic receiver based logging protocols, pessimistic sender base logging protocols, optimistic sender based logging protocols[8].Checkpoints and event logs consume storage resources. As the application progresses, an algorithm that called Garbage collection must be ran to deleting useless recovery information that implies overhead [15]. Another disadvantage of message logging protocols is rollback propagation may causes domino effect. Totally all of orphan detection methods have broadcast overhead or logging burden on the disk. In this paper we present two novel methods that overcome to previous method’s drawbacks such as Broadcast overhead and logging burden on the disk.

Remote Procedure Call

Proceedings of the 19th International Conference on Advanced Information Networking and Applications (AINA’05) 1550-445X/05 $20.00 © 2005 IEEE

2. Previous approaches for orphan detection Nelson presented some methods to cope with the orphan processes [5]: In first method the client makes one log entry about what it wants to perform before performing a RPC. When client is rebooted again, the log will be examined and the orphans are killed clearly. This method is called extermination. The problem of this method is the high cost of logging per each RPC. On the other hand, the orphans may create new RPC or process that is difficult access of which (grand orphan) due to network failures. Therefore even if the orphans are located they can not be accessible. In the other approach named Reincarnation there is no need to logging. There is a concept in this method which is called epoch. When a client is rebooted again, it broadcasts epoch message. It means new course has been started. When this message is arrived, the orphan processes are killed. Problem of this method is its broadcast overhead in network and resulted traffic. There is another version of Reincarnation method: In this approach when epoch message is received, at first it will be checked whether the process has an owner, if there is no owner, the process will be killed. This approach is known as Gentle Reincarnation. In the forth approach a deadline shall belong to each RPC. If the work is not completed within the specified time, then one new deadline shall be requested. At this point, calculation of deadline is important. Suppose that orphan locks one or several records of database or several resources. If the orphan is killed suddenly, the said resources are remained as locked for ever.

3. Our Methods In this section we present two new methods for orphan detection:

3.1. DEDICATED SERVER GROUP (DSG) Method In this method, each client is dedicated to a specific server and therefore the request of clients can be sent only to the dedicated server. Therefore after crash and reboot of client, the client sends epoch message to the dedicated servers only. In order to improve this method, k factor can be used to remove the defect of this method. It means that k number of cps(client per server) group communicate with k number of servers(server group). Client Client

K=2 Server

Client Client

Server

Client Client

Server

Client

Server

Client

Figure 1. k groups of clients communicate with k servers

In our method, number of epoch messages respect to broadcast to entire of network, equals to (1 − S #−1) . S# Where S# is number of servers. If it is applied with k factor, number of epoch messages with respect to broadcast to entire of network equals to (1 − S #−1 − (k − 1) ) . S#

The above formula gives us a limitation. It means that if S# goes toward indefinite, then product of the above said formula goes toward zero. Disadvantage of this method is that other servers may be idle and dedicated servers become busy and RPC operations are performed with delay. For cope with this problem another dynamic version of this method is presented here: In this method we use server group concepts; there is a token containing three fields: UA, SG#, time_stamp. AU means that the utilization amount of DSG and SG# is the related server group. The token moves around the server groups on a ring topology. In each step if a server group realizes that its utilization amount is less than the UA of the token then it overwrites its UA and SG# in the token. When ever a server group overwrites in the token it generates two copies of the token and sends them in opposite direction for speed up. When two tokens reach to a server group simultaneously two cases can be occurred: First case, if two tokens have identical Time stamp, then each of them that has a lower UA can be taken and another token is destroyed and Second case: if two tokens have different time stamp each of them that has higher time stamp can be taken and another token is destroyed. During the said transfer, each server group takes a copy from token for itself and overwrites it on the older version. Once new client restarts, it sends a request to the nearest server group. Considering its copied token the server group redirects the request of the clients to the efficient server group that has lowest utilization amount. After it, all of this client’s requests are sent to this server group (DEDICATED). After it if the DSG realizes that the all of its servers are busy, considering its copied token, redirects the input request to the another server group that its SG# has been written in the token. In this situation, all of requests of that client are sent to both DSG and second server group. When all requests to the second server group have been responded, DSG may select a different server group for the next time. Therefore traffic is distributed in the network. In this method epoch massage sent at most to (2 * N; N is number of servers in each group). The advantage of this method is that it neither logs like Extermination method that causes high cost of logging, garbage collection, domino effect and memory consumption, nor broadcast to entire of networks like Reincarnation method that causes high traffic. Another advantage of our method is that requests of clients can be redirect to other idled servers adaptively. DSG method algorithm has been presented in Figure 3.

Proceedings of the 19th International Conference on Advanced Information Networking and Applications (AINA’05) 1550-445X/05 $20.00 © 2005 IEEE

3.2. Binder method One of the methods for locating the servers is use of dynamic binder. In this method, the servers reach to export binder. In the export, address of servers and their unique identifier as well as the client which are supported, are recorded. When a client wants to perform one RPC, client stub sends a message to the binder and binder checks whether any server has exported itself. If any server not register itself, then the request shall be failed otherwise the specified server’s address is sent to the client and client stub sends its request to the said address. This method is very flexible because: One: The binder can distribute the clients between servers and two: The binder can deregister the servers which are failed to client request. Our idea in this method is that the client after rebooting send epoch message to binder. Since the servers register the clients at the time of export to support of clients and if server not to be bind, then the request is failed, therefore the binder

sends epoch message only to the related server. Advantage of this method is that instead of sending epoch message to entire of network (like Reincarnation method), it is sent only to a binder and the binder sends the message only to the specified server. Another advantage of this method is that for each request, log is not taken. It means that this method utilizes binder capability and also any further repeat is prevented. Disadvantage of this method is the centralized method which has a bottle neck but by redundancy the said problem can be solved. Comparison between DSG and Reincarnation methods 1200 Number of exchanged messages

Figure 2 shows that our method considering the number of messages that must be exchanged between nodes in an environment with N servers in each group is better than reincarnation method. N can be any number. In this chart N is fifty.

1000

1000 900

800

Minimum of exchanged messages in reincarnation method

800 700

600

600 500

400

400 300

200

200 100 50 50 50 50

0 1

2

3

4

50 5

50 6

50 7

50 8

50 9

50

Maximum of exchanged messages in DSG method

10

Numbe of servers (Scale=1/100)

Figure 2. Comparison between DSG and Reincarnation methods

Token Subroutine SIMULTANEOUSLY (x, y As Token) // when two tokens arrive to a group simultaneously this subroutine is called { if ( x.time_stamp = = y.time-stamp ) then if ( x.UA < y.UA ) then { Destroy (y); Return (x) ; } else { Destroy (x); Return (y) ; } else if ( y.time_stamp < x.time-stamp) then { Destroy (y); Return (x) ; } else { Destroy (x); Return (y) ; } } Subroutine INITIATE (Client) // Every client after reboot call this subroutine { NSG = nearest server group; Client sends its request to the NSG; If NOT BUSY (NSG) then DEDICATE (Client, NSG); //Client is dedicated to NSG Else DEDICATE (Client, TOKEN.SG#) ; //Client is dedicated to the Server group that written into token } Subroutine LOAD_BALANCING (Client) // when server group realize that is busy it calls this subroutine { if BUSY (server group) then { SECOND_SERVER_GROUP = TOKEN.SG# ; REDIRCET (Client.request , SECOND_SERVER_GROUP) ; BACKUP(server group , SECOND_SERVER_GROUP ); // SECOND_SERVER_GROUP is backup of server group } if RECEIVE_RESPONSE(ALL Client.request , SECOND_SERVER_GROUP) then BACKUP(server group , NULL); }

Figure 3. Pseudo code of DSG method

Proceedings of the 19th International Conference on Advanced Information Networking and Applications (AINA’05) 1550-445X/05 $20.00 © 2005 IEEE

Table 1. Comparison between our methods with previous ones Method name

Number of messages

Advantages

1 Extermination

Reincarnation

Binder

At least equals to number of servers 2 At most equals to (2 * N) N = Number of servers in each group

DSG

1. RPC operation is done rapidly 1. No logging 2. No broadcast 3. Do load balancing 1. No logging 2. No broadcast 3. Efficient use of resources 4. Adaptively sends the traffic to the idle servers 5. Is a distributed method 6. No need for running the garbage collection algorithm 7. There is not domino effect 8. There is not Exponential roll back

4. Conclusion Different orphan detection methods offer different tradeoffs between performance, storage overhead, and simplicity of recovery. In this paper, initially we present a survey of roll back recovery protocols and then we discuss about four most important approaches for orphan detection. After it we present two new methods that overcome to previous method’s drawbacks such as Broadcast overhead and logging burden on the disk. Table 1 shows a comparison between our methods and older ones.

References: [1] Joachim Baumann, Kurt Rothermel, “The Shadow Approach: An Orphan Detection Protocol for Mobile Agents”, Institute for Parallel and Distributed High-Performance Systems,http://www.informatik.uni-stuttgart.de/ipvr/vs/ws/ma98 /presentations/Baumann.pdf. [2] Maurice P. Herlihy, Martin S. Mckendry, “Timestamp-Based Orphan Elimination”, IEEE transaction on software Engineering, VOL. 15, NO. 7, 1990. [3] L. P. Barreto, I. Jansch-Porto, “Open and Reliable Group Communication”, Proc. Sixth Euromicro Workshop on Parallel and Distributed Processing, Madrid, Spain, 1998, pp. 389-394. [4] Valerie Issarny, Gilles Muller, and Iisabelle Puaut, “Efficient Treatment of Failures in RPC Systems”, Proc. IEEE Transaction on Computer, 1994, pp. 170-78. [5] Andrew S. Tanenbaum, Distributed Operating System, Prentice-Hall, 2003. [6] Lorenzo Alvisi, Keith Marzullo, “Message Logging: Pessimistic, Optimistic, and Causal”, IEEE Transactions on Software Engineering, VOL. 24, NO. 2, 1998.

Disadvantages 1. RPC operation is done slowly 2. Memory consumption 3. Need for garbage collection algorithm 4.There is domino effect 1. Overhead due to broadcast traffic 2. Exponential roll back Bottleneck

[7] Om P. Damani, Vijay K. garg, “How to Recover Efficiently and Asynchronously When Optimism Fails”, IEEE Transactions on Software Engineering, VOL. 1063-6927, 1996, pp. 108-114. [8] Alvisi, L. and Marzullo, K., “Non-blocking and orphan-free message logging protocols”, Proc. 23rd IEEE International Symposium on Fault-Tolerant Computing, Toulouse, France, 1993, pp. 145-154. [9] R. Baldoni, J. Brzezinski, J.M. Helary, A. Mostefaoui and M. Raynal, “Characterization of consistent global checkpoints in large-scale distributed systems”, Proc. 5th IEEE Workshop on Future Trends of Distributed Computing Systems, Chenju, Korea, 1995, pp. 314 -323. [10] Fabio Panzieri, Santosh K. Shrivastava, “A Remote Procedure Call Mechanism Supporting Orphan Detection and Killing”, Proc. IEEE Transaction On Software Engineering, VOL.14, NO.1, 1988. [11] K. Ranvindran, Samuel Chanson, “Failure Transparency in Remote Procedure Calls”, IEEE Transaction on Computer, VOL. 38, NO. 8, 1989. [12] Ahmed K.Ezzat, “Orphan Elimination in Distributed ObjectOriented Systems, Proc. Second IEEE Workshop on Future Trends Distributed Computing Systems, Cairo, Egypt, 1990, pp. 403-412. [13] By B. Yao, W. Fuchs, ”Message Logging Optimization for Wireless Networks”, Proc. 20th IEEE Symposium on Reliable Distributed Systems, 2001, pp. 182-187. [14] Stefan Pleisch, Arnas Kupsys and Andre Schiper, “Preventing Orphan Requests in the Context of Replicated Invocation”, Proc. IEEE 22nd International Symposium on Reliable Distributed System,2003. [15] E. N. (Mootaz) Elnozahy, Lorenzo Alvisi, Yi-Min Wang and David B. Johnson, “A survey of rollback-recovery protocols in message-passing systems”, Proc. ACM Computing Surveys (CSUR), Vol. 34, Issue 3, 2002, pp. 375 – 408. [16] Kwang-Sik Chung , Ki-Bom Kim , Chong-Sun hwang ,jin gon Shon and Heon-Chang yu, "Hybrid checkpointing protocol based on selective-sender-based message logging", Proc. International Conference on Parallel and Distributed Systems, 1997, pp. 788-793.

Proceedings of the 19th International Conference on Advanced Information Networking and Applications (AINA’05) 1550-445X/05 $20.00 © 2005 IEEE