Java RMI versus .NET Remoting Architectural ... - CiteSeerX

Java RMI versus .NET Remoting Architectural Comparison and Performance Evaluation Roland Schwarzkopf1 , Markus Mathes2 , Steffen Heinzl2 , Bernd Freisleben2 , Helmut Dohmann1 1

Department of Applied Computer Science University of Applied Sciences of Fulda Marquardstr. 35, D-36039 Fulda, Germany E-Mail: {roland.schwarzkopf,helmut.dohmann}@informatik.hs-fulda.de 2

Department of Mathematics and Computer Science University of Marburg Hans-Meerwein-Str. 3, D-35032 Marburg, Germany E-Mail: {mathes,heinzl,freisleb}@informatik.uni-marburg.de

Abstract The implementation of distributed applications can be based on a multiplicity of technologies, e.g. plain sockets, Remote Procedure Call (RPC), Remote Method Invocation (RMI), Java Message Service (JMS), .NET Remoting, or Web Services. These technologies differ widely in complexity, interoperability, standardization, and ease of use. To achieve the highest efficiency, the use of plain sockets is advisable, whereas highest interoperability and standardization argue for Web Services. Real world applications requirements are a trade-off between efficiency, development costs, usability, reliability, interoperability, and so on. For example, a multimedia streaming application requires high performance to avoid delays, whereas Internet trading platforms depend on high interoperability, reliability and availability. Since Java RMI and .NET Remoting support fast development of highly efficient applications, many developers choose one of these technologies to realize their applications. Besides differences in design, the two technologies offer diverse performance characteristics. This paper has two main contributions: (1) a comparison of the design and architecture of Java RMI and .NET Remoting is presented, (2) an experimental performance analysis with respect to various aspects of both technologies is conducted.

1. Introduction The diversity of available middleware technologies to implement distributed applications is quite large. These

technologies differ widely in the underlying communication paradigm, ease of use for developers, interoperability, and standardization level. For the implementation of an efficient distributed application, developers often choose a widespread middleware technology like Java RMI [17] or .NET Remoting [9]. Java RMI is platform independent since it is based on the virtual machine concept of Java, whereas .NET Remoting is based on Microsoft Windows, which enjoys a large market share. The decision for Java RMI and against .NET Remoting or vice versa often depends on the architecture and the performance of each technology. To enable well-founded decisions of developers for one technology and against the other, this paper has two main contributions: (1) a comparison between the design and the architecture of Java RMI and the design and the architecture of .NET Remoting is given, (2) an experimental performance analysis of various aspects of both technologies is conducted. The comparison between Java RMI and .NET Remoting is based on their main features: remote object requirements, discovery and activation of remote objects, general design of the communication layer, distributed garbage collection, and security. Since both technologies are based on the invocation of remote methods, our performance analysis investigates this functionality. Method invocations in real world applications have different characteristics. Typically, there are three main types: (1) invocation of a method with few primitive parameters, (2) invocation of a method with large binary data (bulk data), (3) method invocation with complex pa-

rameters (e.g. objects). The first type of method invocation – invoking a method with few primitive parameters – is the most frequent one. For example, setting a timestamp, triggering an action, or setting an application-specific flag require only few or no primitive parameters at all. The second invocation type is the transfer of bulk data, e.g. binary audio/video data or measurements of a physical experiment. The third type of method invocation – invoking a method with complex parameters – is common in business applications. Personnel data, rows in a table, orders, or bills are represented as object types containing several fields. Consequently, these three types of method invocations are investigated in three different test scenarios for each technology. The rest of the paper is organized as follows. Section 2 gives a comparative overview of the architecture of Java RMI and .NET Remoting. Section 3 presents the performance analysis and section 4 its results. Section 5 discusses related work. Section 6 concludes the paper and outlines areas for future work.

2. Architecture of Java RMI and .NET Remoting Both Java RMI and .NET Remoting are object-oriented middleware systems. They allow to create remote objects that can be used across system boundaries, making use of local stubs or proxies that act on the client-side for the remote object and hide the distribution from the developer. Parameters and return values of remote method invocations are transmitted using the platform-specific serialization mechanisms. In Java RMI, remote objects need to implement a socalled remote interface, containing all remotely invocable methods. The remote interface will also be implemented by the generated stub. In Java RMI, it is possible to transfer objects with arbitrary superclasses or the special superclass UnicastRemoteObject. .NET Remoting forces the use of a specific superclass called MarshallByRefObject, but works without a remote interface by using extracted metadata or the implementation of the object to dynamically generate a proxy. The discovery of remote objects in Java RMI can be achieved with a specialized name service which is limited to the Java RMI domain, or with a generic directory service like LDAP using JNDI. Missing classes can be automatically obtained during this process. In .NET Remoting, such a feature is missing, thus the locations of the remote objects have to be hard-coded or specified in a configuration file. Both systems can create required remote objects during runtime using a technique called object activation. While this feature was already available in .NET Remoting in its first version, it is part of Java RMI since version 1.2. To stay backward compatible, the integration of the activation

system is not as seamless as in .NET Remoting. Remote objects require a different superclass or a change in the initialization code to be recognized as activatable in Java RMI. The actual activation is done by a special daemon, which can even start a new Java Virtual Machine (JVM) if needed. In .NET Remoting, no changes to a remote object are necessary, it just needs to be registered with the .NET Remoting infrastructure instead of being created and published manually. Additionally, there are different modes of activation in .NET Remoting: (1) one instance for all invocations (Singleton), (2) one instance for each invocation (SingleCall), (3) creation of remote objects on the client-side using the new-operator. The latter activation pattern is unfortunately not compatible with the recommended use of remote interfaces. The communication in Java RMI is based on plain sockets or HTTP. Socket factories allow the use of custom sockets, for instance to encrypt the data using SSL, or to use other protocols or interconnection networks. The parameters and return values of a remote invocation are transferred using the Java RMI Wire Protocol [19]. .NET Remoting, on the other hand, uses a message-based communication system, which supports TCP, HTTP, and interprocess communication (IPC). The parameters and return values of a remote invocation are transmitted in a serialized message object. Every layer is based on the concept of sinks and chains of sinks. Sinks can modify the message object and in the lowest layer also its serialized form. The developer is able to implement custom sinks, which can be placed in object-specific or global chains, to customize the serialization process. Additionally, the proxy can be replaced by a customized version, e.g. to check the parameters for a method prior to the actual remote invocation. Another important aspect of both systems – especially in conjunction with object activation – is distributed garbage collection. Java RMI takes complete care of this using a system of leases [16] and automatic lease-time extension. This prevents the existence of stubs for already deleted objects, but burdens the network with additional communication. .NET Remoting uses a completely different approach, where each remote object is assigned a lifetime that will be extended with every incoming method invocation. Sponsors provide another way to prevent a remote object from being deleted, by granting additional lifetime to that object by request. To achieve that, the sponsor, which may also be a remote object, needs to be registered with the remote object. It is then automatically queried at the end of the remote objects liftime. If the extension is not granted or no sponsors are registered, the object is deleted. Due to this concept, it is possible that there is a proxy for a non-existing remote object. In the field of security, Java RMI only provides the possibility to encrypt the data using SSL. In .NET Remoting,

this can be done by hosting the remote object in Microsoft’s Internet Information Services (IIS) or using IPSec. Besides that, .NET Remoting provides authenticated streams for TCP and IPC, which allow identity switching and delegation as well as encrypted and signed communication. In summary, both platforms provide comparable opportunities for the developer, but there are some differences as well. The high adaptability of the communication layer, the different object activation patterns, garbage collection without additional communication, and the advanced security features are advantages of .NET Remoting. Java RMI, on the other hand, has a naming service, guarantees the existence of remote objects represented by stubs, and allows the use of classes with arbitrary superclasses as remote objects. For some applications some of those differences may be vital, but for many applications those differences are not sufficient to make a decision between the platforms. Therefore, we take closer look at the performance characteristics of both technologies.

3. Performance Evaluation The performance of both systems is compared in three scenarios, which are explained here in detail. All scenarios are tested using TCP and HTTP as communication protocols in combination with binary serialization. Additionally, we implemented a small library for remote method invocation on both platforms. It is limited to single-argument methods and a single object, and requires manually created stubs and skeletons. The transmission of arguments is done using the platform-specific serialization mechanism (using binary serialization in .NET Remoting). The library uses plain TCP sockets for communication. A comparison of the results should provide an indication of the overhead imposed by the use of Java RMI and .NET Remoting. It has to be noted that there is no portable way to force Java RMI to use HTTP, because it is only a fallback mechanism in case TCP communication is blocked by a firewall. Here, a factory from a non-standard package below sun.rmi is used to achieve this, which is normally not intended for the use by developers, because it may be not supported in future versions or other Java implementations. An Intel P4 2.0GHz with 1,280MB RAM is used as the client, whereas an AMD Athlon XP 1600+ with 512MB RAM is used as the server. Both are running under Windows XP SP 2. They are connected to a 100Mbit switch using 100MBit network adapters. No other device is connected to the switch during the measurements. Sun’s Java Runtime Environment 1.6.0 Update 2 and Microsoft’s .NET Framework 3.0 are used for the tests.

3.1. Scenario 1 – remote method with primitive parameter In this scenario, invocations of a remote method with primitive parameters are investigated. It consists of 50,000 invocations of the following method in a loop: void scenario1Method(int data). The counter variable of the loop is used as a parameter for the invocation. There is an important difference in the object model of Java and .NET that may affect the measurements. The Java type system knows not only object types, but also primitive types such as int or double. For each primitive type, a so-called wrapper type exists, i.e. an object type containing the value of a primitive type. The conversion between primitive type and its corresponding wrapper type had to be done manually, but with the release of Java 5 a feature named autoboxing has been introduced to automate this conversion, if necessary. Because of the distinction between primitive types and object types, the two methods void scenario1Method(int data) and void scenario1Method(Integer data) are not identical in Java. Therefore, it is possible to use primitive types for remote method invocation. The object model of .NET does not contain primitive types, but instead so-called integrated types. The types int and double are of this kind and must not be mixed up with their counterparts in Java, because they are only aliases to the structures System.Int32 and System.Double. Structures are something between a primitive type and an object type: like a primitive type their values are copied during method invocation (call by value), like object types they can contain any number of fields and methods and can even implement interfaces. Literals of integrated types are automatically boxed, i.e. they are copied into a suitable structure. Therefore, the two methods void scenario1Method(int data) and void scenario1Method(Int32 data) are identical in .NET. Since for this test scenario primitive types are used in Java, a better performance for Java RMI may be expected.

3.2. Scenario 2 – remote method with bulk parameter In this scenario, invocations of a remote method with bulk parameters are examined. We selected a remote method with textual data as parameter since the transmission of XML-based data becomes more and more important. Since the internal representation of characters in Java and .NET is based on Unicode, the transmission of textual data encoded as plain bytes introduces an additional conversion step. We avoid this conversion step by using an array of char as parameter. The

scenario consists of 10 invocations of the method void scenario2Method(char[] data). An array with a size of one megabyte will be used as parameter, which has 524,288 elements, because the size of a char is 2 bytes. The amount of data actually transferred is slightly bigger than one megabyte, because arrays are treated as objects and therefore some management information is stored.

3.3. Scenario 3 – remote method with object parameter In this scenario, invocations with complex object trees are examined. This is tested by using the class S3Object that consists of an array of dummy objects, a counter and an enum-value. The dummy objects contain 0, 10, 20, 30, 40, or 50 fields of type double. We have chosen double-values since they are often used in scientific computing, e.g. as coefficients in a matrix. The method removes one of the dummy objects from the array (by storing null at its position) and returns the modified S3Object instance. The method signature is as follows: S3Object scenario3Method(S3Object data). A complete run of this scenario consists of the following steps: 1. The client fills the S3Object with 350 dummy objects. 2. The client calls the method S3Object scenario3Method(S3Object data), which removes an element on the server and returns the modified S3Object. 3. The client also removes a dummy object from the array. 4. The client checks if the array contains any more dummy objects. In this case, it continues at step 2, otherwise the scenario is completed. Since both client and server remove one element, a run will consist of 175 invocations.

performance of a complete distributed system consisting of middleware and underlying communication infrastructure, i.e. sockets and interconnection network, is considered. No attempt is made to measure only the underlying communication infrastructure, which is sometimes done to be able to calculate a kind of middleware-only performance. Since Java and .NET are completely different platforms, it is difficult to compare their clocks. It is of course possible to do the measurements outside of the platform on the system level, but the results would by falsified by the initialization of the runtime environments. Thus, the time should be measured within the platform. Although the standard timers offer nanosecond resolution on both platforms, the achieved precision is operating system dependent, e.g. on Windows only 10 milliseconds in Java and 15 milliseconds in .NET [1] are achievable. Windows provides a high precision timer called Performance Counter [10]. This counter is increased with a fixed frequency. Using a Windows API, both the counter value and the frequency can be queried. On the client that does the measurements, the frequency is 3,579,545Hz corresponding to a resolution of about 0.28 microseconds. To use that timer in Java, a library written in C is necessary, which is called using JNI. .NET Remoting permits to call any Windows API directly. On both platforms, the use of this timer causes overhead, because it is outside the runtime environment. To check the achieved resolution, a test case was implemented. It queries for 10 values of the Performance Counter and calculates the time differences between two values. The average of those differences, 1.76 microseconds in Java and 1.58 microseconds in .NET, is a good approximation of the achieved accuracy. As expected, the resolution of this timer is more fine grained compared to the standard timers. Additionally, the difference of the resolution on the two platforms is much smaller, which makes the Performance Counter a good choice for the measurements.

3.5. Evaluation of the Serialization Mechanisms

3.4. Measurement Method The time to execute each of the three scenarios was measured. To compensate the effects of concurrent processes on our measurements, every scenario was repeated 10 times and the average execution time was calculated. Additionally, the serialization mechanism will be examined in subsection 3.5, because it is vital to both Java RMI and .NET Remoting. Java RMI and .NET Remoting are considered as black boxes, thus no kind of optimization was done. Since the objective of this paper is to enable wellfounded decisions between the two platforms, the measurements should be as realistic as possible. Therefore, the

A serialization mechanism is used to convert the parameters and result values of remote method invocations into a transferable format. Thus, an evaluation of this mechanism is very helpful for the interpretation of the results. Special test cases were designed that simulate the serialization in the real scenarios: • Scenario 1: 50,000 serializations of the counter value (type int) done within a loop. • Scenario 2: 10 serializations of a char-array with 528.288 elements done within a loop.

• Scenario 3: (1) Creation of a S3Object instance filled with 350 dummy objects. (2) Serialization of the S3Object instance. (3) Removal of a dummy object. (4) Repetition of steps 2 and 3 until the S3Object instance contains no dummy objects anymore.

with char data as parameters.

test case One-Byte Two-Byte Three-Byte

For each serialization in Java, a new ObjectOutputStream is used, which simulates the behavior of Java RMI, where each remote invocation creates a new serialization stream [19]. There is no need to care about this in .NET, since it creates serialization streams implicitly.

1 2

3

parameter type UTF-8 character type number of doubles

Primitive Wrapped One-Byte Two-Byte Three-Byte 0 10 20 30 40 50

Java (MB) 0,4768 3,8624 10,0003 10,0003 10,0003 0,8512 5,5679 10,2876 15,0074 19,7271 24,4468

.NET (MB) – 2,5749 5,0003 10,0003 15,0003 0,9689 5,6856 10,4054 15,1251 19,8448 24,5646

Table 1. The amount of data generated for serialization of the parameters in the particular scenarios. Table 1 shows the amount of data generated during the serialization tests. In scenario 1, not only the invocation using the primitive type, but also the invocation using the corresponding wrapper type was evaluated for Java. Since primitive types are only aliases in .NET, its results are listed in the category “Wrapped”. The second scenario reveals an essential difference between the serialization mechanisms in Java and .NET. Whereas Java writes 2 bytes for every char – which is exactly the way the char is stored in memory – .NET seems to use a kind of UTF-8 for serialization of chars, which uses one, two or three bytes depending on the character. The four byte UTF-8 representation of a char is not needed, because it applies only to 21bit characters that cannot be stored in the char type. Unfortunately, the documentation does not contain any details about this. To get objective results, the scenario is divided into three test cases with arrays containing specific values (table 2). In reality, the data of course contains a mixture of characters leading to a mixture of corresponding UTF-8 representation lengths, so the results of this scenario may only give a rough estimate of the efficiency of .NET Remoting in combination

length of the UTF-8 representation 1 byte 2 bytes 3 bytes

Table 2. Serialization of different char values.

Figure 1 shows the time needed to execute each test. Java’s serialization mechanism outperforms its .NET counterpart in almost every test, even with a greater (scenario 1) or equal (scenario 2) amount of data . The only test with a different result is the “One-Byte” case of scenario 2. The reason in this case seems to be the array which can be serialized fast in .NET Remoting, because there is only little metadata compared to the amount of actual data. Additionally, the length of the serialization stream is smaller compared to Java because of the UTF-8 encoding. In the other tests, the serialization process is either more complex (scenario 3) or the metadata-to-data ratio is worse (scenario 1).

time required for serialization (sec)

Scenario and subtype

char value 0x0000 0x0400 0xFFFF

5 4.5

Java .NET

4 3.5 3 2.5 2 1.5 1 0.5 0 (1) (2) Scenario 1

(1) (2) (3) Scenario 2

(1) (2) (3) (4) (5) (6) Scenario 3

Figure 1. Required time for serialization of the parameters on the client side. For scenario 1 (1) shows the use of primitive, (2) the use of wrapped parameters. For scenario 2 (1) is the One-Byte, (2) the Two-Byte and (3) the Three-Byte case. For scenario 3 (1)-(6) show the use of a dummy object with 0, 10, 20, 30, 40, or 50 double fields.

During the measurements, two issues were identified that may cause problems. The first issue is the HTTP mode of the Java RMI Wire Protocol that collides with the standard socket settings in Microsoft Windows during the execution of scenario 1. When used over HTTP, Java RMI uses the SingleOpProtocol [19], which creates a new HTTP connection for every remote method invocation. In scenario 1, this causes the creation of 50,000 connections (lookup and garbage collection not taken into account). After a few thousand invocations, the benchmark program is closed due to an unhandled BindException. In Microsoft Windows, a boundary exists for anonymous port numbers assigned to sockets during the connect operation. This boundary can be configured with the key MaxUserPort in the Windows system registry, whose default value is 5,000. After subtracting the first 1024 reserved ports, there are approx. 4,000 ports that can be used by programs. This is no problem during normal use, since the port number of a closed socket can be reused after a short period of time. The period is configurable with the key TcpTimedWaitDelay. Unfortunately, the benchmark program creates and closes connections faster than closed ones can be reused. Although changing TcpTimedWaitDelay to the smallest allowed value may solve the problem, here MaxUserPort was changed to its maximum (65,534) [8]. The second issue is method overloading in .NET in combination with .NET Remoting. Scenario 1 was extended with a custom wrapper type, overloading the method void scenario1Method(int data), resulting in the methods void scenario1Method(int data) and void scenario1Method(CustomWrapper data). Just the existence of this overloaded method caused a serious overhead during the next measurements (TCP 58 seconds, HTTP 89 seconds, cp. subsection 4.1 for comparison values without the overloaded method). This indicates a problem with the dispatching of a remote method invocation. Until this issue is further analyzed, we discourage the use of method overloading with .NET Remoting.

4. Results This section presents the results of the experimental performance analysis. The results of scenario 1 to scenario 3 are discussed in detail. If not stated otherwise, TCP and HTTP results cover Java RMI and .NET Remoting using these protocols, manual covers our own library for remote method invocations.

4.1. Scenario 1 – remote method with primitive parameter The results of this scenario are shown in table 3 and figure 2.

platform RMI

.NET

protocol TCP HTTP manual TCP HTTP manual

mean value (sec) 11,499 335,784 11,392 27,518 55,275 11,035

standard deviation (sec) 0,106 0,667 0,008 0,290 0,083 0,017

Table 3. Results of scenario 1.

350 Java .NET 300 250 time (sec)

3.6. Implementation Issues

200 150 100 50 0 TCP

HTTP transport protocol

manual

Figure 2. Results of scenario 1. The results can be summarized as follows: Java RMI is the best choice for TCP, .NET Remoting for HTTP (in this scenario). The difference in the HTTP measurements is a result of the different handling of connections: .NET Remoting uses only a single connection throughout the complete benchmark, while Java RMI creates a new connection for each invocation. Another problem with both platforms is the overhead caused by the use of the HTTP protocol. Java uses around 10 bytes, .NET around 54 bytes for the serialization of an integer object (values are estimated by dividing the amount of data measured in the serialization tests (subsection 3.5) by 50,000). Comparing the actual data with the length of both HTTP headers (request and response), the data-to-overhead ratio is rather bad in this scenario. The reason for the bad results of .NET Remoting using TCP are not obvious. The differences in the object model

varies from 0.5 to 1.5 times the length of the Java serialization stream. This leads to varying measurement results for the different arrays used. Therefore, there is no obvious best choice for TCP. In specific applications, one of the platforms may still be more suitable than the other. If the use of HTTP is necessary, .NET Remoting is clearly the best choice. The overhead caused by HTTP is very small in this scenario due to the large amount of data transmitted with a single call, as it can be seen when the TCP and HTTP results of .NET Remoting are compared. The reason for Java’s bad performance using HTTP is again the number of created connections in relation to the short time the complete run of the scenario takes.

2 1.8

Java .NET (One-Byte) .NET (Two-Byte) .NET (Three-Byte)

time (sec)

1.6 1.4 1.2 1 0.8 0.6 0.4 TCP

HTTP transport protocol

manual

Figure 3. Results of scenario 2. do not seem to have a big influence on the results, because .NET nevertheless outperformed Java using our own library. Another possible reason is the use of message objects instead of a specialized protocol, which most likely causes a small overhead which could explain the differences we measured, because of the high number of invocations.

4.3. Scenario 3 – remote method with object parameter Tables 5, 6 and 7, and figure 4 show the results of scenario 3. platform

Java RMI

4.2. Scenario 2 – remote method with bulk paramter Table 4 and figure 3 show the results of scenario 2.

platform

test case

Java

— OneByte

.NET

TwoByte ThreeByte

protocol TCP HTTP manual TCP HTTP manual TCP HTTP manual TCP HTTP manual

mean value (sec) 1,039 1,670 0,992 0,532 0,563 0,487 1,036 1,051 1,029 1,521 1,565 1,467

standard deviation (sec) 0,020 0,030 0,031 0,019 0,018 0,001 0,011 0,017 0,001 0,005 0,015 0,001

Table 4. Results of scenario 2. Because the serialized representation of the char type is value-dependent in .NET, the length of the serialization stream that contains the char array used in this scenario

.NET Remoting

type of dummy object 0 doubles 10 doubles 20 doubles 30 doubles 40 doubles 50 doubles 0 doubles 10 doubles 20 doubles 30 doubles 40 doubles 50 doubles

mean value (sec) 0,561 0,980 1,387 1,833 2,303 2,886 1,114 3,381 5,452 7,524 9,558 11,646

standard deviation (sec) 0,003 0,003 0,004 0,020 0,035 0,108 0,005 0,005 0,008 0,008 0,008 0,128

Table 5. Results of scenario 3 (TCP). Considering the results of .NET in the serialization tests (subsection 3.5) – primarily the time aspect – the measurement results of this scenario were predictable, especially because these tests only covered the serialization and not the deserialization. Even though the length of the serialization stream is comparable, the amount of time needed for serialization increases rapidly with the growing size of the dummy objects. The weaknesses of the .NET serialization mechanism [2] lead to bad results for .NET Remoting in this scenario. While there is almost no difference between the measurements using Java RMI and our custom implementation in Java, there is a noticeable difference in the case of .NET Remoting (see figure 4). The result in the first case (0 doubles) is very strange, when the amount of data transfered

platform

Java RMI

.NET Remoting

type of dummy object 0 doubles 10 doubles 20 doubles 30 doubles 40 doubles 50 doubles 0 doubles 10 doubles 20 doubles 30 doubles 40 doubles 50 doubles

mean value (sec) 1,762 2,451 3,063 3,839 4,524 5,491 1,232 3,543 5,658 7,786 9,902 12,004

standard deviation (sec) 0,017 0,028 0,041 0,025 0,082 0,079 0,002 0,005 0,004 0,007 0,016 0,034

Table 6. Results of scenario 3 (HTTP). in that case is compared to the other cases (section 3.5). It seems to be a problem with buffering. Without the use of buffered streams the result in the first case is better than the results in the other cases. In all other cases, our library performs better than .NET Remoting. The difference even grows with the size of the dummy objects. This is also strange, because the transmission of the arguments is done using standard serialization in our library too, and the amount of protocol data should actually be constant. In this scenario, Java is the best choice for both TCP and HTTP, even though .NET Remoting was faster in one tested case. This case (dummy object with no doubles, HTTP) is a theoretical case, because the dummy objects are completely empty, which is not common in practical applications. In all other cases, Java RMI clearly outperforms .NET Remoting. The main disadvantage of Java RMI using HTTP also applies in this scenario, which consists of 175 remote method invocations. Thus, the usage of HTTP vaguely doubles the time needed for a complete run of this scenario. Because of the growing amount of data transferred for each call, the influence of the protocol overhead on the result is becoming smaller when the size of the dummy object increases.

5. Related Work The primary resources for Java RMI and .NET Remoting are the web pages of Sun Microsystems [17] and Microsoft [9]. Since the serialization mechanism is a central part of both Java RMI and .NET Remoting, their performance is affected by the efficiency of serialization. Hericko et al. [2] compared binary and XML serialization mechanisms of Java 1.4.1 and .NET 1.0.3705 from the performance and

platform

Java RMI

.NET Remoting

type of dummy object 0 doubles 10 doubles 20 doubles 30 doubles 40 doubles 50 doubles 0 doubles 10 doubles 20 doubles 30 doubles 40 doubles 50 doubles

mean value (sec) 0,541 0,957 1,375 1,825 2,299 2,864 5,453 2,482 3,459 4,743 5,980 7,190

standard deviation (sec) 0,003 0,002 0,003 0,011 0,048 0,141 0,381 0,121 0,004 0,070 0,005 0,008

Table 7. Results of scenario 3 (manual).

size perspective using a synthetic object containing a mix of most commonly used data types and two real-world objects from an insurance application. Considering the binary serialization, Java constantly outperforms .NET by factors ranging from around 1.3 to 2.3 and produced 20% smaller serialization streams. The results of comparing the XML serialization are not relevant for this paper, because there is no native XML serialization support in Java. The reason for using XML serialization is primarily portability, which is an issue concerning .NET and Mono (a platform-independent, open-source implementation of the .NET framework) when different system architectures are involved, but not for Java, due to the use of the Java Virtual Machine and its uniform data representation. Elbers et al. [1] have compared the performance of Java RMI and .NET Remoting using the now outdated versions 1.4.2 respectively 1.1, but their work contains several problems. (1) They exclusively evaluate HTTP for data transmission, due to its widespread use. While this statement may be true for .NET Remoting, it is definitely wrong for Java RMI. In RMI, HTTP is used as a fallback system in case of firewall problems only and its use has severe consequences for the efficiency of the distributed application. (2) The authors use special tests (so-called null-measurements) to measure the performance of the underlying communication infrastructure and use the results to correct the real measurements in order to figure out the middleware performance. Not only is the sense of this correction questionable in this case, at least if the goal of the measurements is to provide information for a well-founded decision between the techniques, but also the details about the procedure used are missing. The latter raises several questions about the inclusion of connection creation, method dispatching and se-

6. Conclusion

14 12

time (sec)

10

Java RMI (TCP) Java RMI (HTTP) Java manual .NET Remoting (TCP) .NET Remoting (HTTP) .NET manual

8 6 4 2 0 0

10 20 30 40 number of doubles per dummy object

50

Figure 4. Results of scenario 3.

rialization into the null-measurements as well as about the general parameter representation and the consideration of the overhead caused by the use of HTTP. (3) The description of a test scenario called object invocation is inaccurate, because it misses details about used data objects and therefore prevents a verification of the results. (4) The authors give only little detail about the timers they used, but claim an accuracy of 10 nanoseconds. Using the Performance Counter, the highest precision timer of the Windows platform, we achieved an resolution of 0.28 microseconds, which is 28 times worse, and an average accuracy of 1.76 respectively 1.58 microseconds, which is around 176 respectively 158 times worse. Considering all those problems, the authors’ results are considered to be uncertain. The following papers deal only with Java RMI for distributed application development, but not with .NET Remoting. Juric et al. [4] compared Java RMI using TCP and the different tunneling options for HTTP with Web Services. Huet et al. [3], Nieuwpoort et al. [14], Nester et al. [13], Maassen et al. [6, 7] compared the standard Java RMI implementation of Sun with optimized RMI implementations such as Ibis and KaRMI. Juric et al. [5] analyzed the use of Java in heterogeneous environments, comparing normal Java RMI applications with the use of the CORBA IDL (Interface Definition Language) and RMI-IIOP (RMI on Internet Inter-ORB Protocol). Several benchmarks have been evaluated using the popular Pet Shop application [18, 12] with varying results. Since those sample applications are based on the enterprise frameworks J2EE [15] and .NET Enterprise Services [11], these results are not relevant for the study performed in our paper.

The diversity of available middleware technologies to implement distributed applications is quite large. In this paper, we compared Java RMI and .NET Remoting, two widespread middleware technologies often chosen by developers to implement efficient distributed applications. Since the goal of this paper was to enable developers to make well-founded decisions for one technology and against the other, we (1) compared the design and the architecture and (2) analyzed the performance of both technologies using three different scenarios that cover a great part of typical applications. The comparison of the serialization mechanisms of Java and .NET showed that here are significant differences in efficiency, which affect distributed applications using Java RMI and .NET Remoting. Thus, an evaluation of the serialization mechanism with application-specific objects should be done at the beginning of a decision making process, because the results may be a good reason for one platform and against the other. After the evaluation of Java RMI and .NET Remoting over TCP and HTTP using the three most common types of remote invocations, the first result was the predominance of TCP over HTTP on both platforms. This was an expected result that is most likely caused by the overhead of creating, transferring and processing the required HTTP headers. The impact of this overhead on the general performance depends on the data-to-overhead ratio and the number of remote method invocations done. This could be seen when the TCP and HTTP results of .NET Remoting in the three scenarios were compared. While there was a huge difference when the data-to-overhead ratio and the number of invocations is high (scenario 1), the difference was minimal when the data-to-overhead ratio is better and/or the number of invocations is smaller (scenarios 2 and 3). In general, the use of HTTP is a question of principle. One of the major reasons to use one of those platforms is efficiency. The logical consequence is the use of TCP, because it is more efficient than HTTP, that is mostly used to bypass firewalls. If firewalls need to be passed with remote method invocations, Web Services might be a better choice, because they gracefully pass any firewall when bound to HTTP and are highly interoperable at the price of less efficiency. Besides that, Java RMI over HTTP is by design not intended for frequent use, because it uses a single connection for every remote invocation. Interoperability is also no reason for the use of HTTP, because both technologies are not intended to be used in conjunction with other distributed application frameworks. Using TCP, Java RMI delivers the better or at least competitive results compared to .NET Remoting. Scenario 1 showed an unexpectedly bad result for .NET Remoting

when it comes to many invocations using less data. The reasons could not be finally determined, maybe it is a problem with the buffer polling rate [1]. Although .NET Remoting in some cases was faster than Java RMI in the second scenario, this is no universal result. With the same amount of data, both platforms were head-to-head, which should generally be the case for arrays of other data types. Although the use of UTF-8 might have some advantages in Europe and the USA, in countries of the Middle East or Asia this might be a drawback as well, because their languages consist of characters that are stored with three bytes. In the last scenario, Java RMI profited from the worse serialization mechanism of .NET. This is a major issue not only with .NET Remoting, but with the complete .NET platform. The comparison with our own library shows that the use of Java RMI causes only a small overhead, while the overhead caused by using .NET Remoting is much greater, depending on the scenario. This shows, that Java RMI is the best way to do remote method invocations in Java, whereas using custom mechanisms are maybe the better choice for .NET. Considering the arguments against the use of HTTP and the better results of TCP, the latter should be used if possible. In this case, Java RMI is the better choice, not only considering the measurement results, but also other advantages of Java, e.g. platform independence. Possible areas for future work are: (1) The examination of the bad performance of .NET Remoting compared to our own implementation and in combination with overloaded remote methods. (2) The results of the serialization tests indicate that .NET still has problems with serialization. This should also be investigated for the current versions of .NET, since it is a general disadvantage of the platform that does not only concern .NET Remoting. (3) A comparison of Java RMI and .NET Remoting with other middleware technologies like Web Services, involving heavy-load situations with more than one client.

References [1] W. Elbers, F. Koopmans, and K. Madlener. Java RMI and .Net Remoting Performance Comparison. http://www. cs.ru.nl/˜marko/onderwijs/oss/, 2004. [2] M. Hericko, M. B. Juric, I. Rozman, S. Beloglavec, and A. Zivkovic. Object Serialization Analysis and Comparison in Java and .NET. ACM SIGPLAN Notices, 38(8):44–54, 2003. [3] F. Huet, D. Caromel, and H. E. Bal. A High Performance Java Middleware with a Real Application. In Proceedings of the 2004 ACM/IEEE Conference on Supercomputing, 2004. [4] M. B. Juric, B. Kezmah, M. Hericko, I. Rozman, and I. Vezocnik. Java RMI, RMI Tunneling and Web Services Comparison and Performance Analysis. ACM SIGPLAN Notices, 39(5):58–65, 2004.

[5] M. B. Juric, I. Rozman, and S. Nash. Java 2 Distributed Object Middleware Performance Analysis and Optimization. ACM SIGPLAN Notices, 35(8):31–40, 2000. [6] J. Maassen, R. v. Nieuwpoort, R. Veldema, H. Bal, T. Kielmann, C. Jacobs, and R. Hofman. Efficient Java RMI for Parallel Programming. ACM Transactions on Programming Languages and Systems, 23(6):747–775, 2001. [7] J. Maassen, R. v. Nieuwpoort, R. Veldema, H. E. Bal, and A. Plaat. An Efficient Implementation of Java’s Remote Method Invocation. In Proceedings of the 7th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pages 173–182, 1999. [8] Microsoft. Microsoft Knowledge Base, 196271. http:// support.microsoft.com/kb/196271/EN-US/. [9] Microsoft. MSDN Library - .NET Remoting. http://www.microsoft.com/germany/msdn/ library/net/NETRemoting.mspx. [10] Microsoft. MSDN Library - QueryPerformanceCounter Function. http://msdn2.microsoft.com/ en-us/library/ms644904.aspx. [11] Microsoft. .NET Enterprise Services. http: //msdn.microsoft.com/webservices/ enterpriseservices/default.aspx. [12] Microsoft. .NET Pet Shop. http://msdn2. microsoft.com/en-us/library/ms978487. aspx. [13] C. Nester, M. Philippsen, and B. Haumacher. A More Efficient RMI for Java. In Proceedings of the ACM 1999 Conference on Java Grande, pages 152–159, 1999. [14] R. v. Nieuwpoort, J. Maassen, R. Hofman, T. Kielmann, and H. E. Bal. Ibis: an Efficient Java-based Grid Programming Environment. In Proceedings of the 2002 Joint ACMISCOPE Conference on Java Grande, pages 18–27, 2002. [15] Sun Microsystems. Java EE at a Glance. http://java. sun.com/javaee/. [16] Sun Microsystems. LE - JiniTM Distributed Leasing Specification. http://java.sun.com/products/jini/ 2.1/doc/specs/html/lease-spec.html. [17] Sun Microsystems. Remote Method Invocation (RMI). http://java.sun.com/javase/ technologies/core/basic/rmi/index.jsp. [18] Sun Microsystems. Sample Application Design and Implementation. http://java.sun.com/blueprints/ guidelines/designing_enterprise_ applications_2e/sample-app/sample-app1. 3.1.html. [19] Sun Microsystems. Java Remote Method Invocation Specification, 2004. http://java.sun.com/j2se/1.5/ pdf/rmi-spec-1.5.0.pdf.