was written for the following operating systems: Windows. NT 4.0, Windows 95, Linux 2.1, and MS-DOS. The first program, sender, sends Ethernet test frames to ...
Performance of Distributed Systems Based on Ethernet and Personal Computers
Andro Milanović, Siniša Srbljić, and Joško Radej University of Zagreb School of Electrical Engineering and Computing Unska 3, 10000 Zagreb, Croatia {andro, sinisa, jole}@zemris.fer.hr
Abstract - In recent years, many real-time applications have emerged on the Internet. Among other types of networks, the Internet also contains a large number of local area networks (LANs) that are based on personal computers and Ehternet. Real-time systems designed for the Internet must consider the impact of this technology. The designers of these systems have a number of computer and network configuration options available and are faced with the problem of choosing the appropriate one. Since it is difficult to find performance results that cover real-time requirements for Ethernet-based personal computer systems, we have developed simple measurement software to obtain performance data. In this paper we present the measurements obtained and discuss how the data transmission speed is affected by different components of a distributed system. We show that if the appropriate components are chosen, it is possible to build a distributed system, based on Ethernet and personal computers, that has a high data transmission speed and that can efficiently use Ethernet bandwidth.
1.
INTRODUCTION
The Internet has experienced extraordinary growth. In recent years, many real-time applications have emerged such as teleconferencing, computer telephony, multimedia broadcasting, real-time control of distributed systems, and distributed simulations. These applications require constant and predictable bandwidth and latency. Since LANs used in Internet are mostly based on Ethernet and personal computers, real-time systems should use this technology as well. However, designers of real-time systems are faced with the problem of choosing an appropriate computer and network configuration. Since it is difficult to find performance results that cover real-time requirements for Ethernet-based personal computer systems, we present simple measurements of data transmission speed in this paper. We present the measurements for different configurations of systems based on Ethernet and personal computers. We vary the type of processor, operating system, operating system networking configuration, network interface card, and Ethernet frame size used. The authors in paper [1] present similar measurement results. However, they tested distributed system based on UNIX workstations (DECstation 5000 and SparcStation 1) in order to create a low-latency RPC-based communication.
In Section 2, we describe the measurement software and the procedure used to generate the performance results. The different system configurations used in measurements are described in Section 3. Section 4 presents the performance results, and Section 5 gives a performance comparison of the different configurations. 2.
MEASUREMENT SOFTWARE
The measurement software is simple and consists of two programs: sender and receiver. The measurement software was written for the following operating systems: Windows NT 4.0, Windows 95, Linux 2.1, and MS-DOS. The first program, sender, sends Ethernet test frames to a certain Ethernet address. The frames can be sent at different speeds and they can be of different sizes. The second program, receiver, receives and counts the test frames. Fig. 1 shows the configuration of the measurement system. In all measurements, we used the fastest computer, a Pentium 200 system, as the sender.
sender
receiver
10 Mb/s Ethernet Fig. 1: The measurement configuration
The programs provide three metrics: the number of frames sent, the time needed to send the frames, and the number of frames received. Based on this information it is possible to calculate the sending speed, the percentage of received frames, and the receiving speed. Sending and receiving speeds are expressed in bits per second. Since the data transmission speed depends on frame size, this information is also provided. 3.
MEASUREMENT ENVIRONMENT
We have created a number of measurement environments that use a wide range of system parameters: processor (Intel 486 DX2 running at 66 MHz and Pentium at 133 MHz), operating system (Windows NT 4.0, Windows 95, Linux 2.1.132, and MS-DOS), operating system networking configuration for Windows NT
(NetBEUI, TCP/IP, and client services), network interface cards (D-Link DFE500TX, D-Link DE530+, 3Com Etherlink3, Compex RL2000, Western Digital SMC, NE2000 compatible), and Ethernet frame size (64 and 1518 bytes). For each configuration we measure how the receiving speed and the percentage of received frames depend on the sending speed. We measured the sending and receiving speeds for four frame sizes: 64, 150, 1000, and 1518 bytes. Using Ethernet, the smallest possible frame is 64 bytes long (46 bytes of data), and the largest possible frame is 1518 bytes long (1500 bytes of data). Since Internet protocols like HTTP [2-4] typically use two sizes of packets (control packets and data packets), we chose frame sizes of 150 and 1000 bytes to represent control and data frames, respectively. In this paper, we present only the results obtained for 64-bytes frame size, except for discussion on how the frame size affects the speed. We compare two frame sizes for this purpose, 64 bytes and 1518 bytes. 4.
RESULTS
Each set of measurements is presented by two graphs. The first graph shows how the receiving speed depends on the sending speed. The sending speed is given on the xaxis, while the receiving speed is given on the y-axis. The second graph shows how the percentage of received frames depends on the sending speed. The sending speed is given on the x-axis. The y-axis gives the percentage of received frames, which is calculated as the number of received frames over the number of sent frames. Although each of these values, receiving speed and percentage of received frames, can be calculated from the other one, we present both graphs. We use the first graph when discussing the data transmission speed, and we use the second graph when discussing bandwidth capacity. Figures 2 and 3 compare two hardware configurations, one using the Intel 486 DX2 processor and the other using the Intel Pentium processor. Fig. 2 presents the results for a 64-byte frame size and Fig. 3 presents the results for a 1518-byte frame size. Since it is very difficult to find the exactly same hardware configurations that use different processors, Table 1 presents the details of the tested configurations. In addition, both systems were running Windows NT 4.0 Workstation.
TABLE 1: HARDWARE CONFIGURATIONS Processor speed (MHz) Cache size (KB) RAM (MB) Type of RAM Bus
Intel 486 DX2 66 128 20 70 ns ISA
Intel Pentium 133 256 32 60 ns, EDO PCI
The results in Fig. 2 show that the Pentium system is much better than the 486 system when receiving 64-byte frames. Fig. 3 shows that the difference between these two systems decreases if the larger packet size is used. In addition, Figures 2 and 3 show that both systems achieve better results if larger frame sizes are used. Fig. 2a shows that for 64-byte frames and for sending speeds lower than 0.5 Mb/s results are similar. If the sending speed is greater than 0.5 Mb/s, then the difference between the Pentium and the 486 increases. The speed of the Pentium processor increases, while the speed of the 486 processor decreases. The difference in speed could be of an order of magnitude. The speed of the Pentium system begins to fall at 3.5 Mb/s. Fig. 3a shows that for the 1518-byte frame size, the speed of the 486 system does not significantly differ from the speed of the Pentium system if the sending speed is less than 7.5 Mb/s. By comparing Figures 2a and 3a, we can conclude that the difference in speed is much smaller if larger frames are used. In our measurements, we used 10 Mb/s Ethernet. Neither the 486 nor the Pentium system can use the full Ethernet capacity of 10 Mb/s when 64-byte frames are used. This is shown in Fig. 2b. The system based on the 486 processor has a maximum speed of 0.5 Mb/s which translates to only 5% of capacity. The Pentium system has a maximum of 3.3 Mb/s, which is 33% of capacity. For 1518-byte frame size, the 486 system has a maximum of 6.7 Mb/s using 67% of capacity. The Pentium system can use the 100% of Ethernet capacity for this frame size.
Frame size of 64 bytes
Frame size of 64 bytes
120%
10,000
Percentage of received packets
Receiving speed (kb/s)
9,000 8,000 7,000 6,000 5,000 4,000
Pentium 133
3,000 2,000
486 DX2 66
1,000
100% 80% Pentium 133
60% 40%
486 DX2 66
20% 0%
0 0
2,000
4,000
6,000
8,000
0
10,000
1,000
2,000
3,000
4,000
5,000
6,000
7,000
8,000
9,000
10,000
Sending speed (kb/s)
Sending speed (kb/s)
(a)
(b)
Fig. 2: The comparison of hardware configurations that use different processors for 64-bytes frame size Frame size of 1518 bytes
Frame size of 1518 bytes
10,000
120%
Percentage of received packets
Receiving speed (kb/s)
Pentium 133
Pentium 133
9,000 8,000 7,000 6,000 5,000 4,000
486 DX2 66
3,000 2,000 1,000 0
100% 80%
486 DX2 66
60% 40% 20% 0%
0
1,000
2,000
3,000
4,000
5,000
6,000
7,000
8,000
9,000 10,000
0
1,000
2,000
Sending speed (kb/s)
3,000
4,000
5,000
6,000
7,000
8,000
9,000
10,000
Sending speed (kb/s)
(a)
(b)
Fig. 3: The comparison of hardware configurations that use different processors for 1518-bytes frame size
Frame size of 64 bytes
Frame size of 64 bytes 120%
6,000
Receiving speed (kb/s)
Percentage of received packets
Linux 2.1.132 Linux 2.1.132
5,000 4,000
MS-DOS
3,000
Windows NT 4.0
2,000 1,000 Windows 95 0
100% MS-DOS 80% Windows NT 4.0
60% 40%
Windows 95
20% 0%
0
1,000
2,000
3,000
4,000
5,000
6,000
0
1,000
Sending speed (kb/s)
2,000
3,000
4,000
5,000
6,000
Sending speed (kb/s)
(a)
(b) Fig. 4: Operating systems comparison
The comparison of four different operating systems is given in Fig. 4. The speed was measured for a 64-byte frame size, with the same Pentium 133 system and the same network card. Fig. 4a shows that there is no difference in results for speeds of less than 2 Mb/s. Windows 95 and Windows NT have similar speeds for all values of sending speed except for an interval from 2 Mb/s to 3 Mb/s. At the beginning of this interval, the speed of Windows 95 drops significantly, but increases at the end of the interval. Outside of this interval, there is only a small difference between Windows 95 and Windows NT. The speed of Linux and MS-DOS does not start to fall until 5.2
Mb/s, whereas the speed of both Windows systems begins to fall after 3 Mb/s. Fig. 4b shows that Linux and MS-DOS can use over 55% of capacity for the 64-byte frame size, while Windows NT cannot use more than 33% of capacity. Although Windows 95 can receive up to 3.3 Mb/s that is 33% of capacity, it has the significant drop of speed at 1.9 Mb/s, which makes the Windows 95 unreliable for speeds higher than 1.9 Mb/s.
Frame size of 64 bytes 1st configuration
Receiving speed (kb/s)
800
st
700 600 500 3rd configuration
400
4th configuration
300
Frame size of 64 bytes
120%
2nd configuration
Percentage of received packets
900
200 100 0
1 configuration
2
nd
configuration
100% 3rd configuration
80% 60%
4th configuration
40% 20% 0%
0
100
200
300
400
500
600
700
800
900
1,000
0
100
200
300
Sending speed (kb/s)
400
500
600
700
800
900
1000
Sending speed (kb/s)
(a)
(b)
Fig. 5: The comparison of different networking configurations in Windows NT
Frame size of 64 bytes D-Link DFE530+ PCI
700 3Com Etherlink3 ISA
600 500 400
Compex RL2000 ISA
300 200
NE2000 Compatible ISA
WD SMC ISA
100 0
D-Link DFE500TX PCI
Percentage of received packets
D-Link DFE500TX PCI
Receiving speed (kb/s)
Frame size of 64 bytes
120%
800
D-Link DFE530+ PCI
100% WD SMC ISA
80% 60%
Compex RL2000 ISA
40%
NE2000 Compatible ISA 3Com Etherlink3 ISA
20% 0%
0
100
200
300
400
500
600
700
800
900
1000
0
100
Sending speed (kb/s)
200
300
400
500
600
700
800
900
1000
Sending speed (kb/s)
(a)
(b) Fig. 6: Network cards comparison
The complexity of the microkernel architecture results in lower speed for Windows NT. The overall architecture of Windows NT, which is modular and uses NDIS1 for network communication, generates significant overhead for frequent and small tasks. There is a long chain of events (interrupts, function calls, and IPC2 procedures) that need to happen before an application program can send or receive an Ethernet frame. Linux provides much better results, because it is a more simple operating system with all drivers residing inside the kernel. The measurement for 1518-byte Ethernet frames shows that there is no significant difference between operating systems if the Ethernet frames are large. To determine how different networking configurations of Windows NT affect the results, we tested four different networking configurations as described in Table 2. The results for different networking configurations are shown in Fig. 5. In the first configuration, no
1 Network Driver Information Specification - a standard for writing network drivers, which should be independent of operating system and network technology used. 2
Inter-Process Communication
communication protocols and services are installed. For each of the following configurations, the speed falls as protocols or services are added. By adding NetBEUI protocol or client services, the speed does not change significantly. Moreover, the speed increases if we add the client services. The only significant slowdown occurs if the TCP/IP protocol is installed. These results confirm that the networking configuration of operating system affects the data transmission speed by up to 20%. To test different network cards, we use the same operating system Windows NT 4.0 and the same computer hardware configuration. The graphs for this group of measurements are shown in Fig. 6. TABLE 2: NETWORKING CONFIGURATIONS Configuration 1 2 3 4
Installed protocols and services No protocols and clients NetBEUI protocol NetBEUI and TCP/IP protocols NetBEUI, TCP/IP protocols and client services
Each graph has six plots, two of them represent the results for the PCI cards used and four of them represent the results for the ISA cards used. All of the PCI cards produce similar results and have higher speeds than the ISA cards. The difference between the PCI cards and the best ISA card is not large; it is lower than 5%. The worst ISA card has very poor results and it can use less than 55% of capacity of the best PCI card. These results indicate that the difference between the ISA and the PCI cards is not significant, but that the chipset used in network card significantly affects the performance. 5.
CONCLUSION
Although designers are forced to use Ethernet-based personal computer systems for real-time applications, it is hard to find even simple performance results for such systems, such as data transmission speed. In addition, it is difficult to determine how different components of a distributed system affect the data transmission speed. In order to find how processors, operating systems, operating system networking configurations, network interface cards, and Ethernet frame sizes affect the data transmission speed, we decided to run a number of simple measurements on different system configurations. The results of the research presented in this paper show a large difference between different distributed system configurations for small Ethernet frame sizes (for example 64 bytes). The difference is less significant if larger frames are used (for example 1518 bytes). The difference in speed for small frame sizes can be up to an order of magnitude. For example, the Intel 486 processor based distributed systems can achieve only 10% of the data transmission speed of the systems based on Pentium 133. In addition, the results for different operating systems show that data transmission speed for the distributed systems based on Linux is more than 60% faster than the speed of the systems based on Windows. Some type of network cards can also significantly affect the data transmission speed. The difference between best and worst ISA card used is 50%. However, the difference between the best ISA card and the worst PCI card is less than 5%.
The difference for different networking configurations in Windows NT is not significant. Different configurations consist of different sets of communication protocols. By installing more levels of communication protocols, the data transmission speed could be slowed down by up to 20%. The measured performance results presented in this paper show that it is possible to build a distributed system based on Ethernet and personal computers that has high data transmission speed and efficiently uses the Ethernet bandwidth. To obtain high data transmission speed for small frames, the components of distributed system should be chosen carefully. However, for large frames the configuration of the system does not significantly affect the data transmission speed. Since the measurement is time-consuming, we plan to improve our measurement software. The changes will allow the measurement software to run automatically. In addition, we are planning to test more components, including different network packet drivers. 6.
ACKNOWLEDGMENTS
The authors wish to thank J. Gracin for his help in constructing the measurement software for Linux, and A. Knez, I. Benc, M. Štefanec, M. Matić, M. Blažević, M. Kralj, D. Ivošević for their help in conducting the measurements. References [1]
Chandramonah A. Thekkath and Henry M. Levy: "Limits to Low-Latency Communication on High-Speed Networks", ACM Transactions on Computer Systems, Vol. 11, No. 2, May 1993, pp. 179-203.
[2]
Douglas E. Comer: “Internetworking with TCP/IP”, Vol. 1, Prentice Hall Inc., 1991
[3]
T. Socolofsky, C. Kale: “A TCP/IP Tutorial”, RFC 1180, Spider Systems Limited, January 1991
[4]
R. Fielding, et. al: “Hypertext Transfer Protocol - HTTP/1.1”, RFC 2068, UC Irvine, January 1997
