CARRYING ATM CELLS OVER ETHERNET. José Manuel Arco, AgustÃn MartÃnez,. Bernardo Alarcos, Antonio GarcÃa, Daniel Meziat. Dpto de Automática.
CARRYING ATM CELLS OVER ETHERNET
José Manuel Arco, Agustín Martínez, Bernardo Alarcos, Antonio García, Daniel Meziat Dpto de Automática. Escuela Politécnica. Universidad de Alcalá. Crta. Madrid-Barcelona, Km. 33,600 28871 Alcalá de Henares (Madrid) Spain e-mail: {jmarco, hellin, bernardo, antonio, meziat}@aut.alcala.es Fax: +34 91 8856641
Topics areas: • Multimedia systems architectures • Telecommunication systems for Multimedia • Hardware/Software for Multimedia • Network and operating systems for Multimedia applications • Specification and management of Quality of Service (QoS) constraints
"Neither this paper nor any version close to it has been or is being offered elsewhere for publication. All necessary clearances have been obtained for the publication of this paper. If accepted, the paper will be made available in camera-ready forms by June 14th, 1999, and it will be personally presented at the EUROMICRO 99 Conference by the author or one of the co-authors. The presenting author(s) will pre-register for EUROMICRO 99 before the due date of the camera-ready paper."
1
CARRYING ATM CELLS OVER ETHERNET
José Manuel Arco, Agustín Martínez, Bernardo Alarcos, Antonio García, Daniel Meziat Dpto de Automática. Escuela Politécnica. Universidad de Alcalá. Crta. Madrid-Barcelona, Km. 33,600 28871 Alcalá de Henares (Madrid) Spain e-mail: {jmarco, hellin, bernardo, antonio, meziat}@aut.alcala.es Fax: +34 91 8856641
Abstract This paper describes the research carried out to obtain the advantages of ATM technology over legacy Ethernet networks. A protocol called “Cells In Frame” (CIF) allows to encapsulate ATM cells in Ethernet frames. We have implemented the CIF protocol in the kernel of the Linux operating system. Measures of speed, delay and delay variation have been carried out in order to check that the developed system fulfils the requirements of the multimedia traffic. Obtained results show that it provides the necessary ATM quality of service, paying a minimum throughput decrease respect to native Ethernet. Also a queuing algorithm that allows to optimize of traffic management is being researched.
1. Introduction The global objective of our research consists on providing the advantages of ATM quality of service to classic networks as Ethernet. It is desired to bring the advantages of this technology to the user without big investments in migration to ATM [1]. The scenario in which this development is being applied, is the used in most of the work environments, networks LAN interconnected by WAN. The current tendency in WAN networks is the use of the ATM (Asynchronous Transfer Mode) technology [2][3], although as it is well-known, this technology offers big advantages. For Local Area Networks (LAN) the dominant technology nowadays is Ethernet, since it parts with the advantage of being a cheap, well-known and broadly spread technology (more than 75% of the current LAN networks are Ethernet). It is not likely that ATM will impose in this environment in the medium term [4] because it has the disadvantage of requiring bigger investments for switching and the network adapter, besides being a more complex technology. However, Ethernet presents the disadvantage of not being adapted for video and audio transmission, since it doesn't offer guarantee of quality of service. To overcome this difficulty we have decided to introduce some variations on Ethernet network to endow it with the benefits of ATM, without losing the own ones. A possibility is the use of a new protocol that integrates the local network inside ATM, this protocol is the called Cells In Frame" (CIF) [5]. The installation of CIF requires two actions, adding software in the final systems, and the development of a special switch that allows to connect both technologies. The generic topology is illustrated in figure 1. The CIF protocol on the PC introduces a head and the payload of several ATM cells inside each Ethernet frame. The function of the CIF switch is extracting the payload of ATM cells from Ethernet frames, to reconstruct their head and to send them to the ATM network. Using CIF, what we get is an ATM signaling arriving to the user while maintaining the Ethernet technology.
2
In section 2 of the present article the CIF architecture is described in detail, in section 3 the carried out implementation is exposed, in section 4 the tests and the obtained results are presented, concluding in section 5 with the summary and future works.
Video
Video
PC-CIF
PC-CIF WAN ATM
Ethernet
Ethernet
CIF Switch
CIF Switch
PC-CIF
PC-CIF
Figure 1. Network Topology.
2. Architecture of CIF Protocol CIF can be considered like an implementation of ATM in which the physical layer is Ethernet. The architecture of protocols to develop, represented in figure 2, should fulfil the CIF specification 1.0 [6] of the CIF Forum. APLICATION
APLICATION
CS: AAL5
CS: AAL5
SAR
SAR
AAL
ATM-CIF
ETHERNET
PC-CIF
ATM-CIF
ETHERNET
ATM-CIF ATM Physic Layer
ATM Physic Layer
CIF SWITCH
ETHERNET
CIF SWITCH
AAL
ATM-CIF
ETHERNET
PC-CIF
Figure 2. Protocols Stack. The stack of protocols ATM-CIF is similar to ATM stack [7], with the difference that the physical level is Ethernet. Next the particularities of each level of CIF respect ATM are described in short. The ATM adaptation layer, AAL (ATM Adaptation Layer) provides a standard interface at superior levels that can be directly native ATM applications, or another stack of protocols like TCP/IP. For the convergence sublayer CS, although with CIF we can use any AAL, we will use AAL5 because is the most efficient. The segmentation and reassembled sublayer (SAR) chops the information in 48 bytes blocks that form the cell. As it is explained in the next paragraph, this function is not carried out in CIF except when the length of data block is bigger than 1480 bytes, because 31 payload cells fill a frame. The normal ATM-CIF layer does not need to generate a head for each cell as ATM layer does. Instead it generates only one head for a group of cells belonging to the same connection and that will be encapsulated in same Ethernet frame, for efficiency sake.
3
The CIF switch should carry out several functions: • User Parameters control (UPC) simulates the police function of the ATM network, to check that the received traffic agrees with the conventional speed. It uses the leaky bucket algorithm. • Cells extraction from Ethernet frame and their insertion in the ATM network. • Administration of the output queues to guarantee the quality of service. • Maintenance of connection with final systems. Figure 3 illustrates the flow of data for the different levels, supposing that the message is smaller than 1480 bytes.
DATA
APLICATION
DATA
AAL5
CIF Head
ATM CIF
ETHERNET
ETHERNET Head
CIF Head
ATM Head
ATM Head
AAL5 Tail
ATM Payload
ATM Payload
ETHERNET Tail
Figure 3. Data flow. The switch and the final system CIF basically use the CIF header to determine who and how will carry out a series of ATM procedures. It is also used to distinguish between control or data information.
3. Implementation of the CIF norm 3.1. Scenario The implementation of a final system and a CIF switch is developed, in accordance with the outline of figure 1. The development has been carried out under the Linux operating system, to be a Unix system of public domain, very diffused and also has sources, debuggers and documentation for development. It has also been kept in mind that implementation does not exist. The first phase of the implementation is according to figure 4, which is a simplification of the network of figure 1. The simplification consists on connecting the two PCs for a unique CIF switch. We should note that this is not the real scenario, but it is valid since the CIF switches of figure 1 should guarantee the QoS, the same as the ATM network. The guaranteed QoS of the CIF switch will be the same as if it were an ATM network. The connection between the PC and the switch CIF is Ethernet half-duplex. Sometimes problems arise related to environment sharing with high loads due to this fact. There are two solutions to avoid this effect: implementing a connections admission control or using a fullduplex Ethernet connection. It is also a real scenario when quality of service in LAN-LAN connections is a priority.
4
Video
Video
PC-CIF
PC-CIF
CIF Switch
Figure 4. Considered scenario.
3.2. Description of the implementation The basic aspects of the CIF norm has been implemented without including some aspects in this first phase, for example the errors treatment of the ATM head (HEC) since in practice they are not frequent, and when there are errors, detection is done at Ethernet level. This first phase supports the CBR and UBR traffic types. 3.2.1 Modular entity The implementation is made in the kernel, so that it can be used efficiently by several applications [8], although it means a more difficult development. To make the development of the entity more efficient, it takes advantage of the capacity of the Linux kernel to load and to unload modules dynamically, so that every time that a change is written in the entity it is not necessary to compile again all kernel and to reboot the machine, it is only necessary to unload the module, to compile it and load it again [9]. Previously it is necessary to compile again the kernel indicating that a new protocol exists and that it will be loaded in a modular way. 3.2.2 Linux protocols stack To make the implementation in the kernel, it has taken advantage the scheme that it is used in protocols implemented in Linux, adding to the families [10], a new one (AF_CIF) and a specific structure of addresses (sockaddr_cif). In this initial phase sockets not connected orientated are used. BSD socket INET socket TCP
UDP CIF IP dev.c specific driver
Figure 5. Stack implemented. When a process communicates via the network, it uses the functions provided by the BSD [11] socket layer. This takes care of a range of tasks similar to those handled by the Virtual File System and administers a general data structure for sockets which we shall call BSD sockets. The BSD socket simplifies the porting of network applications.
5
Below this layer is the INET socket layer. This manages the communication end points for the CIF protocol. This is represented by the data structure socket, which we shall call INET sockets. The layer under the INET layer is the CIF protocol stack. Below the CIF layer are the network devices that have a common interface (dev.c) and an specific part, the network drivers. The interaction between the different layers is illustrated in figure 5, and the relationship of files that take part in the transmission of data is showed in figure 6. User Aplication
User Aplication
Send data
Receive data
socket.c
socket.c
af_cif.c
af_cif.c
datagram.c
dev.c
dev.c
network driver
network driver
Figure 6. Files. The data have been copied only two times: from the user area of the user process to kernel memory, then to the network card. In the Linux implementation of the network code a great deal of care has been taken to avoid unnecessary copy operations. The functions implemented in the CIF switch are the establishment of the connection, the frames forwarding, and the queue management. 3.2.3 CIF switch We have implemented a software switch that supports QoS. For it, the used algorithms are those of the family Weight Fair Queuing WFQ [12][13], in short the self-clocked fair queuing SCFQ has been chosen [14], for its simplicity. In this phase, the signaling for the connection parameters establishment and specification is made in a manual way. There is a series of permanent circuits and each connection parameters are sent to the CIF switch from the application before sending data. In the standard operation of Linux, all the messages would follow the same treatment through an only queue. This way, if we sends a CIF message, it will pass through the functions dev_queue_xmit () and do_dev_queue_xmit () that put in a queue the frame, and through the function dev->hard_start_xmit () that calls the specific driver of the network interface. In order to obtain QoS we have modified the previous outline, adding so many lines like connections there are and implementing the SCFQ algorithm to manage them. The figure 7 show the basic schema to own development. We must to highlight that we have modified fields in the device structure that is the one that contributes the necessary functions for the communication with the network drives [15]. The fundamental modification is due to the increment of the number of lines in the field DEV_NUMBERS. In order to manage queues we must use the array sk_buff_head buffs [DEV_NUMBERS]. 6
Each frame sk_buff struct contain a frame. A queue is formed by structures doubly connected sk_buff that are negotiated by the structure sk_buff_head. In order to manage the queue, we use the functions skb_dequeue() and skb_queue_tail () for queuing and dequeuing respectively [16]. The input algorithm classifies the frames and calculates the out label. The out algorithm selects to send frame that have the lower out label. dev_queue_xmit()
do_dev_queue_xmit()
af_cif.c
dev.c
input algorith skb_queue_tail()
....
sk_buff_head
skb_dequeue() output algorith
dev->hard_start_xmit()
device driver
Figure 7. Queuing scheduling.
4. Tests All the tests have been carried out using a switching CIF and two final systems CIF (figure 4), the equipment used is a PC with Pentium 133 MHz, 16 Mbytes RAM and the operating system used is Linux network Hat 4.2. To compare the results, the same tests have been accomplished in the same network but using only Ethernet. The tests carried out are the typical to obtain the parameters required by the traffic multimedia [17]: • Transfer speed (throughput)[18]. • Transmission Delay. • Variation of transmission delay (jitter). All the tests have been achieved both unidirectional and bidirectional ways.
4.1. Methodology of the tests All the measures are carried out during a period of 20 seconds, which is big enough to avoid the effect of the execution of some Linux daemon. To carry out the tests we have taken the following significant sizes of user's messages, 40, 64, 128, 256, 512, 1024, and 1480 bytes. The size of 40 bytes is the maximum to fill a cell ATM. Then we take 64 bytes like base to double it until arriving at 1024, the last value is the one that corresponds with the maximum message that fits in a frame with CIF information (31 cells). 4.1.1 Throughput tests. User's process in a PC sends a burst of data in a continuous way to a certain speed during 20 seconds. The receiver takes the beginning and final times of the burst, it counts the received 7
bits and calculates the transfer speed from these data. This process repeats at increasing speeds, until it loses some messages, then we have the throughput. 4.1.2 Delay tests. Measures of Round Trip Time (R.T.T.) average have been carried out, in a period of 20 seconds. The latency is approximately half of the R.T.T. To avoid the error that causes the loss of messages, these are sent with sequence numbers to check that the received and emitted message is the same. 4.1.3 Delay variation tests To find out the delay variation or jitter, messages are sent in a periodic way during the test interval. Each message is sent with a mark of the emission time, so the receiver calculates the emission interval. In the receiver the arrival interval is calculated, subtracting the arrival times of consecutive messages. Subtracting both intervals we obtain a jitter sample. This process is repeated with the rest of the messages and we calculate the average. We also use sequence numbers to avoid the effect of messages loss.
4.2. Obtained results 4.2.1 Transfer speed. The throughput has been measured for different sizes of messages, and it has been observed that it is limited by the capacity of the switch. It has been proven that if emission speed increases above the throughput, the reception speed decreases, as it was expected. We should keep in mind that user's maximum speed is a function of the size of the message and heads, for Ethernet the theoretical limits maximum goes from 5,48 Mbps with a size of 40 bytes to 9,75 Mbps with 1480 bytes. For CIF among 4.25 Mbps with 40 bytes and 9,65 Mbps for 1480 bytes. In figures 8 and 9 the throughput results are presented. The theoretical maximum speed of Ethernet is also represented. We observed that the maximum speed in CIF is a little less than that of Ethernet. This is because the switch CIF should carry out additional functions to those of a switch Ethernet. The loss of speed in CIF regarding Ethernet is less than 0.8 Mbps.
1480
Message size
1024
512
256
64
128
40
10 9 8 7 6 Traffic (Mbps) 5 4 3 2 1 0
40
64
128
256
512
1024
1480
CIF throughput
3,1
3,94
6,2
7,63
8,89
9,26
9,46
Ethernet throughput
3,13
4,22
6,44
8,37
9,31
9,61
9,72
Maximun throughput
5,48
6,48
7,79
8,73
9,32
9,64
9,75
Figure 8. Unidirectional throughput.
8
Message size
1480
1024
512
256
128
64
40
5,00 4,50 4,00 3,50 3,00 Traffic (Mbps) 2,50 2,00 1,50 1,00 0,50 0,00
40
64
128
256
512
1024
1480
CIF throughput
0,62
0,75
1,36
1,95
2,64
3,04
3,89
Ethernet throughput
1,06
1,57
2,13
2,87
3,34
3,52
3,99
Maximun throughput
2,74
3,14
3,86
4,35
4,65
4,82
4,87
Figure 9. Bidireccional throughput. 4.2.2 Delay. The figure 10 shows the results of the delay, comparing them with transmission values in a Ethernet segment. 14,00 12,00 R.T.T. (msec.)
10,00 8,00 6,00 4,00 2,00 0,00
40
64
128
256
512
1024
1480
Unidireccional CIF
0,96
0,97
1,12
2,03
3,28
6,26
8,63
Unidireccional Ethernet
0,61
0,72
1,07
1,78
3,22
6,06
8,61
Bidireccional CIF
0,96
1,21
1,54
2,53
4,23
8,26
11,50
Bidireccional Ethernet
0,85
1,12
1,42
2,22
3,66
6,87
8,96
Minimum R.T.T.
0,25
0,33
0,53
0,94
1,76
3,40
4,86
Message size
Figures 10. Delay. The obtained results show that the CIF delay is a little longer than the Ethernet delay. These results were forecast because the entity CIF and the switch carry out a bigger quantity of tasks. It is also observed that the delay increases when the size of the message is increased too; this is because the operating system spends more time in the processing of biggest messages. 4.2.3 Delay variation. The unidirectional jitter is very small (12 µsec. as maximum) and constant since it does not vary with the emission speed neither with the size of the message. This result was anticipated because there are not other causes that produce a deviation in the delay, except for the execution of some processes. 9
900
Jitter microsec.
800 700 600 500 400 300 200 100 0 0
50
100
150
200
250
300
350
400
450
500
Samples
Figures 11. Bidireccional Jitter. The bidirectional jitter increases (200 µsec.) fundamentally due to two reasons: first, collisions that take place when the final system and the switch try to transmit at the same time, and second, that both the final system and the switch must wait to finish receiving before they can start to transmit. Figure 11 represents samples of the jitter measured for a size of message of 1480 bytes and 3,57 Mbps. These results are much smaller than other measured [19] in Ethernet networks, since there are only two machines in each segment.
4.3. Analysis of results. The results of speed and delay are similar to those obtained in the implementations carried out by the University of Cornell, with equivalent scenarios on the operating systems Mac/OS, Windows 95 and Windows NT [20]. We should also highlight that the obtained speed (more than 2 Mbps) is adapted for applications that require quality of service, such as videoconference, fulfilling most standards [21]. The obtained delay, that does not reach 10 msec. in the worst cases, is much below the tolerances admitted for applications of videoconference that requires a latency smaller than 300 msec.
5. Conclusions and lines of future works. The introduction of the protocol CIF allows that guaranteed parameters of the traffic can be respected for different types of services. These guarantees are not possible using simple Ethernet, because with this technology, it is not possible to specify the requirements of each type of traffic. An implementation into the kernel of the operating system has been made, that allows it to be used by several processes. The support of quality of service in the switch has been implemented too. We are working in a suitable queuing algorithm for the Ethernet-ATM and Ethernet-Ethernet interconnections. To complete the real scenario, it is necessary the incorporation of interfaces ATM in the switches. It will be completed both the CIF specification and support of classic IP, that will allow communication between computers with applications over both TCP/IP and native ATM protocols.
10
References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]
[13] [14] [15] [16] [17] [18] [19] [20] [21]
J. M. Arco, A. Martínez, B. Alarcos, A. García, D. Meziat. “Quality of service over Ethernet”. Online Educa Berlin, 2-4 December 1998. V. Castelo, “Informe de la gestión de RedIRIS durante el año 1997”. RedIris Bulletin, nº 41-42, pp 5-7, December 1997. A. Tanenbaum "Computer Networks, third edition", Prentice Hall, 1997. J. McQuillan, “Convergencia de routes y conmutadores”. Global Communication, nº 10, pp 42-43, January 1998. L. Robers "CIF: Affordable ATM, At last" Data Communication, April, 1997. Cells In Frames Version 1.0, specification, http://www.ziplink.net/~lroberts/Atmf961104.html Prycker M. "Asyncronous Transfer Mode, Solution for Broadband ISDN, Third Edition", Prentice Hall, 1997. Armitage, J. "The Application of Asynchronous Transfer Mode to Multimedia and Local Area Network" PhD thesis, January 1994. University of Melbourne, Australia. http://www.li.org/Resources//HOWTO/Module_HOWTO.html, Lauri Tischler, v1.1, Octuber 1996. J.M. Arco, B. Alarcos, A. Domingo, " Programación de aplicaciones en redes de comunicaciones bajo entorno Unix". Edited by University of Alcalá, 1997. M. Beck, H Bohme, M Dziadzka, U Kunitz, R Magnus, D Verworner. “Linux kernel internals” Second Edition, Addison-Wesley, 1998. Hui Zhang, "Service Disciplines for Guaranteed Performance Service in PacketSwitching Networks" Proceeding of the IEEE, Vol 83, nº 10 October 1995, pp. 13741396. W. Stalling, “High-speed networks TCP/IP ATM design principles”, Prentice Hall, 1998. S Golestani, "A self-clocked fair queuing scheme for broadband applications" Proceeding of the IEEE INFOCO ’94 Toronto, CA, June 1994, pp. 636-646. Alessandro Rubini, “Linux device drivers”. O’Really & Associates, Inc.1998. Alan Cox, “Network Buffers and memory Management” Linux Journal, September 1996. F. Kuo, W. Effelsbeg, J. Garcia-Luna-Aceves “Multimedia Communications, Protocols and Applications” Prenticel Hall, 1998. S. Branner, “Bechmarking Terminology for Network Interconnection Devices”, RFC 1242. C. Venkatramani, “The design, implementation and evaluating of RETHER: Areal-Time Ethernet Protocol.” PhD thesis, January 1997. University of New York. http://www.cif.cornell.edu/CornellResults.html Keinath, R. And D. Minoli, “Distributed Multimedia Through Broadband Communications” Artech House, 1994.
11
