A Cross-Layer Optical Circuit Provisioning

SUN2 LAYOUT

1/22/08

1:55 PM

Page 56

TOPICS IN OPTICAL COMMUNICATIONS

A Cross-Layer Optical Circuit Provisioning Framework for Data Intensive IP End Hosts Weiqiang Sun, Guowu Xie, Yaohui Jin, Wei Guo, Weisheng Hu, Xinhua Lin, and Min-You Wu, Shanghai Jiao Tong University Wentao Li, Rong Jiang, and Xueqin Wei, Fiberhome

ABSTRACT Using circuit-switched optical networks for next generation e-science applications is gaining increasing interest. In such applications, circuits are provisioned for end hosts to accomplish data-intensive or QoS-stringent communication tasks. Existing provisioning methods provide point-to-point connectivity for end hosts, that is, an established circuit connects one end host to another, and during the lifetime of the circuit, only communication tasks between the connected end hosts can be served. This inhibits circuits from being used in more general cases, where each end host communicates with different remote parities simultaneously through a single network interface. We propose V-STONES — a data flow-based VLAN tagging and switching technique to increase the connectivity of end host network interfaces in circuit-switched networks. With V-STONES, not only can an IP end host communicate with different remote systems concurrently through bandwidth guaranteed connections, but also protocol entities at different stack layers can talk to their counterparts through dedicated bandwidth pipes. In this article, we review the existing circuit provisioning methods and then discuss V-STONES and the architecture of cross-layer circuit provisioning for end hosts. We also introduce a prototype implementation in an optical network testbed and present the experimental results.

INTRODUCTION Distributed storage, high performance computing, and next-generation e-science applications have long been research interests of the networking and computing communities. Such applications generally require that large volumes of data be transferred from one place to another, or a set of steering and control operations from a centralized node be distributed to visualization/computing nodes in a timely manner. Although research in grid computing has made tremendous advances in

S30

0163-6804/08/$25.00 © 2008 IEEE

connecting widely distributed resources using ubiquitous Internet infrastructure, the fact that the Internet is a shared packet-switched network and thus, is unable to provide the required bandwidth or quality of service (QoS) guarantee has generated much interest in building dedicated networks for that purpose. Because of their huge bandwidth and the guaranteed QoS performance, circuitswitched optical networks are regarded as excellent transport infrastructures for such applications. Given the dynamic and heterogeneous nature of applications, the following requirements make the problem of provisioning circuits to applications even more challenging. Provisioning must: • Meet the arbitrary communication requirements of applications, while at the same time maintain a high level of efficiency. • Provide a user-friendly interface, such that no or little additional complexity is incurred in application design. • Provide optimized performance for a variety of applications with different requirements. A finely tuned provisioning model that couples the applications with the network management system or control plane is essential to meet such requirements. Intensive research efforts have been made to build efficient provisioning models for different purposes based on different switching technologies. A widely and implicitly adopted assumption in these models is that communications in such applications are generally single-task based. At any specific time, there exists only one communication task in an end system, and it will consume all the provisioned bandwidth. This addresses a wide range of applications, characterized by large data transfer from one site to another. In this article, we also address the circuit provisioning problem for next-generation e-science applications. One factor that distinguishes our model from the existing ones is that we drop the assumption of a single communication task. End hosts are assumed to have arbitrary concurrent communication requirements with different remote parities. This can be regarded as a generalization of the aforementioned models. We fur-

IEEE Communications Magazine • February 2008

SUN2 LAYOUT

1/22/08

1:55 PM

Page 57

Networking technology

Provisioning method

Provisioned channel capacity

Applications

USN [5]

SONET/10 Gigabit Ethernet (core) and multiservice provisioning platform (edge)

Centralized scheduling and signaling within a VPN

10 Gb/s, 1 Gb/s, and high-precision channels such as SONET OC-1

Large data transfer

CHEETAH [6]

SONET (core) and multiservice provisioning platform (edge) with packet switching for dual homing

GMPLS with hardwareaccelerated signaling

10 Gb/s and 1 Gb/s

Large file transfer and remote visualizations

UCLP/CA*net4 [1]

Lightpath switching

Centralized provisioning through Web service

Wavelength (10 Gb/s) and subwavelength

High-performance computing

DRAGON [2]

DWDM switching (core) and Ethernet, TDM, IP (edge)

GMPLS with centralized broker

Wavelength, Ethernet, and IP

e-VLBI

SURFnet6 [3]

Lambda switching and packet switching

User controlled provisioning

Wavelength

Large data transfer

OptiPuter [4]

Lambda switching with packet switching for dual homing

Client provisioning through Web service

Wavelength

Distributed virtual computer (DVC)

OMNInet [7]

DWDM switching and L2/L3 devices

GMPLS with OIF UNI

10/100/1000 b/s

Grid applications

■ Table 1. Testbeds (Projects) related to provisioning circuits to end hosts (users).

ther present a framework that allows entities at different protocol layers on an end host to request dedicated bandwidth channels on their own. This provides opportunities for designing more sophisticated e-science applications where bandwidth assurance on a system-wide basis cannot meet the requirements. The rest of the article is organized as follows. First, we review existing circuit provisioning models and point out the connectivity issue caused by dedicated interface usage in such models. We also present a brief overview of available networking technologies that can provide end-to-end circuits. We then present the data flow-based virtual local area network (VLAN) tagging and switching approach, followed by the cross-layer provisioning mechanism that takes advantage of this approach. We introduce a prototype implementation and present the experimental results. Finally, we conclude this article and discuss future work.

PROVISIONING CIRCUITS TO ESCIENCE APPLICATIONS: A REVIEW Intensive research and development efforts on provisioning optical circuits for e-science applications have been reported in the recent literature. Testbeds using either all optical or hybrid electronic/optical switches are being deployed. Efforts have been made to address the challenges mentioned in the previous section. For example, in all the reported testbeds (projects), optical circuits either in the form of wavelength channels [1–4] or Ethernet connections [2, 5–7] are provisioned dynamically upon request. This enables bandwidth sharing in the core network between a potentially large number of users. Dynamic provi-

IEEE Communications Magazine • February 2008

sioning is performed through distributed controlplane solutions, such as generalized multiprotocol label switching (GMPLS) [2, 6] or through centralized Web service platforms [1, 4]. Through standardized user network interface (UNI) or Web service invocation interface, it is not necessary for end hosts to implement complex intelligence for optical networks, thus simplifying deployment. In general, the tighter a provisioning model is coupled with specific applications, the better performance it is likely to achieve [6]. However, a loosely coupled provisioning model may provide more flexibility in deployment [1]. Hardware accelerated provisioning is reported to achieve better performance in [8]. We summarize these testbeds (projects) in Table 1. Among these efforts, a widely and implicitly adopted assumption is that communications in escience applications are generally single task-based. The interface connecting each end host to optical networks has dedicated connectivity and is exclusively used by a single data transfer request at any time. We refer to interfaces with exclusive usage as dedicated connectivity interfaces (DCIs) and the corresponding service model as a DCI model. A DCI enables end hosts to take advantage of the huge bandwidth that circuits provide. However, it also inhibits them from communicating with different remote systems simultaneously. At the same time, the DCI model exhibits little flexibility in provisioning circuits with finer granularity. The limited connectivity problem introduced by a DCI can be partially resolved by a dual-homing strategy [4, 6]. In addition to a link that connects each end host to a circuit-switched network, another link connects it to a packet-switched network. Different routing policies can be applied to determine which link to use when communication requests are generated. This strategy has two

S31

Page 58

Network element

IF#1: VLAN1 IF#2: VLAN2

Ethernet line card Network element

VCG VCG

VCG VC XC

VC XC

VCG

ETH MAC PHY

VCAT

GFP framing

MAC VLAN switching

Client 1/(IF#1) Tagged frames

ETH MAC PHY

Ethernet line card

Client 2/(IF#2)

802.1Q compliant switch

Network element

MAC VLAN switching

1:55 PM

GFP framing

1/22/08

VCAT

SUN2 LAYOUT

Tagged frames

Network element Other line cards (Ethernet, SDH)...

Other line cards (Ethernet, SDH)...

■ Figure 1. VLAN-based switching in transport network elements.

interesting features. First, as packet-switched networks provide “always on” connectivity, the limited connectivity of circuit-switched networks can be remedied. Second, as the bandwidth of provisioned circuits is equal to interface capacity and can be well beyond the requirement of a single communication task, falling back to the packetswitched network can improve bandwidth efficiency. However, due to the fact that packet-switched networks have neither bandwidth nor QoS guarantees, this strategy is actually a trade-off between connectivity and the provisioned QoS. One other way to circumvent this problem is scheduling multiple communication requests onto the dedicated interface. The basic idea of scheduling is to arrange local data transfer requests in a time domain; and at any specific time, only one of them is served. The objective is to achieve the least amount of resource consumption, a minimized overall finishing time, and maximized throughput or other similar performance metrics [4, 9]. As provisioning latency may incur significant overhead in the performance, it also is desirable to reduce provisioning latency [8]. Although the scheduled model is suitable for applications in which data transfer requests can be rearranged so that they have no overlap in the time domain, it is not adequate for applications that have concurrent communication requests.

BACKGROUND FOR CIRCUIT PROVISIONING TECHNOLOGIES In this section, we briefly discuss the available technologies that provide end-to-end circuits. In particular, we are interested in solutions for endto-end Ethernet connections, as Ethernet is now the predominant technology in connecting various types of end systems. We assign them to two categories: circuit-switched network and packetswitched network. Circuit-switched networks that can provide end-to-end connections include: • Wavelength routed all optical networks • Time division multiplexed (TDM) synchronous optical network/synchronous digital hierarchy (SONET/SDH) networks With the first, wavelengths are used as transparent links between each pair of Ethernet network interface cards. With the second, an Ethernet connection is native after adopting multi-service provisioning platform (MSPP) at the network edge [5, 6]. In [8], the authors argue that the use of elec-

S32

tronic switches is preferable to using all-optical switches in the near term because such devices offer lower switching time, while at the same time they can handle end-to-end circuits at 1 Gb/s and 10 Gb/s. As more SONET/SDH-based Ethernet transport networks are being deployed and related technologies are still evolving, we also believe that for e-science applications, electrical SONET/SDHbased networks will continue to be one of the mainstream solutions for the time being. The ever-growing data communication and multi-service convergence demands have motivated the development of packet-based transport technologies. Technologies that can provide endto-end connections on a packet-based network include: IP/MPLS, Transport MPLS (T-MPLS), and provider backbone transport (PBT). With pseudo wire emulation (PWE3), Ethernet connectivity can be established over existing IP/MPLS packet switched networks. Transport MPLS, being defined by the International Telecommunication Union-Telecommunication (ITU-T), goes one step further by reformulating MPLS. Client signals including Ethernet can be carried by a T-MPLS sever layer in a connectionoriented fashion. PBT is yet another concept to provide a carrier Ethernet transport solution. With PBT, it is possible to provide carrier grade transport entirely on Ethernet without the need to support protocols such as MPLS. This will simplify network operation greatly and thus reduce operational expenditures (OPEX). In general, packet-based transport networks will provide more efficient and flexible service offerings. Nevertheless, massive deployment of TMPLS and PBT devices is not envisioned in the near future, as much of the standardization of TMPLS and PBT is still ongoing, whereas the enhanced features added to existing SONET/SDH and IP/MPLS networks provide fair solutions to carrier-grade Ethernet transport. Based on 3TNet [10], a regional testbed deployed in Eastern China, we propose VSTONES — a data flow-based VLAN tagging and switching technique to provision end-to-end Ethernet connections to end hosts. The basic idea of V-STONES is that different data flows from end hosts are tagged and then mapped to different circuits in optical networks, through which they are transported to the desired destinations. In the following sections, we address the dedicated connectivity problem in the context of Ethernet over SDH networks. However, as

IEEE Communications Magazine • February 2008

SUN2 LAYOUT

1/22/08

1:55 PM

Page 59

Massive deployment

Control connection

of T-MPLS and PBT Visualization

Control plane

devices is not envisioned in the near future, as much standardization of T-MPLS and PBT is still ongoing,

Storage

whereas the Optical fibers

Data plane

enhanced features Computing cluster

Auto-provisioned end to end circuit

Tagged ethernet frame

SONET/SDH and IP/MPLS networks

User console 802.1Q comliant OXC

added to existing

provide fair solutions to carrier-grade

Control plane entity Circuit provisioning entity

Ethernet transport.

■ Figure 2. Network model. As an illustration of a multiple connectivity interface (MCI), the network interface on the computing cluster is used to establish two circuits for storage and visualization nodes, respectively. Thus, the computing cluster is able to store the computation result and at the same time display it.

VLAN has become one important feature of typical Ethernet services, such as Ethernet private line (EPL) and Ethernet virtual private LAN (EVPLAN) [11, 12], VLAN features support in transport technologies other than SDH is not doubted. For this reason, after specific technological changes have been made, the proposed V-STONES framework can be readily adapted to Ethernet transport solutions, such as IP/MPLS, T-MPLS, and PBT.

MULTIPLE CONNECTIVITY INTERFACE SOLUTION IN ETHERNET OVER SONET/SDH NETWORKS VLAN SWITCHING IN STATE-OF-THE-ART ETHERNET OVER SONET/SDH NETWORKS Figure 1 shows a typical Ethernet over SONET/SDH (EoS) network. Client traffic is aggregated and tagged in an 802.1Q compliant Ethernet switch. VLAN tagging usually is based on the incoming interface. Because each client connects to the switch through a dedicated interface, VLAN tags are client-specific. Tagged Ethernet frames are sent to an EoS network element, where they are switched and mapped to different virtual concatenation groups (VCGs) according to their VLAN memberships. On the other side of the network, Ethernet frames are de-encapsulated and sent to the destination LAN through standard Ethernet procedures. A circuit in such networks is the concatenation of a routed VCG and an Ethernet link on both sides of network. Bandwidth guarantee is native in the routed VCG, whereas it is realized through Ethernet flow control in the Ethernet links. As the communication demands of enterprise costumers

IEEE Communications Magazine • February 2008

are usually long lasting, circuit provisioning is usually manual and is thus performed through management systems. VLAN-to-VCG mapping in this scenario is generally static.

THE MULTIPLE CONNECTIVITY INTERFACE APPROACH FOR END HOSTS We propose a data flow-based VLAN tagging and switching technique to provide interfaces with multiple connectivity in EoS based networks. In the network model illustrated by Fig. 2, end hosts connect to EoS networks through dedicated Ethernet interfaces. The basic idea of tagging in our approach is that on each end host, Ethernet frames are tagged according to their network or transport layer addresses before they are actually sent to the link. As an example, all the frames generated by the computing cluster and destined for a visualization station are tagged with VLAN ID 1, whereas those for a storage system are tagged with VLAN ID 2. This differs in two aspects from the tagging strategy mentioned previously. First, VLAN IDs can be used to distinguish multiple network or transport layer sessions, even though they are generated by the same end host. Second, because tagging is performed locally on end hosts, it is more versatile for reconfiguration. End hosts can relinquish VLAN IDs from expired sessions and allocate them to newly generated ones. As should become clear in the following sections of this article, this is important for dynamic end-to-end circuit provisioning. In addition to VLAN tagging on end hosts, we introduces dynamic VLAN-to-VCG mapping into EoS devices. Whenever a communication task is scheduled on end hosts, a free VLAN ID will be allocated, and a circuit is established in the form of a routed VCG. At the same time,

S33

SUN2 LAYOUT

1/22/08

1:55 PM

Communication

Page 60

Data plane

tasks can be served with different

Application

bandwidth because an arbitrary number of containers can be grouped together to

Control plane

Provisioning agent Service access point

Transport

TNA/ MAC address

form a VCG. In short, an MCIenabled end host

Network

LARP Address resolution module

OC provisioning table

can establish connectivity with

Extended UNI VLAN_ID/ TNA

different remote

Extended UNI-C

interfaces through circuits of varied

Tagging module

granularity.

VLAN 1010

MAC address VLAN_ID

VLAN 1003 VLAN 1001

Tagged Ethernet frames

■ Figure 3. Cross-layer optical circuit provisioning.

the ingress network element is informed of VLAN and VCG correspondence. After VLANto-VCG mapping is established on the ingress network element, tagged Ethernet frames are processed in much the same way as was described in the previous section, and therefore bandwidth guarantee is realized for each VLAN. As shown in Fig. 2, by coordinating VLAN tagging on end host and dynamic VLAN-to-VCG mapping on network elements, our approach enables an Ethernet interface to establish more than one connection with different remote interfaces and thus, realizes the multiple connectivity interface (MCI). For EoS networks where the minimum switching granularity is VC-4 (155Mb/s), a gigabit Ethernet interface can establish connectivity with up to eight different remote interfaces. Each of these connections has dedicated bandwidth of a single VC-4. Substantial connectivity increase is anticipated as switching granularity finer than VC-4 is also becoming mature and commercialized. In addition, communication tasks can be served with different bandwidth because an arbitrary number of containers can be grouped together to form a VCG. In short, an MCI-enabled end host can establish connectivity with different remote interfaces through circuits of varied granularity.

V-STONES: A CROSS-LAYER CIRCUIT PROVISIONING FRAMEWORK Now consider the situation in which the computing cluster in Fig. 2 wants a dedicated channel to storage nodes for all traffic between them and at the same time, two other channels for each of its

S34

TCP connections, both to the visualization node. Tagging data packets according to their network layer addresses or transport layer addresses alone may not solve the problem. In this section, we incorporate the MCI approach into our crosslayer circuit provisioning framework called VSTONES. With V-STONES, end hosts can allocate circuits for data flows generated at different protocol layers. An end host that is capable of cross-layer circuit provisioning is shown in Fig. 3. It has a clear separation of the control plane and the data plane.

V-STONES ARCHITECTURE The control plane in Fig. 3 consists of a provisioning agent, a user network interface client device (UNI-C), and an address resolution module. A data transfer request generated at any of three protocol layers will be propagated to the provisioning agent through layer-specific service access points (SARs). Upon receiving the request, the provisioning agent assigns a free VLAN ID for this data flow and consults the address resolution module for a corresponding destination transport network address (TNA). The address resolution module obtains the destination TNA from the address resolution server. After it receives the TNA, the provisioning module informs the UNI-C to initiate a circuit set-up process. As a VLAN ID is required to establish the VLAN-VCG correspondence on the connected optical network element, it can be propagated through a private interface or signaled through UNI. After the connection is set up and the VLAN-VCG correspondence is established, transporting Ethernet frames from source to destination is ready.

IEEE Communications Magazine • February 2008

SUN2 LAYOUT

1/22/08

1:55 PM

VLAN 1002

Page 61

Static VLAN mapping 1001

VCG provisioned through control plane

LC-ETH

Computing

VLAN switching

VC XC VCG (1*VC-4) VCG (2*VC-4)

LC64

LC64

VC XC

Static mapping

LC-ETH

VLAN switching LC64

Building E5, Minhang campus

Visualization

LC64

Network information center, Minhang campus LC64

VC XC

LC-ETH

VLAN switching LC-ETH: Ethernet line card

LC64: STM-64 line card

LAN interface

LC64

WAN interface

Network information center, Xuhui campus

Storage

■ Figure 4. Data plane implementation and experimental setup. The data plane in Fig. 3 illustrates the actual data flow in end hosts. In addition to layered processing as in traditional IP end hosts, all frames are tagged before they are transmitted to the interface. To achieve this, a tagging module is implemented in the network interface card driver. When receiving a data packet from the IP layer, the tagging module consults the provisioning agent for data flow and VLAN membership mapping. A locally managed mapping cache is desirable to improve lookup performance. Under circumstances where Address Resolution Protocol (ARP) starts before a circuit is provisioned, the MAC address resolution will fail. To handle this problem, MAC address resolution is combined with TNA resolution to form Lightweight Address Resolution Protocol (LARP), as shown in Fig. 3. LARP is designed as C/S architecture in which a centralized lightweight address resolution (LAR) server connects to the control plane network. While MAC addresses can be resolved on the fly, TNA addresses can be reported to the LAR server in an online manner, or simply by offline configuration, because TNAs are usually less prone to change than MAC addresses.

DISCUSSION As can be learned from the previous discussion, V-STONES can provide diverse, fine grained, and bandwidth-assured connectivity to application layer, transport layer, and IP layer sessions alike. Moreover, it exhibits native compatibility with dual-homing. It also opens up new research issues for cross-layer design in optical circuit switched networks. Compatibility with Dual-Homing Strategy: Under the V-STONES model, dual-homing can

IEEE Communications Magazine • February 2008

be achieved through a single network interface. If an application “prefers” a packet-switched network, default VLAN ID is tagged to the corresponding data flow. At the border of the optical network, the network element maps a default VLAN to a static VCG, routed to a packetswitched network. Instead of enforcing complex routing policies, an end host need only configure its local mapping table. Through VLAN switching, the static VCG can be shared among all end hosts connected to the same Ethernet line card, as long as the end hosts use the same VLAN tag for traffic desired for the packet-switched network. Cross-Layer Issues in V- STONES: Under the DCI model, a natural and widely adopted assumption is that a single transport layer session will consume the overall interface bandwidth, which is constant with the time being. Thus, the main concern of transport layer protocol adaptation over DCI is to maintain a constant transmission rate [13, 14]. However, under the V-STONES model, because portions of interface bandwidth can be provisioned to TCP sessions, the available bandwidth is not constant. To maximize link utilization, it would be interesting to investigate how active transport sessions respond to bandwidth change when the number of active sessions changes. On the other hand, mechanisms can be devised so that once an upper layer notices a decrease in its bandwidth consumption, it can notify the underlying provisioning entity to adjust the provisioned bandwidth such that bandwidth utilization efficiency can be achieved. In addition, bandwidth contention on an end host is not addressed by V-STONES. Although the tranport protocol on end hosts will adapt to

S35

SUN2 LAYOUT

1/22/08

1:55 PM

Page 62

2e+007

1e+007

2e+007

Mean throughput = 141.7 Mb/s

1e+007

0

0 (a) 4e+007

4e+007

3e+007

3e+007

2e+007

Mean throughput = 282.4 Mb/s

2e+007 1e+007

1e+007 0

0 (b)

■ Figure 5. Measured traffic (in bits per second) on data interfaces on a) visualization node; b) storage node against time. The on-off curve illustrates a set of randomly generated communication requests scheduled on network interfaces: on represents an active data transfer; off represents no data transfer. The circuit provisioned for the visualization node is a single VC-4 at 155 Mb/s and that for the storage node is two VC-4s at 310 Mb/s. Thus, the achieved signal mapping efficiency is about 91 percent. the allocated bandwidth in circuit-switched networks, novel cross-layer mechanisms can still be investigated to further increase end-to-end performance.

PROTOTYPE IMPLEMENTATION AND EXPERIMENT A prototype based on the proposed V-STONES model is implemented in a deployed SDH network testbed with a GMPLS control plane. As illustrated in Fig. 4, the prototype consists of three FonsWeaver SDH nodes. Three Dell PowerEdge servers are configured as computing, visualization, and storage nodes; each equipped with dual gigabit Ethernet (GE) interfaces. One GE interface connects the servers to one GE LAN port on an SDH node. Another connects the servers to the control plane network. GNU/Debian Linux (Sarge), together with iptables and development files for the packet queuing library (libipq), are installed in the servers. The UNI-C implementation in the optical network is based on OIF UNI 1.0 release 2. The signaling protocol used is Resource Reservation Protocol with Traffic Engineering Extensions (RSVP-TE ) with Ethernet extension. To the provisioning circuits for TCP sessions, IP_Queue deployed on a computing node intercepts each outbound TCP packet. If a TCP SYN flag is detected, the built-in provisioning module informs the UNI-C process to initiate a circuit set-up process. IP_Queue holds the TCP packet with SYN flag in memory until a positive response is received from UNI-C. IP_Queue bypasses all subsequent TCP data packets after the circuit is established. IP_Queue also looks for TCP packets with reset (RST) or FIN flags. After TCP packets with RST or FIN flags are

S36

received from both inbound and outbound TCP packets, IP_Queue informs the UNI-C process to release the circuit. In the current version, the LARP component has not been implemented. Instead of requesting the LAR server to resolve an IP address to corresponding TNA and MAC addresses, the LAR client reads a local configuration file containing IP-TNA-MAC mapping entries. VLAN tagging on end hosts is implemented in a network interface card driver. Upon receiving a data frame, the tagging module looks up a static mapping table according to its transport layer address. If an entry is found for the particular data flow, the data frame is tagged with the VLAN ID as indicated in the table. Otherwise, a default VLAN ID is tagged instead. Uni-directional data processing in the prototype implementation is shown in Fig. 4. On the ingress network element, VLANs are statically mapped to VCGs. VCGs are signaled in both directions through the control plane. On the egress network elements, the processing of Ethernet frames is unchanged. No tagging is required in the reverse direction because WAN interfaces on network elements have a built-in MAC address learning function. In our experiment, two VLANs are configured on the computing node, one for the visualization node and another for the storage node, respectively. We schedule a set of randomly generated data transfer requests on this testbed. For each transfer request, a circuit is provisioned prior to data transfer and released when data transfer completes. The measured provisioning latency is less than 70 ms for set up and 30 ms for release. The latency in between the provisioning process and the data transfer process is about 20 ms. A minimum of 500 ms set-up interval and 100 ms hold time are achieved in the control plane, without any connection set up/release failure after hours of continuous testing. The traffic volume on the visualization and storage node is illustrated in Fig. 5. Throughputs of 141.7 Mb/s and 282.4 Mb/s are measured for the TCP connections served by one and two VC4s, achieving a signal mapping efficiency of approximately 91 percent. During active data transfer, instabilities are observed in both cases. This is because no transport layer adaptation is employed in our implementation. By incorporating a well-designed transport layer protocol, more stable performance can be achieved. For more information on optimizing transport layer performance on static circuits, see [13, 14]. It also is worth noting that optimizing transport layer performance under V-STONES is more challenging because circuits provisioned in VSTONES are more dynamic.

CONCLUSION Given the observation that interfaces with dedicated connectivity have inherent limitations in serving end hosts with arbitrary communication requirements, we propose in this article a VLAN-based approach with enhanced connectivity and granularity flexibility. The proposed approach enables end hosts to communicate with multiple remote counterparts concurrently through bandwidth guaran-

IEEE Communications Magazine • February 2008

SUN2 LAYOUT

1/22/08

1:55 PM

Page 63

teed circuits. It also allows bandwidth to be allocated on a per-task basis, instead of a system-wide basis, which has been investigated thoroughly. As Ethernet traffic proliferates in the operating networks, VLAN support in state-of-the-art optical networks is becoming mature and prevalent. This makes the proposed V-STONES approach economical and practical. The VSTONES approach also can be readily adapted to future carrier-Ethernet transport solutions such as PBT and T-MPLS. Although originally V-STONES was proposed for e-science applications, we believe that as data plane and control plane support for finer granularity circuits continues to evolve, the V-STONES concept and VSTONES-enabled end hosts are likely to emerge even in low-end networking environments. Nevertheless, similar to other approaches, VSTONES raises unresolved issues. To achieve maximum throughput, the cross-layer issues under the V-STONES model are imperative. Novel scheduling algorithms are to be explored. With multiple circuits provisioned concurrently for different destinations, V-STONES alleviates the need for frequent circuit provisioning and thus improves the network scalability. However, in a network with a considerable number of network elements and end hosts, the overhead of provisioning end-to-end circuits can still be significant. Large-scale deployment of V-STONES architecture will benefit from future research on the scalability and resilience of control plane technology.

ACKNOWLEDGMENT The authors would like to thank Professor Wende Zhong (NTU, Singapore), Dr. Tong Ye (SJTU), and Ms. Kai Jiang (Shanghai Super Computer Center) for their discussions. The authors also would like to thank the anonymous reviewers for their insightful comments. This work was sponsored by the National Natural Science Foundation of China (NSFC) under grant 60602010 and 60502004 and the 863 program.

REFERENCES [1] B. St Arnaud et al., “Web Services Architecture for User Control and Management of Optical Internet Networks,” Proc. IEEE, vol. 92, no.9, 2004, pp.1490–1500. [2] T. Lenhman, Jerry Sobieski, and Bijan Jabbari, “DRAGON: A Framework for Service Provisioning in Heterogeneous Grid Networks,” IEEE Commun. Mag., vol. 44, no. 3, 2006, pp.84–90. [3] http://www.surfnet.nl/ [4] N. Taesombot et al., “The OptIPuter: High-Performance, QoS-Guaranteed Network Service For Emerging E-Science Applications,” IEEE Commun. Mag., vol.44, no.5, May 2006, pp. 38–45. [5] N. S. V. Rao et al., “Ultrascience Net: Network Testbed for Large-Scale Science Applications,” IEEE Commun. Mag., vol. 43, no. 11, 2005, pp. s12–s17. [6] X. Zheng et al., “CHEETAH: Circuit-Switched High-Speed End-to-End Transport Architecture Testbed,” IEEE Commun. Mag., vol. 43, no. 8, 2005, pp. s11–s17. [7] J. Mambretti et al., “Optical Dynamic Intelligent Network Services (ODIN): An Experimental Control-Plane Architecture for High-Performance Distributed Environments Based on Dynamic Lightpath Provisioning,” IEEE Commun. Mag., vol. 44, no. 3, 2006, pp.92–99. [8] M. Veeraraghavan, X. Zheng, and Z. Huang, “On the Use of Connection-Oriented Networks to Support Grid Computing,” IEEE Commun. Mag., vol. 44, no. 3, 2006, pp.118–23. [9] J. Kuri et al., “Routing and Wavelength Assignment of Scheduled Lightpath Demands,” IEEE JSAC, vol. 21, no. 8, 2003, pp. 1231–40.

IEEE Communications Magazine • February 2008

[10] W. Hu. et al., “Advanced ASON Prototyping Research Activities in China,” Proc. SPIE 5626, 2005, pp.675–82. [11] Metro Ethernet Forum, “Ethernet Services Definitions — Phase I,” MEF 6, June 2004. [12] Metro Ethernet Forum, “User Network Interface (UNI) Type 1 Implementation Agreement,” MEF 13, Nov. 2005. [13] A. P. Mudambi et al., “A Transport Protocol for Dedicated End-to-End Circuits,” IEEE ICC ’06, vol. 1, 2006, pp. 18–23. [14] X. Zheng et al., “FRTP: Fixed Rate Transport Protocol – A Modified Version of SABUL for End-to-End Circuits,” Proc. Pathnets 2004 on Broadnet 2004, Sept. 2004.

BIOGRAPHIES WEIQIANG SUN [M] ([email protected]) received his B.Tech. from the Special Class for the Gifted Young (SCGY) at the University of Science and Technology of China (USTC) in 1999 and his Ph.D. from the same university in 2004. He is a lecturer in the Department of Electronic Engineering at Shanghai Jiao Tong University (SJTU), China. His research interests include dynamically configured optical networks, QoS in packet-switched networks, and IPTV.

In a network with a considerable number of network elements and end hosts, the overhead of provisioning end-toend circuits can still be significant. Largescale deployment of V-STONES architecture will benefit from future research on the scalability and resilience of control

GUOWU XIE [S] ([email protected]) is a Master’s candidate in the Department of Electronic Engineering at SJTU. He received his B.Tech. from Tong Ji University in 2005.

plane technology.

Y AOHUI J IN [M] ([email protected]) is a professor at the State Key Laboratory of Advanced Optical Communication Systems and Networks, SJTU. His research interests include optical networking, optical grid, and switch scheduling. Prior to joining SJTU, he was a member of the technical staff at Bell Labs Research China from 2000 to 2002. He has served as the Transaction Processing Performance Council (TPC) member in many international conferences. He has published more than 50 papers in technical journals and conferences. WEI GUO ([email protected]) has been an associate professor at the State Key Laboratory of Advanced Optical Communication Systems and Networks of SJTU since 2003. Her research interests include optical grids, network planning, and optimization algorithms. Prior to joining SJTU, she was a senior engineer and project manager at Fiberhome Telecommunication Technologies Co., Ltd. from 2001 through 2003. She has published over 50 publications in technical journals and conferences. W EISHENG H U [M] ([email protected]) is a professor and director of the State Key Laboratory on Fiber-Optic Local Area Networks and Advanced Optical Communication Systems, SJTU. His research interests are in generalized automaticswitched optical networks and optical packet switching. He has published over 100 journal and conference papers. MIN-YOU WU [SM] ([email protected]) is an IBM Chair Professor in the Department of Computer Science and Engineering at SJTU. He serves as chief scientist at the Grid Center of SJTU. He also is a research professor at the University of New Mexico. His research interests include grid computing, wireless networks, sensor networks, overlay networks, multimedia networking, parallel and distributed systems, and compilers for parallel computers. He has published over 140 journal and conference papers in the previously mentioned areas. X INHUA L IN ([email protected]) received his Master’s degree in computer science from SJTU in 2005. He is a research assistant in the Department of Computer Science and Engineering at SJTU. His current research focuses on grid computing and high performance computing. WENTAO LI ([email protected]) received his B. Tech. in electrical and information science from Tianjin University before he joined Fiberhome in 2002. He is now a senior engineer in the Department of Technical Support at Fiberhome. RONG JIANG ([email protected]) joined Fiberhome in 1998 and is now the manager of Ethernet service in the Department of Optical Networks at Fiberhome. X UEQIN W EI ([email protected]) is in charge of strategic research on optical networking at Fiberhome. He was the manager of the ASON product line from 2004 until 2006 at Fiberhome.

S37