The Network Flow Language: A Mark-Based Approach ... - Columbia CS

The Network Flow Language: A Mark-Based Approach to Active Networks Yechiam Yemini, Sushil da Silva, Danilo Florissi, Hao Huang {yemini, dasilva, df, hhuang}@cs.columbia.edu http://www.cs.columbia.edu/dcc

Technical Report Department of Computer Science Columbia University New York, NY 10027 July 1, 1999 Abstract This paper describes a novel approach to active networks based on an extensible marked language, similar to XML. The Network Flow Language (NFL) uses marked packets embedded in network flows to perform flowprocessing functions traditionally handled by protocol headers. Marked packet flows invoke execution of active elements that can be dynamically loaded into mark-processors at network nodes. These mark processors interpret marked elements embedded within a flow to perform routing, error/flow/admission-control, bandwidth management, caching and other network functions. The paper introduces NFL via a detailed example: an active, extensible application layer transport protocol designed to replace TCP/HTTP. This protocol demonstrates the effectiveness of NFL in enabling introduction of significant, dynamically programmable network functionality. Index terms—Active networks, extensible transport protocols.

1 Introduction HTML[1] has demonstrated the value of marked languages in creating a simple, universal paradigm for programming distributed information access, display, and distributed applications. Its recent generalization into XML[2] is already driving the use of marked languages as a universal paradigm for programming distributed systems. The underlying idea of XML is simple. A marked element (or tag) is viewed as a general operator on the text in its scope. When XML is used as a document format, the syntax of a mark is defined in a Document Type Definition (DTD) file [2], while its semantics is defined in an eXtensible Stylesheet Language (XSL) file [3]. A document processor uses these DTD and XSL files to parse, interpret, and operate on marks embedded in a text string. One programs an XML application by defining libraries of marked syntactic elements and binding them to programs that parse and interpret these marks. This provides a simple and universal paradigm for programming distributed applications. The Network Flow Language (NFL) is a marked la nguage to program processing of network flows. NFL marks are embedded in network packet flows to perform the functions traditionally handled by protocol headers. NFL views a network as a pipeline of programmable mark processors, constituting an NFL channel. The marks guide the routing and processing of flow through a channel, much like protocol headers guide packet processing through element hardware and software. Marks can flexibly program routing, flow, and admission control, resource reservation, or content filtering and caching functions. Marks replace the traditional rigid allocation of functions among layers with a

flexible nesting of stacked operations on a flow. This enables greater efficiency and flexibility in handling network functions such as error or flow control. For example, error control may be handled by respective marks on a content-specific basis; loss of continuous media data (e.g., an image) is handled differently than loss of more sensitive data (e.g., text, or database) within the same flow. Similarly, marks may be embedded in a flow by congested routers to control the windows of senders, thus avoiding the need for adaptive guesswork of network congestion as in TCP slow-start mechanisms. NFL may be viewed as a small set of extensions to XML to support extensible programming of protocols. A protocol is defined by a library of marks that may be dynamically loaded into network nodes, thus programming new capabilities into the network. In contrast with XML, which is concerned with display and computations of information files, NFL is concerned with moving and processing network flows. A mark processor is programmed by respective definition files to process NFL marks and act accordingly on flows of data moving through a network. This provides a data-flow processing model where the computations are defined and scheduled by embedded marks. The meaning of these marks can be programmed, much like XML, through appropriate configuration files. The syntax of marked NFL packets follows that of XML and utilizes a Flow Tag Definition (FTD) analogous to DTD files for definition of NFL packet syntax. The semantics of NFL tags is defined by the eXtensible Flow Language (XFL), analogous to XSL. XFL associates a network processing semantics with an NFL mark. It is possible to view NFL as a marked language and execution environment (EE) (e.g., [4-7]) for programming active networks[8-10]. The FTD and XFL scripts define the syntax and meaning of the marks. These files and associated active code are dynamically downloaded into nodes when NFL encounters a hitherto unknown mark. The mark-processor parses a flow based on the syntax of marks defined by FTD files. It then invokes routines associated with respective tags and applies these routines to respective flow segments. In contrast with alternative approaches to active networks where packets contain active code, NFL packets are passive whereas the mark processors are active. These mark processors deployed in nodes process and interpret passive data streams as in conventional networks of today. One benefit of this approach is that security checks and optimizations are performed on a per-protocol basis when a markprocessor is installed, rather than per-packet. In what follows, the paper describes NFL through an extended example: the Universal eXtensible Transport Protocol (UXTP). UXTP is an NFL-based application layer transport protocol intended to replace TCP and HTTP. Section 3 then proceeds to provide a brief architectural overview of NFL. Section 4 compares NFL to related active network projects. The paper ends with preliminary conclusions.

2 NFL By Example: UXTP The Universal eXtensible Transport Protocol (UXTP) is an NFL-based protocol that provides a unified framework for programming application-oriented transport functions 1. UXTP can be viewed as an extensible replacement of HTTP[11, 12] and TCP. The section describes the architecture, structure, and operations of the core UXTP protocol. Later, Section 3 shows how the base UXTP protocol can be dynamically extended with new functions to monitor QoS.

1 This paper focuses on transport and application layer applications of NFL. However, NFL can also be used to implement functions of lower layers, assuming NFL-enabled routers and switches.

2

2.1 UXTP Architecture The overall architecture of UXTP is depicted in Figure 1. WWW Srvr

UXTP Proxy

UXTP Proxy

NFL

NFL

UDP

UDP

Browser

Figure 1: UXTP Stack

The NFL mark processors are considered, for the purpose of this example, as a protocol layer above the UDP transport. Various application protocols, coded in NFL, can be implemented above NFL. In this example, UXTP is an NFL-based transport protocol that replaces HTTP and TCP. UXTP utilizes a proxy server and clients to bridge the HTTP server and clients and provide transparent interoperability with existing browsers. Consider a typical Browser-server interaction. The browser issues an HTTP GET request to a specific URL. Ordinarily, HTTP will open a TCP connection to the respective server and retrieve the respective pages. With UXTP, these interactions are pursued as follows. The client browser is configured to forward all HTTP requests to the local UXTP client proxy. An HTTP GET request is transformed by the UXTP proxy client into respective UXTP marked packets interpreted by NFL. These commands initiate a duplex flow between the proxy client and server. The GET request is encoded as a sequence marked packets inserted in the client-to-server flow. These marks define, among other, packet boundaries used by NFL to form the respective UDP packets.

2.2 UXTP Operations UXTP is a simple request-response protocol that provides three primary operations: , , and . An packet is sent by a proxy client to invoke an HTTP method (e.g., GET, POST) on an UXTP server. The server responds with an ordered sequence of packets, each encapsulated within a UDP datagram. A response can contain an optional . Each such segment represents a fragment of the requested object. The client acknowledges segments with packets. These acknowledgments pace the server by setting window size and posting selectiverepeat requests. (See the Appendix for a syntax definition of UXTP packets.) Consider the exchange shown in Figure 2. Here the UXTP client sends an HTTP GET request to a server; the server responds with the first segment of the requested object; the client acknowledges the receipt of the first two segments and notifies the server of a new window size. Subsequent responses from the server (not shown) return the remaining segments of the requested object.

3

Proxy Client: < request-line method=”GET” URI=”http://www.cs.columbia.edu” version=”HTTP/1.0/>

Proxy Server:

Section 3.2 below describes how the base NFL protocol can be extended with new tag packet formats, including those to monitor QoS on UXTP flows. The Appendix gives the complete syntax definition for above tags.

3 NFL Architecture This section shows how the NFL protocols are programmed and deployed in networks. In particular, it describes the Flow Tag Definition (FTD) language and Extensible Flow Language (XFL), the NFL execution architecture, and shows how NFL protocols are dynamically deployed to and executed at network nodes. FTD is a language to declare the syntax of NFL packets; XFL is a simple configuration language that binds NFL tags declared in an associated FTD file to flow processors. A flow processor is a program (software or hardware) that processes a stream of tag packets. Flow processor can be written in any high-level language that adheres to the NFL APIs. On arrival at a node, NFL packets are parsed according to their syntactic definition in the FTD file and associated flow processors are invoked to handle each tagged element. If a hitherto unknown tag is encountered by the parser, NFL retrieves FTD and XFL files from a designated server, parses the tagged element and uses the associated XFL file to download, install and configure a flow processor to handle the parsed packet. The rest of this section shows how NFL packets are declared, processed and configured at network nodes.

3.1 Flow Tag Definition Language (FTD) FTD is a minimal extension of XML’s DTD. Whereas the XML DTD declares the markup syntax of text documents, FTD focuses on syntax of tagged packets. FTD seeks to address the specific requirements of tag packet processing and, as such, differs from XML DTD in two ways. First all markup elements in XML are parsed as un-typed text. In contrast, FTD provides first-class support for fundamental data types such as integers, floats, characters, and so on. Support for fundamental types ensures that type errors are caught at parse time and that fundamental types can be automatically mapped to respective la nguage-level objects. Furthermore, parse-time knowledge of data types makes it possible to efficiently encode and parse NFL packets to and from a binary representation for transmission on the network. Second, an FTD packet declaration can be dynamically extended with new tags without editing the original FTD file. This form of extension is important because it enables two types of dynamic protocol composition [14]. The first is composition for interoperability, which is typically achieved through encapsulation and layering. For example, users could encapsulate a new protocol inside UXTP by declaring a new NFL packet type in a FTD file, and defining the encapsulation relationship and flow-processor bind6

ings in an associated XFL file. Protocols implemented in NFL use an element tag as a demultiple xing key to identify and process embedded sub-flows. Thus, new protocols can be dynamically deployed by introducing tagged packet formats. NFL also supports composition for extensibility. This form permits extension an existing protocol with new operations. As an example, the HTTP protocol could be extended with new request methods (e.g., POST, PUSH, etc). Consider a simple example. The listing below shows part of the declaration of a UXTP uxtp-request packet. (See Appendix A for a complete FTD specific ation of UXTP.) | general-header | #EXTENSION)* (uxtp-segment)*>

The syntax of an FTD file follows that of XML DTD. An ELEMENT declaration defines the syntax of an NFL tagged element; the ATTLIST declaration specifies a list of attributes for an NFL element. The above declaration says that a uxtp-request packet consists of an optional request-line followed by a sequence of zero or more headers followed by an optional sequence of uxtp-segments. The example illustrates two novel aspects of an NFL packet declaration. First, the request-id, attribute is declared as an integer. This attribute will be mapped to integer fields in the Abstract Syntax Tree created by the NFL packet parser. Second, the #EXTENSION keyword indicates that alternative elements in enclosing selection rule can be added later. Any syntactically valid NFL packet can appear as data within a UXTP frame as long associated FTD and XFL files define how this unknown tag is to be parsed and processed.

3.2 Extensible Flow Language (XFL) XFL is a simple configuration language whose programs bind tag packet declarations in a DTD to associated flow processors, written in a high-level language such as NetScript or Java. An XFL script consists of one or more rules, each of the form head body. The rule head is a slash-delimited string that identifies an NFL tagged element. The rule body as an URL that points to a Java la nguage script to run in order to install and configure a flow processor for the new tag. When an NFL node encounters an unknown tagged packet, it queries the sending node (or other designated server) for respective FTD and XFL files. NFL then searches the XFL file for a rule whose head matches the new tag. If one is found, the script pointed to by the rule body is run at the NFL node. This script is passed a reference to NFL node state (which can include routing tables, MIB variables, etc). The configuration script examines local node state if necessary and installs a flow processor program to handle subsequent packets with the new tag. The following is a rule from an XFL file that extends UXTP with operations to generate reports of QoS statistics on UXTP flows.

7

“nfl/uxtp-request/uxtp-qos-report-request” “ftp://nfl.cs.columbia.edu/uxtp/UXTPQoSReportRequestInstaller.class”

The rule head says that if NFL encounters a packet with a uxtp-qos-report-request tag encapsulated within UXTP and NFL respectively, then NFL is to download and run the designated script in the body of the rule. Notice that the hierarchical (‘/’ delimited) syntax of the rule head constrains the context in which uxtp-qos-report-request headers can appear. Note also that the configuration is run just once when the first such packet arrives.

3.3 NFL Programming Model An NFL Flow Processor is a program that processes a stream of tagged packets. Typically, a flow processor is installed by a configuration script as described above. In principle, a flow processor could be written in any convenient language. However, the current NFL system is designed to work particularly well with NetScript[15, 16], a dataflow extension to Java for programming active network protocols. The fundamental primitive in NetScript is called a box, a dataflow node that operates on streams of packets. A box consists of typed input and output ports, internal boxes and connections between internal boxes. Protocol composition is achieved by connecting ports of boxes together. A NetScript node is a reactive system that consists of interconnected boxes. Packets that arrive at a network node are processed by successive boxes as they flow through a node. Typically, boxes perform packet parsing, header analysis and classification, routing, etc. NetScript is useful for programming NFL protocols in part because its dataflow graph is dynamic; boxes (e.g., NFL flow processors) can be deployed at runtime to remote nodes and composed with other resident boxes to introduce new protocols or extend existing protocols with new operations. An NFL protocol or flow processor is easily implemented as a NetScript box.

NFL uxtp-request GET

UDP

POST uxtp-timestamp

uxtp-ack uxtp-response

Figure 3: NFL/NetScript Architecture The NFL system is implemented in a user-space process as a graph of interconnected NetScript boxes. Figure 3 shows the NFL node architecture. NFL is layered directly above UDP. All incoming NFL traffic is passed to a NFL classifier. The classifier (shown as triangles in the figure) plays a central role in NFL. It parses incoming NFL packets into an AST tree (the current version uses a variant of the W3C 8

DOM/SAX APIs[17] to represent XML parse trees), and outputs each marked sub-flow on a respective outport. To perform its task, the classifier utilizes a rule base that can be configured at runtime. Each outport of an NFL classifier is connected to a box that processes the respective marked flow. In the figure, the uxtpOut outport of the classifier in the NFL box has been connected to uxtpIn inport of the UXTP box. Similarly, the classifier in the UXTP box forwards uxtp-request, uxtpresponse, and uxtp-ack packets to respective boxes. 3.3.1 Programming NFL Protocols This section demonstrates use of a XFL script that configures a NetScript box within UXTP to handle a new type of operation. The example shows in particular how the box that processes uxtp-requests is extended to process a uxtp-qos-report-request header. This install method example script is called from the XFL rule shown in Section 3.2 above. This method takes two arguments. The first is a reference to the box in which the new boxes are to be installed; the second is a reference to the tag that triggered execution of the script. // This example shows how to install a NetScript box within // uxtp to handle qos-report-request class UXTPQoSReportRequestInstaller implements nfl.XFLScript { public void install(Box enclBox, String tag) { UXTPRequestBox uxtpRequestBox = (Box) enclBox; NFLClassiferBox classifierBox; UXTPQoSReportRequestBox qosReportReqBox; // get the classifier box installed within uxtpRequestBox classifierBox = uxtpRequestBox.getBox(“classifierBox”); // create and add an uxtpQ to httpBox qosReportReqBox = httpBox.addBox(“qosReportReqBox”, new UXTPQoSRequestBox()); // send all packets that match specified tag to “qosReportReqOut” outport // The outport “qosReportReqOut” is created if it does not yet exist classifierBox.classify(tag, “qosReportReqOut”); // connect outport “qosReportReqOut” of “classifierBox” to // “in1” inport of just-installed “qosReportReqBox” httpBox.connectOutportToInport(“classifierBox”, “qosReportReqOut”, “qosReportReqBox”, “in1”); qosReportReqBox.start(); // additional connection/configuration commands not shown... } }

Figure 4: XFL Script to Configure a NetScript Box An installation script typically performs three functions: query and/or modify the state of box in which new boxes are to be installed; remove existing boxes or install new boxes; connect new boxes to existing boxes. The program shown in Figure 4 installs a new UXTPQoSReportRequestBox within enclosing UXTPRequestBox. To do this, it configures the classifier box in uxtpRequestBox to forward all packets that match tag “uxtp-qos-report-request” to a qosReportReqOut outport. It then connects this outport to the in1 inport of the uxtpReportReqBox. Henceforth, all requests for QoS reports are proc9

essed by the new box.

4 Related Work Several possible approaches to active networks have been proposed. In one principal approach, described in [9], [18], and [19], each packet is a program-capsule that constitutes an active program. Capsules are interpreted by individual nodes as they traverse the network. The packet is thus overloaded as the unit of protocol deployment, extensibility, and of protocol execution. A complementary approach, pursued by NFL and by the authors’ ongoing work on NetScript, is to separate the deployment of active elements from the processing of packet streams on which they operate. Whereas the packet-capsule approach treats packets and programs interchangeably, NFL treats them as distinct entities. Marked NFL packets are passive, much like those of current networks; the programs that operate on NFL packet-streams are active and dynamically extensible. A central benefit of this approach is that security checks and optimizations are performed once when a new protocol is installed rather than per-packet.2 NFL’s approach to protocol extensibility is also novel. Marks embedded in NFL flows are analogous to headers in conventional protocols. However, the syntax and meaning of headers in conventional protocols is rigidly defined at protocol-design time. This paper shows that new protocols can be deployed with NFL and existing NFL protocols are easily extended with new operations. The approach is simple. Use FTD to declare the syntax of extensible packets formats; use XFL to dynamically bind syntactic packet declarations to active programs that parse and interpret NFL flows. FTD can be viewed as an extensible analog of packet presentation languages such as ASN.1[20], XDR[21]. XFL can be viewed as the semantic backend of an extensible interpreter for NFL flows. NFL thus provides unified framework for designing and implementing extensible, programmable protocols.

5 Conclusion This paper presented an overview of NFL, a mark-based approach to programming active network applications. NFL provides a simple framework for programming extensible network protocols in much the same way that XML enables extensible and programmable document formats. Protocols written with NFL may specify flow behavior for routing, resource allocation, and flow and congestion control. In particular, this paper demonstrates use of NFL to develop a flexible transport protocol (UXTP) that is customized for a specific application (hypertext transfer).

6 Bibliography [1]

T. Berners-Lee and D. Connolly, “Hypertext Markup Language -- 2.0,” MIT/W3C RFC 1866, November 1995.

[2]

T. Bray, J. Paoli, and Sperberg-McQueen, “Extensible Markup Language (XML) 1.0,” W3C REC-xml-19980210, http://www.w3.org/TR/REC-xml, February 1998.

[3]

World Wide Web Consortium, “Extensible Stylesheet Language (XSL),” W3C, W3C Working Draft, http://www.w3.org/TR/WD-xsl, 1999.

2 As a simple example, NFL could be extended to only download and install flow processors from trusted sources that provide appropriate certificates. This kind of security check is only performed once when a new NFL protocol is installed.

10

[4]

C. Tschudin, “Flexible Protocol Stacks,” presented at SIGCOMM '91, Zurich, Switzerland, 1991.

[5]

J. Hartman, L. L. Peterson, A. Bavier, P. Bigot, P. Bridges, B. Montz, R. Piltz, T. Proebsting, and O. Spatscheck, “Joust: A Pla tform for Communications-oriented liquid software,” Department of Computer Science, University of Arizona, Tucson, Arizona, November 1997.

[6]

S. Bhattacharjee, K. Calvert, and E. Zegura, “Reasoning About Active Network Protocols,” presented at IEEE ICNP'98, Austin, TX, 1998.

[7]

D. J. Wetherall, G. John, and D. L. Tennenhouse, “ANTS: A Toolkit for Building and Dynamically Deploying Network Protocols,” presented at IEEE OpenArch, 1998.

[8]

Y. Yemini and S. da Silva, “Towards Programmable Networks,” presented at IFIP/IEEE Workshop on Distributed Systems Operations and Management, 1996.

[9]

D. L. Tennenhouse and D. J. Wetherall, “Towards Active Networks,” MIT Laboratory for Computer Science, 1996.

[10]

J. M. Smith, D. J. Farber, C. A. Gunter, S. M. Nettles, M. E. Segal, W. D. Sincoskie, D. C. Feldmeier, and D. S. Alexander, “SwitchWare: Towards a 21st Century Network Infrastructure,” University of Pennsylvania, 1996.

[11]

T. Berners-Lee, R. Fielding, and H. Frystyk, “Hypertext Transfer Protocol -- HTTP/1.0,” W3C/MIT, UC Irvine RFC 1945, May 1996.

[12]

R. Fielding, J. Gettys, J. C. Mogul, H. Frystyk, and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” W3C/MIT, U.C. Irvine, DEC RFC 2068, January 1997.

[13]

D. Clark and D. Tennenhouse, “Architectural Considerations for a New Generation of Protocols,” presented at SIGCOMM'90, 1990.

[14]

S. da Silva, D. Florissi, and Y. Yemini, “Composing Active Services with NetScript,” presented at DARPA Active Networks Worshop, Tucson, Arizona, 1998.

[15]

S. da Silva, “Programming in the NetScript Toolkit,” Columbia University Computer Science Department, New York, http://www.cs.columbia.edu/dcc/netscript, October 1998.

[16]

S. da Silva, “NetScript: A Language-Based Approach to Active Networks,” , 1998.

[17]

World Wide Web Consortium, “Document Object Model (DOM) Level 1,” W3C, Recommendation, http://www.w3.org/TR/REC-DOM-Level-1, October 1998.

[18]

M. Hicks, P. Kakkar, J. T. Moore, C. A. Gunter, and S. Nettles, “PLAN: A Packet Language for Active Networks,” presented at the Third ACM SIGPLAN International Conference on Functional Programming Languages, 1998.

[19]

B. Schwartz, W. Zhou, A. Jackson, W. Strayer, D. Rockwell, and C. Partridge, “Smart packets for active networks,” BBN, http://www.net-tech.bbn.com/smtpkts/smtpkts.ps.gz, Jan 1998.

[20]

CCIT, “Open Systems Interconnection: Specification of Abstract Syntax Notation One (ASN.1),” CCIT Recommendation X.209, 1992. 11

[21]

R. Srinivasan, “XDR: External Data Representation Standard,” Request for Comments 1832, August, 1995.

7 Appendix: UXTP FTD Specification This appendix is a listing of the FTD specification for UXTP, a marked protocol that provides the combined functions of TCP and HTTP version 1.0. The core UXTP protocol consists of three primary packet formats , , and . The request and response packets can contain embedded marked headers, equivalents of respective headers in the HTTP/1.0 specification. For conciseness, the marked equivalents of HTTP headers are not shown here; instead, comments in the listing point to respective sections in the HTTP specification.

7.1 Core UXTP Packet Formats Figure 5 defines the basic UXTP packet formats.

7.2 Extension UXTP Headers Figure 6 defines UXTP extension headers described in Sections 2.2 and 3.2 to support QoS resource allocation.

12




Section 5.1 of HTTP/1.0 spec --> Section 5.2 of HTTP/1.0 spec --> Section 7.1 of HTTP/1.0 spec --> Section 4.3 of HTTP/1.0 spec --> Extension headers can be added later-->









identifies uxtp-request --> HTTP/1.0 version (Section 6.1) --> HTTP/1.0 Status code (Section 6.1.1)--> total number of segments in response-->











Figure 5: Core UXTP FTD Specification

13

->



Figure 6: UXTP Extension Headers to Support QoS

14