the network to freely associate services and resources such as computing, .... While our research and development is in progress, in Section 5 we present some ...
OverMesh: Network Centric Computing John Vicente* †
Sanjay Rungta* *
Gang Ding‡ †
Dilip Krishnaswamy*
Winson Chan*
Kai Miao*
‡
Intel Corporation, Columbia University, Purdue University
Abstract The communications industry significantly lags the personal computing industry in terms of empowering the end-user to innovate; as fundamental architectural shifts have yet to occur to push intelligence to the network edge. While promise of such a potential has been well recognized and promoted by the network research community, its realization in the communications market has fallen well short. The decentralization of the telecommunication system to the edge of today’s hierarchically-formed networks is inevitable with the emergence of key wireless technologies and the proliferation of mobile computing devices and more prevailing usage models. This article envisions a decentralized communication system formed of wireless mesh networks and a highly virtualized, converged computing and communications node-based architecture with emergent management capabilities. We propose to embed the user inside the network to freely associate services and resources such as computing, storage, and bandwidth to rapidly advance network innovation – network centric computing. This article proposes a novel direction and vision for network centric computing, expands on our research and presents the OverMesh platform from which we have begun work to instantiate this vision. Finally, we expect this new envisioned network to co-exist with today’s Internet infrastructure and enable a new generation of applications and usage models. Index words: network centric, wireless mesh, cognitive radios, virtualization, overlays, cross-layer, performability, knowledge plane
1. Introduction The historical significance of decentralization through personal computers demonstrated a remarkable approach to accelerate innovation through broader participation and simultaneously grow the information technology (IT) industry. The future of communications is now predicted to follow the same path, i.e., to evolve from a highly centralized and provider-driven model towards a more decentralized system and to shift intelligence to end-users [1-4]; with significant opportunities for innovation and reduced barriers for market entry, such decentralization is predicted to bring major growth to the communications industry [5-7]. Contrary to these indications, however, networked users today still rely heavily on service provider managed networks and systems for their services. In such a centralized system, users inescapably sit outside the service domain and access resources and services rendered to them by centrally operated service providers as depicted in Figure 1.
Figure 1: Today’s hierarchically formed networks
This, of course, has broad implications to network costs, services and devices which end-users must conform, as well as to service providers themselves who rightfully gain opportunity in such a centralized environment. Furthermore, it is well-known that there is an exacerbation of an end-to-end manageability problem [8-12] for Internet services, when
competing service providers provision and manage autonomously their services to the end-users or access with difficult propositions for multilateral agreements. As shown in Figure 2(b), the current telecommunications hierarchical structure which governs positioning and roles for the entire ecosystem value chain for data and voice communications products and services, conceptually replicates the model of centralized computing, as depicted in Figure 2(a). The capital intensive services in the edge and access networks are independently controlled and managed by different service providers, while the operating model for mobile computing has spawned numerous access networks, disjointed and costly to the user in supporting their various operating locations, media or device types.
Figure 2: (a) Computing and (b) telecommunication structures
Beyond cost, the effects to the end-user or end-user organization include business discontinuity and end-to-end manageability and provisioning complexity. Thus, end-users have little influence or innovation flexibility on the service delivery or resource models in today’s communication systems. The contention in this article is not on the sound principles of network design for which hierarchy and separation are vital to address distributed scale and complexity, but rather on the inability to increase the speed and level of innovation participation in the introduction of new network services beyond the core through the extreme edge1 of the network. Today, users are no longer static in their operating model for voice, data and media communications; this is evident on the consumer as well as the enterprise side of information technology. Low-cost wireless communication systems are introducing an inflection opportunity for rethinking communication systems – the unwiring of users and the free association of communication services on more horizontal and compute-rich devices. We view this change consistent with the ubiquitous computing [1] direction, and the limitations of today’s Internet and telecommunication networks represents a ready opportunity to bridge this transition by shifting our thinking towards alternative connectivity structures and methods for technology and service innovation. As peer-to-peer computing has demonstrated, viral behavior [3] is exhibited when we design systems that embrace a broader social population; open innovation, replication, accelerated adoption, and scale are all evidenced in the market today. There are clear social networking dynamics [13, 14] present in these systems. In [15], Reed refers to Group Forming Networks (GFN) as a capability to increase the value of networks exponentially with the increased number of participants in groups. This insight naturally motivates our research direction – a need to bring the “user inside the network”, as a node in the network and enabling a more peer-to-peer structure to networking systems; thus, creating a connectivity structure that a) matches the network infrastructure with peering, social nature of the users and, b) enables a larger number of physical and virtual group formations. We view these as essential requirements towards accelerating or scaling network value by facilitating wide-ranging use cases that can extend the resilience of decentralized networks and extend the value of networking based on social emergence. This includes logical
1
By edge here, we are referring to the customer edge, which extends beyond the edge and access of Figure 1 into the client or customer presence edge networks. In our proposed framework, the collapse of the hierarchical network formations should also imply convergence to the ultimate edge – client or end-user. 2
connectivity networks, collaborative overlays, network service overlays and application overlays – a broad array of peer-to-peer style group formations that can be created through multiple (or many) virtual groupings. In this paper, we argue for an alternative, parallel network2 where computing and networking conjoin at each leaf node to enable end-users to freely associate services and utilize resources such as computing, storage and bandwidth in a highly decentralized communication system – we refer to this as network centric computing. Starting in Section 2, we present an introduction of our network centric computing vision as a proposed redirection of networking systems and introduce an advanced internetworking system called OverMesh. Section 3 articulates the design and architectural requirements of OverMesh, while we discuss our first-order implementation of the OverMesh system in Section 4. While our research and development is in progress, in Section 5 we present some important research challenges and areas we propose to the research community to help enable network centric computing.
2. Network Centric Computing Decentralization is the first key characteristic in future computing and communication architectures. For communications, we envision a more flat or ‘sensor-like’ networked system with a large number of end nodes and internetworking nodes with similar form and function, but with varying resource and service profiles. These nodes participate not only in network transport (delivery), but also in control (service), and management (operations). In this framework, end-users are associated with these nodes either as a leaf or as an internetworking node. Basically, the network becomes a myriad of personal networks, fully meshed in physical connectivity and service composition – combined peer-to-peer computing and networking. The user is embedded into the network rather than the borders of the network. Communities of virtual networks may form by contemplative design or virally through social cooperation. Physical (wireless) connectivity is instantiated opportunistically without ownership or hardened allocation [16, 17, 50]. Figure 3(a) and 3(b) illustrates this vision with two alternative instantiations of decentralized networks formed through networking and computing convergence – network centric computing.
Figure 3: Decentralized networks a). Sparse networks
b.) Structured networks
Decentralizing or collapsing networks into a flat structure poses complex distributed systems challenges as the complete distribution of resources against distributed demands creates a chaotic situation for achieving stability and optimality. Moreover, it is likely that the network nodes may be highly dynamic, further increasing complexity in managing reliable or deterministic state of the network.
2
The authors are not proposing a displacement of today’s Internet network system, but rather a co-existence of networks which may allow for alternative styles or community grouping of communications without the necessity of centralized infrastructure networks which may not be required. 3
While our proposed system has similarities, it is architecturally distinct to current ad hoc [18] or sensor networking [19] systems as it is with peer-to-peer computing [20]. Unlike sensor or ad hoc networks, we envision richer node constructs and a multi-layer communications and computational service model to scale wireless meshed communications. This includes the use of multiple radio systems and cross-layer methods to facilitate end-end awareness and to manage nodelevel resources. And unlike peer-to-peer computing systems, we are not limiting the peering to computation at the application or application service level. We propose to pursue a more extreme peering model where internetworking services, for robust network control and management, are facilitated by emergent and viral [3] properties by leaf nodes or internetworking nodes without dependency on centralized communication services or operations. Further, borrowing from the PlanetLab [21] research community, we propose distributed virtual machines (DVM) to instantiate networked overlays capable of merging computation and communication requirements using virtualized node constructs to support open service creation and delivery. In summary, we view this system as a parallel edge/access internetworking strategy positioned for novel use scenarios including, for example, residential or local community networks, office networks, home networks and first-response networks. We introduce OverMesh which is our research platform for instantiation of this architectural vision. OverMesh enables a novel wireless mesh internetworking system and realizes network centric computing. The following are collectively the differentiating properties of OverMesh against traditional networking, peer-to-peer computing systems or ad hoc and mesh internetworking systems: -
Infrastructure-free: a peer-to-peer edge/access system is suggested over current hierarchical physical formations. We position an aggressive convergence strategy for node computation, network processing and data storage. Any physical node may be capable of supporting alternative properties of interconnection, source and sink functions in the network. It is possible to have nodes which take on switching or routing functions, in addition to supporting client or compute functions. Thus, the proposed network would conceptually impersonate that of a sensor network, where network nodes may cooperate in a peer or hierarchical fashion and may have varying forms of resource capability. The nodes and the network can support much larger demand profiles and can scale to grid-level capabilities. Further, the network can exhibit semi-static topology, where fixed topology is formed of stationary nodes and variable topology fluctuates according to the degree of community mobility;
-
Network virtualization: based principally on a robust distributed virtual machine overlay strategy, the use of computational service overlays would enable a computational model for provisioning and managing network structure and resources, application services and distributed network services. In addition, a richer model for node virtualization is also to proposed to supporting the creation of clustering functions, functional spawning, partitioning or integration and personalization of alternative form factor functions;
-
Emergent control and manageability: to achieve the level of robustness and resilience seen in today’s internetworking systems under a decentralized networking system, there is an assumption of functional distribution of all three network planes (i.e., transport, control and management). We argue that it is necessary that some services exhibit behavior typically seen in emergent biological systems and view this aspect to be key to the longterm success of ubiquitous computing systems – to discover, adapt, predict and learn distributed network state. Similar to [8], we suggest the use of a distributed abstraction to facilitate a stored prediction system, the use of performability metrics [22-24] to capture and store operational state, and finally, alternative machine learning techniques to off-load human-dependency on operational management and provisioning;
-
Cooperative and adaptive end-to-end control: to support a horizontal (end-to-end) and vertical (node-level) systems orientation to scale and adapt wireless communications, we believe the end-to-end principle and innetwork control must converge. We call for tighter layer integration and automation of application-to-network control and management. Our approach to deal with this issue is cross-layer adaptation across network nodes and network overlays. To illustrate this approach, a cross-layer overlay searching algorithm is proposed to take a short physical route to rapidly process the information. In addition, a proposed cross-layer monitoring overlay can provide the measured information of underlying networks to all nodes and other overlays. These example techniques illustrate how we can enables nodes and overlays to be more aware of underlying network conditions.
The following section elaborates these topics in greater details and further categorizes them into architectural vectors supporting the vision of network centric computing.
4
3. OverMesh: Architectural Vectors In this article, there are several new ideas and associated challenges to realize network centric computing. We acknowledge that there are barriers, which are noticeably absent in the above discussion. Some of these include the cooperation with legacy networks or traditional Internet nodes, security or privacy considerations, spectrum and competing radio network considerations. Our premise is that wireless mesh networks are inevitable, and while many of these barriers have the attention of the research community or policy makers, our research focus will key on scalability dimensions that are oriented more towards scaling mesh networks, realizing overlay service architectures, resource and resiliency management and end-end optimization challenges. As depicted in Figure 4, the OverMesh conceptual architecture is presented. In what follows we present the major vectors of the proposed architecture and our core research.
Figure 4: OverMesh conceptual architecture
3.1. Wireless Mesh Networks: Collapsing the Edge and Access Wireless networks started from a centralized model that holds the potential for bottlenecks, latency and a single point of failure, while wireless mesh networks are emerging as an alternative to wireless switching. Mesh networks distribute intelligence from switches to access points by incorporating a grid-like topology. As mentioned earlier, the development of this topology parallels the architecture evolution in the computer industry. There has been a long history of research on mobile ad hoc networks where every node relays packets for others and there is no need of a fixed infrastructure to manage the network. Recently, wireless mesh networks have been actively studied [25]; supported by increasing interest from the industry. A mesh network can be automatically configured on different wireless access systems. Standardization of such mesh networks has already started, for example, the IEEE 802.11s mesh network for WLAN [26], the ZigBee mesh network for low rate WPAN or sensor networks [27], the mesh mode for IEEE 802.16 [28]. The next generation mobile wireless network is expected to be a hybrid of various mesh networks in different scales which provides a general flat mesh network that can be further integrated with the Internet. In an OverMesh network, nodes are allowed to communicate with other nodes without being routed through a central switching point, eliminating centralized failure. For a network to intercommunicate in a mesh topology, the nodes' selfdiscovery features must first determine whether they are to serve as access points for wireless devices, as backbones for traffic coming from another node, or a combination of roles. Individual nodes locate their neighbors using discovery query/response protocols. Once the nodes recognize one another, they measure link quality and performance metrics such as received signal strength, throughput, packet error rate and latency. This information must be communicated among the neighboring nodes, while this communication must consume minimal bandwidth. Based on the signal values, each node then selects the best path so that the optimum quality of service is obtained at any given moment. The network discovery and path selection services must be light-weight, run in the background and consume no more than 1% to 2% of the available bandwidth. Each node maintains a current list of neighbors and frequently re-computes the 5
best path. If a node is migrated or is removed from the network, the adjacent nodes can quickly reconfigure their routing tables and re-compute paths to maintain traffic flow when the network changes. This self-healing or failover features set mesh topologies apart from hub-and-spoke networks. Moreover, because mesh networks rely on management, control and discovery messages, they must be able to secure this traffic along with user traffic. In-band messages, secured within encrypted tunnels, remain free from eavesdropping and similar attacks. Standards-based security techniques, such as 802.11i and Advanced Encryption Standard (AES), ensure that only authenticated wireless devices and nodes are connected and corresponding traffic properly encrypted. While mesh networks could be deployed based on different wireless protocols, we will focus the discussion in the remainder of the section on the emerging pre-standard 802.11s WLAN mesh network standard, and then discuss extensions of such networks with support for other wireless protocols as well. A WLAN mesh network is defined as a wireless distribution service consisting of mesh points interconnected through wireless links with communication enabled through the underlying WLAN communication protocols and the available mesh services. Different types of devices can exist in such a mesh network. A mesh point provides 802.11 conformant MAC and PHY interfaces to the wireless medium providing the mesh services. Mesh points can connect to other mesh points or to other mesh access points. A Mesh Access Point (MAP) is a mesh point that is also an access point providing additional Basic Service Set (BSS) to support communication with simple wireless stations. Such stations do not provide mesh or access point services but indirectly participate in the WLAN mesh by connecting to a MAP, and they may provide Direct Link Protocol (DLP) services in 802.11e if the direct link between two stations is considered as a better path compared to a route through an access point. The stations can leverage existing services such as establishing connectivity and powersaving provided by the MAP. No new BSS functionality is specified for connectivity between a station and a MAP. As stations move around in the mesh network, they can associate with different MAPs for primary connectivity into the mesh network. A device that serves as a MAP can also be mobile and the stations associated with such an access point could choose to re-associate with other access points as link conditions vary, while new stations may choose to associate with such a mobile MAP. Finally, a mesh portal is a mesh point that specifically serves as an entry or exit point for packets in the network and routes packets into or out of the mesh network from other parts of a distribution service or non-802.11 networks. Mesh networks should be designed to provide for scalable capacity as the number of mesh nodes increase in the network. It would be useful to minimize the number of edges in an interference graph for the mesh network to increase the number of simultaneous transmissions possible (an edge between two nodes in such a graph signifies that the nodes cannot transmit at the same time as their transmissions will interfere with each other). Power control can be used to reduce the range of interference provided by a wireless transmission, so that an optimal reduced power can be used for transmission for the required signal-to-interference-plus-noise ratio (SINR). When multiple channels are available, mesh nodes could configure their radios to transmit data in different channels and thus transmit simultaneously even if they are in close proximity of each other [30]. Each mesh node can have multiple radios such that these radios could be configured to receive and transmit on different channels, at the same time, for increased capacity in the network [31]. MIMO antennas can be used in each radio to provide increased capacity at the physical layer with the use of multiple antennas for transmission and reception [29]. Multiple antennas can also be used to provide increased range in the wireless network. In addition, one could consider hybrid mesh networks that provide support for multiple wireless protocols (such as WLAN, UWB, Cellular, and WiMAX) operating in different non-interfering frequency ranges. Communications using different wireless protocols may exist simultaneously and the nodes in the network will need support for radios to support various protocols or have software-defined-radio implementations to reconfigure radios to different wireless protocols dynamically [16]. When reconfiguring a radio, the cost associated with switching must also be considered as that reduces the time that the radio is available for wireless communication. Therefore, there are several dimensions to consider improving capacity in the network, such as dynamic power management, the use of multiple channels [32], or multiple radios [33], or multiple antennas, or multiple protocols for wireless communications. The above techniques can help to improve the capacity of the mesh network to carry intra-mesh traffic. However, one must also consider the amount of extra-mesh traffic relative to intra-mesh traffic. It is possible that a large fraction of the traffic in the mesh has destinations external to the mesh that must exit through a mesh portal. Similarly, a large faction of the overall traffic may arrive into the mesh through a mesh portal. Therefore, although the network may be scalable in its ability to carry intra-mesh traffic, the network may experience a significant bottleneck for wireless transmissions and receptions at the mesh portals depending on the amount of extra-mesh traffic in the network. This could lead to the design of fat-tree mesh networks with the higher capacity closer to the root of the tree such as at the mesh portals, with reduced capacity for nodes further away from the portals.
6
In addition to capacity-enhancing techniques for network scalability, mesh networks should provide capabilities for adaptive dynamic routing and fault tolerance. Adaptive dynamic routing can be used to find alternate routes when intermediate nodes fail or when link conditions deteriorate on an existing route. When a mesh portal fails or if the demands on a mesh portal exceed its capacity, one must dynamically route traffic to other mesh portals. This could cause additional delays in transmission due to longer routes taken in the network. When a mesh portal is saturated, one could route delay-sensitive traffic through the mesh portal, while re-routing other traffic along longer paths to other mesh portals, ensuring that end-to-end QoS constraints are satisfied for competing flows through the network. Heterogeneous mesh networks can provide additional flexibility. In a hybrid WLAN-WiMAX heterogeneous mesh network, for example, a mesh node with a WiMAX radio could potentially consider routing data through alternate paths such as through a WiMAX network to communicate data out of a mesh network when the WLAN mesh portals themselves have saturated links. A WiMAX base-station node could then serve as a mesh portal, or it could just forward data to another WLAN mesh portal that is further away. For intra-mesh traffic, even if source and destination nodes have the ability to connect through a WLAN mesh network, one can optimize transmission flows for intra-mesh traffic by using a WiMAX network with increased communication range to reach destinations faster by traversing intermediate paths in the mesh network through the WiMAX portal.
3.2. Network Virtualization With the growing attention to peer-peer [20], content delivery [34], PlanetLab [21, 35], grid computing systems [36, 37], and the challenges visible in today’s Internet evolution, there is a subtle architectural shift underway in the Internet from the packet abstraction paradigm to an overlay paradigm. And, while many commercial overlay services have been positioned primarily as a vehicle for computational processing, the use of overlays to provision communication services or for management purposes has started to gain momentum from the research community [38-41]. This section extends the discussion on network virtualization and positions two novel usage model directions of virtualization to enable network centric computing. 3.2.1.
Exploiting Computational Overlays
A key aspect of the OverMesh research is the development and use of a distributed facility for network provisioning and service deployment. We view computational overlays or distributed virtual machines as a template to provision services to address distributed systems challenges in heterogeneous and decentralized networks. The positioning of computational overlays has been recognized [42-44] as an abstraction opportunity to re-architect the Internet. Our premise for the use of overlays is that a similar, organized and layered architecture will be required to support provisioning and managing services or resources in a network centric computing system. In such a model, the service architecture would have some structure and organization in a similar way that processes are spawned or inherited in an OS (e.g., “parent-child”), or IP is layered in packaging (e.g., “envelope”) for basic or enhanced delivery or reception of packets. In an end-to-end or distributed scenario, how do you provision services or manage resources across the platform and over the larger “virtual slice” of the computing environment? What is the service level model for ensuring reliable, consistent service capacity (i.e., performance) across storage, computing and bandwidth? As a building block, DVM overlays can provide for network programmability, provisioning and management of network services or resources. The authors argue that this as a necessary breakthrough area towards addressing the resilience and robustness issues that impede the broader adoption and integration of decentralized networks with current Internet systems. As illustrated in Figure 5 (a, b, c, d), overlay formations can exist to support structural services, traditional network services, management services, content or peer-to-peer services. We also motivate the development of alternative overlays which may match the “elephant or mice” packet flow analogy, but oriented to meet the demanding, dynamic requirements brought on through node mobility, node migration, community formations or mobile applications. Thus, we envision three forms of overlay provisioning classes: i.) hard overlays, fixed network and long duration, ii.) soft overlays, variable network and long duration, and iii.) short-lived overlays, variable network and short duration. An important objective of the OverMesh research is the feasibility of such architecture to deliver DVM layered services over wireless mesh networks. We will investigate how these services may be bundled or organized in scalable fashion or in such a manner as to be easily programmable by an application developer, service provider or even an end-user to deliver novel applications or networked capabilities. Additionally, we will examine the use of alternative types of overlays to study their interactions under different wireless mesh networking constraints (e.g. traffic load or signal fading conditions and alternative network configurations) and node constraints (e.g., compute load, IO load and large number of virtualized services). 7
Figure 5: Overlay service architectures (a) Network structure, (b)management services, (c)collaborative,(d) enhanced network services
3.2.2.
Node Customization
Today, the Internet has a highly physical orientation with clients, servers, router, switches and various forms of internetworking ‘boxes’ forming its source, sink and interconnection structure. End-user devices can be oriented towards personal device assistants, cell phones or any number of alternative client devices to connect, source and sink Internet traffic. Nodes and their physical position in the network have a one-to-one mapping (generally) to their specific function or service which they support in the network. In this article, we propose to move away from this orientation and position a more aggressive convergence strategy for node computation, network processing and data storage. Therefore, any physical node may be capable of supporting alternative properties of interconnection, source and sink computation and storage inside the network. Alternatively, the use of virtualization3 technologies is proposed to provision alternative forms of networking as enabled by the owner of that generalized and virtualized device. While hardware and software-based virtualization technologies [47-49] are broadly gaining commercial adoption; we call for node virtualization on an entirely new level – towards converging, partitioning and integrating computation, communications and storage resources to deliver a new class of node architectures for network centric computing. In Figure 6 (a, b, c, d), we are illustrate four alternative forms of node virtualization, each of which directs a unique form of node formulation and construct. Figure 6a depicts the traditional distributed set of servers formed by combining several nodes to demonstrate clustering through virtualization. Fig 6b shows the different forms of interconnection devices today, such as switches, access points and routers, thus the ability to simultaneously spawn and partition legacy devices and new forms of interconnection devices. The integration of compute and communications may facilitate new platform functions capable of supporting network processing and computational processing through intelligent network computers, as illustrated in Figure 6d, and finally, classic client devices as shown in Figure 6c.
3
The practical realization of this topic extends beyond the subject of virtualization. Other enabling technology areas which we find suitable (e.g., multiple processor cores and cognitive or soft radios) are missing here, but are duly acknowledged as opportunities to realize a vision of a customizable network node. 8
Figure 6: Customized Node Schemes a.) Distributed servers, b.) classic interconnection device, c.)classic intelligent network computer, d.) classic client device In summary, a richer model for node virtualization is proposed for supporting the creation of clustering functions, functional spawning, partitioning or integration and personalization of virtualized form factors and functions. Thus, we advocate personalizing devices to orient to the user’s particular need through device virtualization, rather than device physical instantiation. This model aligns with the notion of empowering the user and releasing their dependency on provider physical networks or equipment; further enabling the end-user’s free-association to a broader set of services or virtual resources within their reach or awareness.
3.3. Emergent Network Control & Management As articulated in [8, 46, 52, 53], the tasks of managing and provisioning Internet networks remain a human-centric activity for network administrators. The complexities of managing heterogeneous and autonomous networks have posed [53] scalability challenges exceeding human capacity. An interesting observation posed in [54] suggests designing systems based on the notion of predictability and its relation to process driven or data-driven descriptions. The current Internet’s design is highly process-driven and deterministic and lacks the online flexibility to change its design against evolving demands and conditional event dynamics. We contend that its current design methodology creates the challenge of human-centered operations and management solutions, and therefore, we suggest the introduction of online predictability using both emergent and self-organizing techniques [55-57]; advocating a non-traditional approach to network control and management. However, this is consistent with many of the works [18, 20] in the areas of ad hoc and peer-peer systems and emerging trends [45, 46, 58] to pursue social or biologically-inspired networking solutions. In the OverMesh system, manageability and resiliency are key operational pillars in its network design and architecture. We bring these separate operational topics together mainly because their solution space overlaps. We employ a data (or state)-driven model to our framework – an emergent approach and adopt the following working definition [55]: “A system exhibits emergence when there are coherent emergents at the macro-level that dynamically arise from the interactions between the parts at the micro-level. Such emergents are novel w.r.t. the individual parts of the system.”
We adopt the emergent philosophy partially due to the anticipated structure and dynamics of the network system – one that is decentralized, robust, highly social, micro-macro effectual, open, and flexible. In essence, an emergent network management and control system matches our proposed networking system. The first observation of these networks (i.e., mesh or sensor) is the lack of an organized or fixed structure, by default. Secondly, there is a common notion of selfdiscovery, organization and network bootstrapping. Finally, these networks may change dynamically for self-healing purposes or to accommodate community fluctuation. The challenges and solutions we anticipate are parallel to those demonstrated in current peer-peer [20] and grid computing [37] systems. Finally, our emergent strategy employs self9
organizing aspects in the various subsystems, where there is clear independence, systems closure, and minimal external control. Figure 7 depicts the representation of an OverMesh emergent framework to help guide our approach in what follows.
Figure 7: Emergent framework 3.3.1.
Performability State Management
A key concept in describing a data-driven system or component is its state. Network state can be static or dynamic, local or global, or operational or policy-based. In what state is a network node, a flow, a link, a path, a route, or a networking subsystem? For any given resource or service, there are three universal operational descriptions that can characterize ‘goodness’ – availability, reliability and performance. In our work, we propose to represent and describe the state of the network and its components or subsystems using these operational data descriptions. A well known, integrated approach to this representation is based on performability [22]. This operational research concept has been used in modeling and predicting computing systems, but also more recently in modeling wireless and mobile systems [23, 24]. Specifically, we intend to use performability as a statistical methodology to characterize networking state due to changing services, nodes, or network dynamics and investigate fundamental research in statistical and emergent techniques for storing, predicting, and learning distributed state. The following are the key data representation requirements supporting the OverMesh network centric computing environment: i. (Performance basis) constant or static data representations of distributed resources or services ii. (Reliability basis) semi-static or changing data representations of distributed resources and services iii. (Availability basis) abrupt or highly dynamic data representations of nodes, links or topological services 3.3.2.
Stored (Knowledge) Prediction
Relieving human burden from the complexity of managing decentralized networks seems intractable, but it is an essential requirement to achieving true emergent and self-organizing systems. We believe a prerequisite to this is a structural adoption of a stored prediction system that can capture performability-based descriptions from the fluctuating state of the networked environment. The OverMesh emergent system must be capable of capturing (online) and storing event-level representations as well as time-dependent (i.e., historical) data representations. It must be capable of storing and reproducing the data and the knowledge without loss, with redundancy, and while the network state is under constant dynamic flux. Similar to the concept in [8], it is essential to enable an online prediction facility which is integrated into the design of the network. Fundamental to achieving this is a stored prediction knowledge base, a decentralized structure for managing networking state, encompassing time, space and associative relations and a system capable of compiling, storing, and managing real-time events to historical knowledge (or experience). We envision, conceptually, a structure capable of supporting i.) state correlation through time and space associations, ii.) knowledge through hierarchy and abstraction, and iii.) learning through evolution and historical feedback. An effective and scalable architecture for a stored prediction system to support performability-based data representations is exploratory research for the OverMesh team.
10
3.3.3.
Distributed Learning and Evolution
We believe that novel approaches through machine learning, biological, and nature-inspired models are needed by the research community to support human-supervised versus human-centered network control and management. This requirement comes both by necessity to increase operational resiliency for the users of the networked system and by architectural value to reduce the dependency on humans to operate and manage extreme distributed systems. The proposed networking system for OverMesh assumes a high-degree of intra-networking and inter-networking movement4and we view them both as a challenge for optimization and an opportunity for scalability [3]. OverMesh nodes can be online, mobile, hibernating or off-line. While the topology may be irregular and fluctuating in path selection; they may also increase the network diversity for provisioning. In support of a knowledge-based stored prediction system, we consider the fields of statistical computing and bioinspired approaches to address state representation and to manage distributed complexity. For example, in the case of constant data representations, random neural networks for reinforcement learning are feasible technique which has been applied in the context of wireless QoS [59-62]. Bayes network techniques have been proposed for predicting state through distributed belief propagation [63] and multivariate statistical process control techniques for network-wide anomaly detection [64]. Evolutionary computing techniques (e.g., genetic algorithms) may be a suitable approach to support the introduction or integration of incoming or departing nodes into the OverMesh networked community. On the reverse side of the control system where the system responds, either reactively or proactively, [65-71] are unique alternatives to controlling distributed systems. Through predictive learning and feedback, knowledge can be returned into the system to continuously build and distribute intelligence and sophistication – experience or wisdom. In this work, the emergent system would be capable of building new knowledge and storing this into more abstract forms of intelligence. A key characteristic we anticipate drawing out of the OverMesh emergent system is the intrinsic ability to have local (e.g., local service, network component or subsystem) behavior cooperate with distinct and independent behavior exhibited by the global system (e.g., network-wide service, internetworking community). Moreover, the global system may have an entirely different set of objectives than the local entities’; yet their synchronicity and independence should be evident. This is consistent with our definition of an emergent system and the direction we view as a vital change to how network centric computing systems should be designed for control and manageability.
3.4. Cooperative and Adaptive Overlay Services and Applications While current overlay systems are mostly based on wired networks, when an infrastructure-free network involving both wired and wireless networks is considered, current overlay techniques may not provide favorable end-to-end performance. This is mainly due to the separation between the upper, virtualized network overlays and the lower network/MAC/PHY layers. As a result, one can experience significant delays and high packet loss rate, especially when wireless networks are considered. Our approach to deal with this issue includes both distributed end-to-end cooperation
Figure 8: Cross-layer overlay services
11
among nodes and adaptive cross-layer control in each node. The following subsections will introduce several overlay services and applications supported by OverMesh, as illustrated in Figure 8. 3.4.1.
Distributed Searching Overlay
A searching overlay provides a common lookup service to various applications such as information queries and distributed file storage and sharing. A (key, value) pair is stored in a randomly-selected node. However, it can be found by any node in the overlay using a search algorithm. The most efficient overlay search algorithms are based on the distributed hash table (DHT [38]). Each overlay node maintains a small overlay routing table for finding the destination with the shortest path length of complexity O(logn), where n is the network size. But these overlay search algorithms make the underlying network transparent to the overlay and only find the shortest search path in terms of the number of virtual hops in the overlay. Consequently, the search may not be efficient for overlays on wireless networks. Given factors such as bandwidth availability, sharing of the medium, power, mobility, dynamically varying topologies and link conditions in wireless mesh networks, the search algorithm should consider the network condition in lower layers and use minimal network resources in order to quickly find a requested key. At the network layer, the most widely used ad hoc routing protocols, such as AODV and DSR, are based on broadcast. Whenever a source node wants to find a route to a destination node, the source node broadcasts a route request until the destination node receives the request and sends a reply. The complexity of broadcasting at the network and lower layers is O(n). We know that the complexity of DHT-based overlay search algorithms is O(logn). So when we apply the current overlay search algorithm on the wireless mesh networks, the complexity becomes O(nlogn). The new overlay search algorithm in OverMesh achieves the complexity of O(n) by taking advantage of the network layer broadcast to route the overlay search request. When a node knows a key k and wants to find its corresponding value, it first maps k to a virtual network address vAddress. The source node then broadcasts a route request for this virtual network address. Any node in the search overlay will check the received route request and compare the requested virtual address against all the keys it has. If the same key is found, the corresponding value will be sent back to the source node as a route reply. Figure 9 illustrates overlay searching on the wireless network when a node A issues a request for some data stored at some other node D. The searching process involves two loops. The outer loop is the search in the overlay, the short virtual path A→B→C→D in the overlay can be found by some DHT based overlay searching algorithm. The inner loop is the real network routing. For example, in order to send the search request from A to B, it should be routed on the physical network. But the virtual neighbors A and B may be physically far away from each other. In a wired network, the overhead in the underlying network is not a big concern due to the high speed cable and dedicated routers. However, for the multiple-hop wireless mesh network, in order to find a physical route for each virtual hop, the source node has to broadcast a route request to all its neighbors repeatedly until the destination is reached. This makes the previous two-loop overlay searching algorithm on wired networks inefficient on wireless mesh networks. Figure 10 illustrates the advantage of using the proposed cross-layer search algorithm. The shortest physical route can be quickly found. This new algorithm requires vertical cooperation between the network layer and the overlay.
Figure 9: Overlay searching on wireless network
Figure 10: Cross-layer searching on wireless network
4
In the context of an OverMesh networking, a domain specific boundary and degree of portal capabilities is undefined. We leave this ‘boundary’ open, while there are clear implications to the discussion on optimization and scalability. 12
3.4.2.
Cross-layer Network Measurement and Monitoring Overlay
Given the resource constraints in underlying wireless networks, many services and applications supported by an overlay should be made more aware of current network conditions via cross-layer information exchange [72-74]. Instead of conducting cross-layer operations in every node, a dedicated overlay on top of a subset of nodes can monitor underlying network information such as link quality and provide the information to all nodes and other upper-layer overlays. In this case, each node being monitored measures the link quality between itself and its neighbors periodically. To guarantee that all nodes in the system are being monitored, each node must be in the vicinity of at least one of the members in the monitoring overlay. Monitored metrics could include, but not limited to, information such as effective link throughput, packet error rate, search query response time, end-to-end transmission delays, received signal strength, SINR, modulation and coding scheme being used for transmission on a link and average number of retransmissions. The monitored information can be queried by other overlays or applications with the help of the distributed searching service. It is possible that the collected data can be accessed from an external network through the help of gateways or mesh portals. 3.4.3.
Positioning Overlay
Positions of mobile nodes in OverMesh can be calculated by a positioning overlay. For a node that is stationary, its position can be pre-measured. Otherwise, it requires positioning hardware such as GPS to find its own location in realtime. The node will measure and record the distance to its neighbors periodically. This can be based on the link quality or by any other ranging techniques. For example, by measuring the received signal strength, the distance between the transmitter and receiver can be estimated based on a given radio propagation model. The distance estimation can be refined by comparing the actual distance and estimated distance between two nodes that are both in the positioning overlay because their actual locations are already known. A node A that is not in the overlay requires the help of nodes in the overlay to find its position. Node A queries every neighbor B who is in the positioning overlay for their locations and respective distances relative to each other. Given such information, node A can estimate its own position via triangulation or any other position-estimation techniques.
4. OverMesh Platform We have developed an advanced platform supporting the OverMesh architecture and to carry out our research. In a decentralized networking environment, applying existing virtualization and overlay solutions that were designed for stationary servers and the wired Internet is not straightforward. In addition, while there have been numerous research and industrial efforts on mobile ad hoc networks, there are no standard implementations for infrastructure-free wireless mesh networks. Our current implementation of the OverMesh platform is an integration of the PlanetLab virtualization solution and a pre-standard IEEE 802.11s prototype. The pre-standard IEEE 802.11s mesh network system provides complete support of metric-based multi-hop routing at link layer, neighbor discovery, link quality measurement, and a user interface. We employ the PlanetLab architecture to facilitate network virtualization and overlay maintenance of the OverMesh system. However, we re-engineered the existing PlanetLab solution to operate on a local, private wireless mesh network. Moreover, this open platform facilitates the development and deployment of various overlays services and serves as a test bed for on-going OverMesh research. The implemented OverMesh platform includes the following components: 1. OverMesh Central maintains the installation and status of all OverMesh nodes. A web site for managing the whole system is hosted at the OverMesh Central. 2. OverMesh nodes provide distributed services through cooperation. Each node can host multiple virtual machines. Nodes communicate through multi-hop wireless communications and can be either stationary or mobile. 3. Clients connect to the nearest OverMesh node through wireless or wired links. They do not need to function as a service provider, but clients can participate in the multi-hop wireless networking if they are mobile or communicate through wireless links. Clients are consumers and are not managed by the OverMesh Central. 4. Gateways, serving as a portal, connect an OverMesh local domain to external networks such as the Internet or other wireless networks. The gateway function can be integrated into the OverMesh Central, as necessary. Figure 11 illustrates the current implementation of the OverMesh system stack. The wireless physical communication network resides below the network layer and can support multiple distributed virtual machines used to provision alternative overlay network services and applications. To make more efficient use of limited resources in wireless networks, we have implemented several cross-layer functions as overlays as described in Section 3.4. In these scenarios, the status of underlying networks is provided to the upper layers for management and control purposes. 13
Figure 11: An OverMesh node
As mentioned above, we have implemented several overlays and services on the OverMesh platform. One of the overlays provides a distributed directory service. The service is based on OpenDHT [75]. Through this service, any client connected to OverMesh network can query the network address of another client by its name. After getting the reply, the client can call another client and start a voice over IP application over the mesh network. We have also performed additional testing and analysis to show that network-wide performance is affected by the ad hoc routing protocol, network load, mobility and even human activities as would be expected. We have conducted a number of experiments to test the cross-layer overlay search algorithm. Some of these results and more detailed analytical results will be reported in a separate paper. The following is a short list of research and development topics that we are currently pursuing to enhance the OverMesh platform: - Communications support for different infrastructure-free wireless networks or multi-radio systems. The current OverMesh platform is based on the IEEE 802.11s WLAN mesh network, but the same concept can be integrated with other forms of infrastructure-free networks such as sensor networks, WiMAX mesh networks through multiple radios and protocols. - Network virtualization - while the current OverMesh platform utilizes the vserver virtualization solution used by PlanetLab, we are investigating and testing alternative virtualization techniques such as Xen [49] and VMWare [48]. As we encountered with PlanetLab, these systems will also need to be customized to integrate with different wireless networks, hardware, and operating systems. Much of our pending overlay research will depend on a robust and secure platform for network virtualization. In addition to the aforementioned cross-layer searching and monitoring overlays, we are currently investigating more cross-layer approaches to optimize resource usage and to improve network efficiency in the OverMesh environment. - Distributed network control and management - on an operational level, each new OverMesh node requires a customized boot CD and needs to contact the OverMesh Central for authorization to join the system. In the future OverMesh system, these requirements will be removed and the difference between the OverMesh Central, node, and client will disappear. Every node or client may automatically participate in an overlay and provide services without any centralized controller. Supporting our research, we intend to employ a number of peer-to-peer style services, from which emergent services may be employed. This includes the concept of a distributed stored prediction knowledge base and the functions necessary to sense and actuate network control and management. - OverMesh Toolkit - we are working on an open source toolkit so that researchers can easily build their own OverMesh system and conduct various research activities on wireless mesh networks. With an open toolkit, we hope to foster this direction for network centric computing and research collaboration with partners. While the current implementation of OverMesh has validated several preliminary objectives, there are a number of enhancements pending on the OverMesh system necessary to achieve our research goals and address the feasibility challenges we will define in the next section.
5. Research Challenges A significant challenge in moving towards network centric computing is scalability, as the size of a decentralized OverMesh network may be unpredictable or planned. In order to scale the environment, we deem necessary the 14
adoption of capacity scaling techniques such as using MIMO antennas, multiple channels, and multiple radios to support larger-scale mesh deployments. In addition, cognitive or smart radios, an effort aiming at enabling a wireless device to sense its environment and then alter its power, frequency, modulation or other parameters so as to dynamically reuse available spectrum, can further improve spectrum efficiency and network scalability. We deem this work necessary, but beyond the scope of our proposed research and focus. In addition, mesh networks should provide capabilities for transport (e.g. dynamic routing) and service diversity under conditions of load variations, failures and network resource constraints. In this environment, dynamic conditions range from variability in topology of mobile infrastructure, selection of alternative radio systems, radio signal fading conditions and rogue or uncooperative nodes. Much of the scalability work will be done through pending simulation, although we have demonstrated some initial validation work towards this. The above complexities may, however, be partially offset by the creation and deployment of virtual overlay networks[76] to help manage scalability by deploying management service overlays or by reducing network complexity through virtual structures to partition or separate communities of interest. This brings us to the second challenge for our research in realizing network virtualization to support the distributed overlay functions (a.k.a., distributed virtual machines) and node level virtualization. On a network-wide level, there are clear challenges of provisioning service or functional overlays – issues such as overlay service discovery, programming overlays in addition to resource management are complex issues that have no formal rigor for structural organization or resilience. These concepts have parallels to process structures in operating systems (OS), but lack the rigor clearly evolved over many years of operating system research. Further, as we argue for a ‘generalized’ node vision based on a more extreme virtualization paradigm, we observe these challenges as both a hardware and software opportunity. First, the organization of physical hardware supporting compute processing, network processing and data storage must have a higher degree of integration. Secondly, a schematic like view of the hardware must be software-defined in a manner that allows the programmer or administrator to design entirely new constructs of the platform, configure or re-configure existing node resource or services in novel ways. To formally characterize the envisioned network centric computing environment with sufficient predictability can be another challenge. Each node in our system may have its unique complex internal composition showing different external behavior. Interactions between nodes may change the state of each node in real-time and in complex ways; new nodes can join and old nodes can leave a network in a highly random fashion, and global emergent behaviors may be intractable and unpredictable. Thus, ensuring network robustness and resilience in this environment is compelling research, but also currently more speculative than substantive. Under our proposed OverMesh internetworking system, this type of complex distributed system cannot be supported with existing human-driven operational models. As discussed in Section 3.3, we will investigate these challenges using non-traditional, human-supervised methods for management and control, including promoting more emergent and peer-to-peer techniques. Closely related, the challenge of managing service consistency or coherence on a local or network-level is a problem which we will also investigate. The areas of cross-layer functionality should help to bridge the application-to-network gap or discontinuity challenge exhibited in wireless or mobile systems. Nevertheless, there are well-documented [77] problems that may be introduced, when we make ad hoc cross-layer changes including optimization conflicts, control instability or code-level intrusion As a research platform, the OverMesh prototype has more fundamental or tactical challenges, which we are addressing – our goal is to build a suitable platform toolkit to carry-out our research and publish for broader research community participation. Today, OverMesh is engineered with the PlanetLab ‘low-grade’ virtualization and non-scalable centralized service model. This can be improved by using [48] or [49] and [78] for safe and secure virtualization, in addition to a peer-to-peer service model, respectively. In addition, we will introduce new radio capabilities, as mentioned earlier, towards increasing the transport diversity of the wireless mesh network in support of the scalability work.
6. Conclusion The decentralization of the telecommunication system to the edge of today’s hierarchically-formed networks is inevitable with the emergence of key wireless technologies and the proliferation of mobile computing devices and powerful usage models. This paper proposes a novel direction and vision for the Internet through network centric computing and presents an architectural staging of our current and pending research and development work to instantiate this vision. The OverMesh system employs a wireless mesh internetworking system supporting distributed virtual machines for constituting overlay services. We call for new ways to address to address the extreme distributed 15
systems challenges that a network centric computing environment poses and thus, call for a new framework based on emergent services as well as cross-layer techniques to manage and control distributed state and end-to-end coherence, respectively. To this end, we have implemented a virtualized, converged computing and communications architecture for OverMesh and have identified several challenges from which we have embarked.
7. References [1] M. Weiser, “Some Computer Science Issues in Ubiquitous Computing,” Communications of the ACM, vol. 36, no. 7, July 1993, 74-84. [2] D. Tennenhouse, “Proactive Computing,” Communications of the ACM, vol. 43, no. 5, May 2000, pp. 43-50. [3] A. Lippman and D. Reed D, “Viral Communications,” Executive Summary Report, 2003. [4] D. Isenberg, “The Dawn of the Stupid Network,” ACM Networker, vol. 2, no. 1, February/March 1998, pp. 24-31. [5] C. Baldwin and K. Clark, Design Rules: the Power of Modularity, MIT Press, 2000. [6] C. Fine, Clockspeed: Winning Industry Control in the Age of Temporary Advantage, Perseus Publishing, 1999. [7] C. Christendsen, The Innovator's Delemma, Harvard Business School Press, 1998. [8] D. Clark, C. Partridge, J. C. Ramming, and J. T. Wroclawski, “A Knowledge Plane for the Internet,” in Proc. of SIGCOMM, 2003. [9] S. Blake et al., “An Architecture for Differentiated Services,” RFC 2475. [10] S. Shenker, et al., “Specification of Guaranteed Quality of Service,” RFC 2212. [11] “Mobility for IPv4”, MIPv4 Working Group, http://www.ietf.org/html.charters/mip4-charter.html. [12] “Public-Key Infrastructure (X.509),” PKIX Working Group, http://www.ietf.org/html.charters/pkix-charter.html. [13] D. Watts, Six Degrees: The Science of a Connected Age, 1st Edition, W.W Norton & Company Ltd., 2003. [14] A-L. Barabasi, Linked Plume, Penguin Group, 2003. [15] D. Reed, “That Sneaky Exponential—Beyond Metcalfe's Law to the Power of Community Building,” Context Magazine, Spring Issue, 1999. [16] R. Murty, “Cognitive Software-Defined Reconfigurable Radios: Smart, Agile, Cognitive, and Interoperable,” Technology@Intel Magazine, July 2003. [17] MIT Communications Futures Program (CFP), Open Spectrum Working Group, http://cfp.mit.edu/ [18] “Mobile Ad Hoc Networks,” MANET Working Group, http://www.ietf.org/html.charters/manet-charter.html. [19] I. F. Akyildiz, W. Su, Y. Sankarasubramaniam, E. Cayirci, “Wireless Sensor networks: a survey”, Computer Networks, vol. 38, 2002, pp. 393-422. [20] D. Milojicic, et al., “Peer-to-Peer Computing,” HP Technical Report, HPL-2002-57, 2002. [21] http://www.planet-lab.org [22] J. F. Meyer J.F, “On Evaluating the Performability of Degradable Computing Systems,” IEEE Transactions on Computers, vol. C-29, no. 8, Aug. 1980, pp. 720-731. [23] E. Gelenbe, R. Lent, and A. Nunez, “Self-aware Networks and Quality of Service,” in Proc. of IEEE, vol. 92, no. 9, 2004, pp. 1479-1490. [24] K. S. Trivedi, X. Ma, and S. Dharmaraja, “Performability Modeling of Wireless Communication Systems,” International Journal of Communication Systems, vol. 16, 2003, pp. 561-577. [25] I. F. Akyildiz, X. Wang, and W. Wang, “Wireless Mesh Networks: A Survey,” Computer Networks Journal, vol. 47, 2005, pp. 445-487. [26] IEEE 802.11s ESS Mesh Network working group: http://grouper.ieee.org/groups/802/11 [27] http://www.zigbee.org [28] http://wimaxforum.org
16
[29] M. Hu and J. Zhang, “MIMO Ad Hoc Networks: Medium Access Control, Saturation Throughput, and Optimal Hop Distance,” Special Issue on Mobile Ad Hoc Networks, Journal of Communications and Networks, 2004, pp. 317-330. [30] A. Raniwala and T. Chiueh, “Architecture and Algorithms for an IEEE 802.11-based Multi-channel Wireless Mesh Network,” in Proc. of IEEE INFOCOM, 2005. [31] R. Draves, J. Padhye, and B. Zill, “Routing in Multi-radio, Multi-hop Wireless Mesh Networks,” in Proc. of ACM MOBICOM, 2004. [32] P. Kyasanur and N. Vaidya, “Capacity of Multi-Channel Wireless Networks: Impact of Number of Channels and Interfaces,” in Proc. of ACM MOBICOM, 2005. [33] M. Kodialam and T. Nandagopal, “Characterizing the Capacity Region in Multi-Radio, Multi-Channel Wireless Mesh Networks,” in Proc. of ACM MOBICOM, 2005. [34] J. Dilley, et al., “Globally Distributed Content Delivery,” IEEE Internet Computing, September/October 2002, pp. 50-58. [35] A. Bavier, et al., “Operation System Support for Planetary-Scale Network Service.” in Proc. of NSDI, San Francisco, March 2004. [36] http://www.google.com [37] Globus Alliance: Open Grid Services Architecture, http://www.globus.org/grid_software/ecology.php#vision [38] H. Balakrishnan, M. F. Kaashoek, David Karger, Robert Morris, and Ion Stoica, “Looking Up Data in P2P Systems,“ Communications of the ACM, vol. 46, no. 2, 2003, pp. 43-48. [39] R. Huebsch, et al., “Querying the Internet with PIER,” in Proc. of the 29th VLDB Conference, Berlin, Germany, 2003. [40] Y. Chu, et al., “Early Experience with an Internet Broadcast Systems Based on Overlay Multicast,” in Proc. of USENIX, 2004. [41] A. D. Keromytis, V. Misra, and D. Rubenstein, “SOS: Secure Overlay Services,” in Proc. of SIGCOMM, August 2002. [42] “The New Net”, Fall 2004 Intel Developer Forum (IDF), September, 2004. [43] L. Peterson, T. Anderson, D. Culler, and T. Roscoe, “A Blueprint for Introducing Disruptive Technology into the Internet,” in Proc. of HotNets I, Princeton, NJ, October 2002. [44] J. Sedayao, et al., “PlanetLab and its Applicability to the Proactive Enterprise,” Intel Technical Journal, Nov. 2004. [45] Committee on Frontiers at the Interface of Computing and Biology, “Catalyzing Inquiry at the Interface of Computing and Biology”, National Research Council, The National Academies Press. [46] Committee on Network Science for Future Army Applications, Board on Science and Technology, “Network Science”, National Research Council, The National Academies Press. [47] R. Figueiredo, P. A. Dinda, and J. Fortes, “Resource Virtualization Renaissance,” IEEE Computer, May 2005, pp. 28-31. [48] http://www.vmware.com [49] http://www.xensource.com [50] P. Gelsinger, “Catching up with Radio-Free Intel”, http://www.intel.com/technology/techresearch/radio/ [51] Autonomic Communications Forum, http://www.autonomic-communication-forum.org. [52] J. Vicente, et al., “Building Intelligent Traffic Engineering Solutions,” in Proc. of Net-Con, 2002. [53] “FCAPS: Fault, Configuration, Accounting, Performance and Security”, http://www.itu.int/, TMN Recommendation M.3400. [54] R. De Lemos, “The Conflict of Self-* Capabilities and Predictability,” SELF-STAR 2004, LNCS 3460, pp. 219-228, 2005. [55] T. De Wolf and T. Holvoet, “Emergence Versus Self-organization: Different Concepts but Promising When Combined,” ESOA 2004, LNAI 3464, pp. 1-15, 2005. [56] P. H. Van Dyke and S. A. Brueckner, “Analyzing Stigmergic Learning for Self-organizing Mobile Ad-Hoc Networks,” ESOA 2004, LNAI 3464, pp. 195-209, 2005. [57] K. Hermrmann, “MESHMdl – A Middleware for Self-Organization in Ad Hoc Networks,” in Proc. of the 23rd IEEE ICDCSW, 2003. [58] Babaoglu, et al., “The Self Star Vision,” SELF-STAR 2004, LNCS 3460, pp. 1-20, 2005. [59] J. Vicente, “IP Radio Resource Control,” in Proc.of International Conference on Management of Multimedia Networks and Services, October, 2001.
17
[60] D. Krishnaswamy and J. Vicente, “Scalable Adaptive Wireless Networks,” Intel Technical Journal, Q4, 2004. [61] E. Gelenbe and R. Lent, “Power Aware Ad Hoc Cognitive Packet Networks,” Ad Hoc Networks Journal, vol. 2, nov. 3, 2004, pp. 205-216. [62] P. Mohaptra, J. Li J, and C. Gui, “QoS in Mobile Ad Hoc Networks,” IEEE Wireless Communications Magazine, June 2003, pp. 44-52. [63] S. Crosby and J. M. Agosta, “Network Integrity by Inference in Distributed Systems,” in Proc. of NIPS Workshop Robust Communication Dynamics in Complex Networks, 2003, Whistler, Canada. [64] A. Lakhina, K. Papagiannaki, M. Crovella, C. Diot, E. D. Kolaczyk, and N. Taft, “Structural Analysis of Network Traffic Flows,” in Proc. of ACM SIGMETRICS, New York, June 2004. [65] K. Kelly, Out of Control, Perseus Publishing, 1994. [66] S. Graupner, A. Andrzejak, V. Kotov, and H. Trinks, “Adaptive Service Placement Algorithms for Autonomous Service Networks,” ESOA 2004, LNAI 3464, pp.280-297, 2005. [67] N. Foukia and S. Hassas, “Towards self-organizing computer networks: A complex system perspective,” in Proc. of the 1st Workshop on Engineering Self-Organising Applications, Melbourne, Victoria, July 2003, pp. 77-83. [68] V. Kalogeraki, F. Chen, T. Repantis, and D. Zeinalipour-Yazti, “Towards Self-Managing QoS-Enabled Peer-to-Peer Systems,” SELF-STAR 2004, LNCS 3460, pp. 325-342. [69] P. Heegaard, O. Wittner, and B. Helvik, “Self-management of Virtual Paths in Dynamic Networks,” SELF-STAR 2004, LNCS 3460, pp. 417-432. [70] D. Krishnaswamy, “Game Theoretic Formulations for Network-assisted Resource Management in Wireless Networks,” in Proc. of IEEE VTC, September 2002. [71] J. Vicente, A. T. Campbell, and D. A. Villela, “Virtuosity: Performing Virtual Network Resource Management,” in Proc. of International Workshop on Quality of Service, London, June 1999. [72] S. Shakkottai, T. S. Rappaport, and P. C. Karlsson, “Cross-layer Design for Wireless Networks,” IEEE Communications Magazine, vol. 41, no. 10, 2003, pp. 74-80. [73] A. Nakao, L. Peterson, and A. Bavier, “A Routing Underlay for Overlay Networks,” in Proc. of SIGCOMM, 2003. [74] Z. Xu, C. Tang, and Z. Zhang, “Building Topology-Aware Overlays using Global Soft-State,” in Proc. of ICDCS, 2003. [75] http://www.opendht.org [76] A. T. Campbell, M. E. Kounavis, D. Villela, J. Vicente, H. De Meer, K. Miki, and K. S. Kalaichelvan, “Spawning Networks,” IEEE Network, vol. 13, no. 4, July/August 1999, pp. 16-30. [77] “Cross-Layer Protocol Engineering for Wireless Mobile Networks: Part 2”, IEEE Communications Magazine, vol. 44, no. 1, January 2006. [78] Intel Virtualization Technology (VT), http://www.intel.com/cd/ids/developer/asmo-na/eng/221874.htm?page=1
18
