Wireless distributed computing: a survey of ...

Wireless  distributed  computing:  a  survey  of  research  challenges     This  is  an  “accepted  article”,  which  is  a  version  which  has  been  revised  by  the   author  to  incorporate  review  suggestions,  and  which  has  been  accepted  by  IEEE  for   publication.           COPYRIGHT  NOTICE:     ©  2012  IEEE.  Personal  use  of  this  material  is  permitted.  Permission  from  IEEE  must   be  obtained  for  all  other  uses,  in  any  current  or  future  media,  including   reprinting/republishing  this  material  for  advertising  or  promotional  purposes,   creating  new  collective  works,  for  resale  or  redistribution  to  servers  or  lists,  or   reuse  of  any  copyrighted  component  of  this  work  in  other  works.    

Published  in:   Communications  Magazine,  IEEE    (Volume:50  ,    Issue:  1  )  

Date  of  Publication:   January  2012   Page(s):  

144  -­  152   ISSN  :  

0163-­6804   INSPEC  Accession  Number:  

12443853   Digital  Object  Identifier  :  

10.1109/MCOM.2012.6122545   Date  of  Current  Version  :  

05  January  2012   Issue  Date  :  

January  2012   Sponsored  by  :  

IEEE  Communications  Society  

1

Wireless Distributed Computing: A Survey of Research Challenges Dinesh Datla, Student Member, IEEE, Xuetao Chen, Student Member, IEEE, Thomas Tsou, Student Member, IEEE, Sahana Raghunandan, Student Member, IEEE, S. M. Shajedul Hasan, Member, IEEE, Jeffrey H. Reed, Fellow, IEEE, Bruce Fette, Senior Member, IEEE Carl B. Dietrich, Member, IEEE, Jeong-Ho Kim, Member, IEEE, and Tamal Bose, Senior Member, IEEE

Abstract— Recent advancements in radio technology provide great flexibility and enhanced capabilities in executing wireless services. One of these capabilities that can provide significant advantages over traditional approaches is the concept of collaborative computing in wireless networks. With collaborative radio nodes, multiple independent radio nodes operate together to form a wireless distributed computing (WDC) network with significantly increased performance, operating efficiency, and abilities over a single node. WDC exploits wireless connectivity to share processing-intensive tasks among multiple devices. The goals are to reduce per-node and network resource requirements, and enable complex applications not otherwise possible, e.g., image processing in a network of small form factor radio nodes. As discussed in this paper, WDC research aims to quantify the benefits of distributed processing over local processing, extend traditional distributed computing (DC) approaches to allow operation in dynamic radio environments, and meet design and implementation challenges unique to WDC with the help of recently available enabling technologies, such as software radios and cognitive radios. Index Terms— Distributed computing, wireless networks and distributed algorithms.

I. I NTRODUCTION

W

IRELESS distributed computing (WDC) exploits wireless connectivity to share processing-intensive tasks among multiple devices. The goals are to reduce per-node and network power, energy, and processing resource requirements, and enable complex applications not otherwise possible. In Section II-A we discuss how some of these goals, particularly energy efficiency, are achieved by applying WDC to a complex data processing task distributed across a collaborative network of radio nodes that possess computing capabilities. To achieve this, WDC research aims to extend traditional distributed computing (DC) approaches to allow operation in dynamic radio network environments, as well as meet challenges unique

D. Datla, X. Chen, T. Tsou, S. Raghunandan, S. M. S. Hasan, J. H. Reed, C. B. Dietrich, and T. Bose are with Wireless@Virginia Tech in the Department of Electrical and Computer Engineering at Virginia Polytechnic Institute and State University, Blacksburg, VA, USA. Bruce Fette is a program manager in the Strategic Technology Office of the Defense Advanced Research Projects Agency (DARPA), USA. Jeong-Ho Kim is with the Department of Electronics Engineering, Ewha Womans University, Seoul, Korea. The views, opinions, and/or findings contained in this article/presentation are those of the author/presenter and should not be interpreted as representing the official views or policies, either expressed or implied, of the Defense Advanced Research Projects Agency or the Department of Defense. Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)

to WDC, with the help of recently available enabling technologies. This research requires development and analysis of both underlying system models and algorithms that enable efficient and robust functionality of multi-node wirelessly networked computer systems. a) Current Trends and Driving Factors: Applications hosted by modern radio systems, such as wireless sensor networks (WSNs), tactical radios, and commercial smart phones, require efficient and robust networks that can support resource intensive computation services. In addition, future wireless standards are expected to adopt computationally complex ad-hoc networking approaches. These applications involve complex communication and computation tasks that impose stringent quality of service (QoS) constraints, such as power/energy consumption, latency, and range, as well as stringent resource (power, communication, and computational) requirements. A single radio node may not be able to meet these requirements, mainly due to resource constraints, incomplete data and/or limited functionality, thereby driving multiple resource-constrained radio nodes to form a WDC network and collaborate on executing tasks in a distributed manner. This approach requires a paradigm shift in the way information services are executed in both military and consumer arenas. b) Enabling Technologies: Earlier, wireless devices did not generally have access to unutilized processing power and information on peer nodes. The advent of self-organizing ad-hoc networks has improved interaction and collaboration among nodes. Recently, radio entities, such as video capable smart phones and multi-sensor equipped soldiers, are seen not simply as objects for communication, but as devices able to collect and process information. Processed information can then be shared with other nodes, creating an avenue for new, enhanced capabilities and applications. WDC is possible today due to the availability of key technologies such as fault tolerant computing, distributed computing, software defined radio (SDR), and cognitive radio (CR). These technologies can offer capabilities such as collaboration among wirelessly connected computational devices, flexible link design, and adaptation and autonomous operation in dynamic WDC environments. These capabilities enable flexible and optimized use of resources, reliable wireless communications, joint optimization of computation and communication processes, and customization for the QoS needs of WDC. c) Potential Benefits: Local processing requires a high resource requirement per node and there can be no QoS

2

guarantee from individual fault prone radio nodes. WDC on a collaborative radio network has the potential to enable: (a) lower energy and power consumption per node as well as for the network as a whole; (b) efficient allocation of computational resources to meet requirements by opportunistically sharing the computational workload with collaborating nodes; (c) achievement of computational latency requirements of complex processing tasks by leveraging network computational resources; (d) robust, secure, and fault tolerant execution of information services; (e) simplified small form factor node designs with lower resources per node; and (f) applications that employ location based processing. In WDC, however, there is an overhead imposed by communication between distributed computational processes which reside in different radio nodes. Power analysis has shown that WDC is beneficial over local processing under certain conditions, particularly when the computational cost dominates over the communication overhead [1], [2]. These conditions are realized when complex computational tasks are distributed in short-range networks. The technology can be used to safeguard applications and improve their fault-tolerance. Different parts of the code and data pertaining to an application can reside in different nodes, instead of having them all located in a single node. If processing over several nodes, a significant number of nodes have to fail or to be compromised in order to adversely affect overall operation or to have access to all software, hardware, data, and codes. In a similar manner, highly secure tasks that require coordination among multiple units cannot be executed by a small number of compromised units. In addition, WDC research is expected to complement parallel efforts in related fields such as wireless grid computing [3] and opportunistic computing [4]. d) Novel Research Contributions: Distributed computing has been previously studied in the context of traditional wired networks and WSNs. WDC research poses some unique challenges over traditional DC. Traditionally, each node in the network has been considered to be a powerful machine. Typically, time required for local computation is ignored while analyzing distributed algorithms. In WDC, however, a node’s processing and communication capacity may be limited. In addition, mobility and wireless channel variations introduce uncertainty into the otherwise traditional DC environment, posing a challenge to the design and execution of distributed algorithms and to ensuring their correctness, efficiency, and promptness. For instance, the intermittent connectivity in typical opportunistic networking scenarios [4] makes it hard to guarantee QoS and have stable knowledge of the network state. While uncertainty in channel characteristics and the availability of collaborating nodes (and computational resources) contribute to the negative effects of mobility on WDC performance, mobility can assist with improvement in information diversity, particularly for social networking applications. WDC networks can potentially operate as an overlay on diverse wireless platforms such as WSNs, hand held radios, and laptops. A comprehensive framework for executing distributed applications in a general WDC network is still in the nascent stages. Some of the DC concepts proposed for WSNs [5]–[7] form the basis for WDC research, however,

WDC network of handheld radios

Long backhaul Long backhaul

Master node

Fig. 1.

Remote super-node

WDC network of UAV radio nodes

Example scenario with WDC nodes and remote super-node.

they need to be significantly adapted before being applied to WDC networks. In addition, WDC research does not restrict itself to WSN-like environments which are characterized by static node positions. WDC networks are intended to operate in heterogeneous mobile wireless environments characterized by stochastic channel variations, diverse node capabilities, and limited a priori knowledge of the environment. Issues unique to such environments have not been addressed in a holistic manner in traditional WSN research. WDC network design intends to incorporate cross-layer optimization approaches with joint allocation of computational and communication resources. Some possible WDC applications are discussed in Section II. An example scenario is presented in Section II-A. Sections III and IV provide an overview of WDC design and implementation challenges. Section V concludes this paper. II. WDC A PPLICATION S CENARIOS Several classes of applications can potentially benefit from WDC, some of which are listed as follows: • Real-time data capture, processing and dissemination This class includes time-critical data collection applications, such as portable mobile scientific computing, where the captured data is processed to provide information to a remote node. • Complex communication waveforms - Robust communication signal processing approaches often involve complex computations which can be executed in a distributed manner. • Information sharing - Real-time sharing of video, images, and voice data may require complex information coding and ad-hoc network capabilities. • Robust process control - Control applications require time-critical coordination among wireless sensors and actuators. Example applications are: image processing and pattern recognition; distributed data storage and database search (to avoid communication with a remote server over unsecure and unreliable links the database is distributed locally among the nodes); application to enable the web indexing of user-generated content that is stored at the mobile device and not on a remote server; Synthetic Aperture Radar (SAR) processing; jamming; signal detection and classification; and position location. A. Example Scenario Energy savings achieved by WDC is demonstrated with the help of the example in Figure 1. Consider a collaborative

3

90

Network energy savings (%)

85

80

75

70

WDC network range = 10 m WDC network range = 400 m WDC network range = 700 m WDC network range = 900 m

65 2000

Fig. 2.

2500

3000 3500 4000 4500 Backhaul communication range (meters)

5000

Energy savings for a WDC network with 5 nodes.

96.5

96 Network lifetime improvement (%)

wireless ad-hoc network which may be comprised of tactical handheld radios, radio nodes attached to unmanned aerial vehicles (UAVs), or sensor nodes. The master node in the network captures data (such as images) that holds the information required at a remote base. The data can be processed in one of the following ways: (a) remote data processing, where unprocessed data is transmitted over a long backhaul to a node abundant in resources located in the remote base, or (b) in-network processing, where data is processed within the network before transmitting the processed information to the remote base. The latter option, when performed in a distributed manner, also allows the use of a large number of relatively small, simple, and inexpensive radio nodes in place of a single complex and expensive one. In addition, WDC minimizes the dependency of the handheld radios on remote super nodes for mission-critical computation services. When considering a master-slave network configuration in WDC, the master node broadcasts data to the slave nodes which return processed data to the master node. The master node, then, transmits processed data to the remote base. Processed data accounts for relatively smaller communication payload compared to raw data. The savings in energy consumption of the wireless nodes is achieved in several ways. First, WDC enables in-network processing, which significantly reduces the number of data bits transmitted over the long backhaul at the cost of some computational energy consumption. Second, power-hungry processor cooling mechanisms activate when the processor operates at a certain high clock frequency. In WDC, the required computational latency can be met by concurrently processing data in all the nodes at a fraction of the required rate. This WDC scenario also applies for smart phones in a social setting. An analysis is performed using computational and communication energy consumption models [1], [2] to determine the network energy savings and lifetime improvement achieved by distributed in-network processing in comparison to remote processing. The slave nodes are assumed to communicate with the master node in a time-division multiplexed manner. Processed data is assumed to constitute 10 % of the raw data’s communication overhead. In this analysis, the computational task comprising of 3000 1024-point FFT operations is uniformly distributed among all the nodes in the network including the master node. The energy savings is plotted in Figure 2 for different backhaul and WDC network ranges. It is shown that an energy saving of nearly 85 % can be achieved for a backhaul range of 3.5 km and a network range of 900 m. The savings is greater for shorter network ranges and longer backhaul ranges because of the communication overhead involved in distributing the workload among the radio nodes. The savings curve flattens when the backhaul transmit power consumption dominates the savings. The network lifetime improvement that occurs as a result of energy savings is computed taking into consideration the lifetime of the master node which consumes the maximum energy among all nodes. Figure 3 shows that WDC provides a lifetime improvement of 94% with a network of as few as two nodes. As the network size increases and more tasks are delegated to slave nodes under the uniform workload

95.5

95

94.5

Backhaul range = 2 km Backhaul range = 2.5 km Backhaul range = 3 km Backhaul range = 4 km

94

93.5

Fig. 3.

2

4

6

8

10 12 14 Number of radio nodes

16

18

20

Lifetime improvement for WDC network with a range of 500 m.

allocation scheme, the master node’s computational energy consumption decreases along with an increase in the communication energy consumption, since the master node has to broadcast (gather) more data to (from) the slave nodes. Thus, beyond a certain number of nodes (about 10 nodes in this case), there is no significant improvement in energy savings and network lifetime. III. D ESIGN C HALLENGES A WDC node is modeled in terms of its three subsystems, namely, power, computation and communication subsystems. An important WDC research challenge is to develop performance evaluation approaches and comprehensive models of the dynamic WDC network environment which is characterized by random node failures, varying channel quality, stochastic availability of computational resources, and random delays in message transit times. In addition to inter-node communication overhead time which is considered in DC analysis, WDC analysis must consider energy consumption and the fact that inter-node communications are stochastic. The main design challenge for WDC over traditional wired

4

DC is meeting application QoS requirements in dynamic mobile wireless environments, which are characterized by uncertainty, high outage probabilities (fault occurrences) and biterror-rates, high energy and power consumption in executing a distributed computational task, and random delays due to re-transmissions. The environmental dynamics include spatiotemporal stochastic variations in link quality and resource availability. WDC models, design tradeoffs, fundamental limitations, and scalability in terms of network range, node density, and workload are greatly influenced by the underlying radio environment. Since the tradeoffs involve interaction between the application and underlying communication layers, and influence of the communication and power subsystems on the distributed computing performance, WDC research takes a cross layer and cross subsystem approach to WDC network design. This approach is evident from the design of the communication subsystem (Section III-A) and scheduling algorithms for WDC networks (Section III-C.4). In addition, most distributed algorithms designed for WDC have to take into consideration the aforementioned challenges. With the aim of facilitating distributed computing in wireless environments, WDC research focuses on three core issues, namely, communication subsystem design, synchronization, and WDC control. A. Communication Subsystem Design During the execution of an application’s computational tasks in a distributed manner, the application’s processes which reside in different nodes in the WDC network communicate with each other in order to enable data passing and WDC management and control, including synchronization and scheduling. Due to the critical importance of inter-node communication, communication subsystem robustness is key to the successful execution of a WDC application. In addition, random channel losses and transceiver components, such as channel decoders, can drastically increase WDC communication overhead. When analyzing wireless environments, three broad categories of network models can be identified, namely, infrastructure-based wireless networks (e.g., cellular networks), infrastructure-less wireless networks (e.g., ad hoc networks, vehicular networks, opportunistic networks), and wireless networks with partial infrastructure (e.g., mesh networks, hierarchical WSNs, infostations). Each one of these models presents different characteristics and issues. In other words, distributing the computational tasks in a WLAN where there is a base station that is an intrinsic coordinator for the wireless clients associated to it, is different with respect to the same task performed in a multihop ad hoc network where there are no centralized entities. The communication subsystem is expected to possess three basic functionalities, as discussed next. 1) Scatter and Gather: These operations differ fundamentally between WDC and wired DC systems. Unlike the pointto-point bi-directional nature of wired links, the wireless link is inherently broadcast and interference prone. In order to minimize interference, WDC networks require an efficient MAC during gathering that schedules message transmissions in synchronization with the computational process execution.

2) Routing and Relaying: Multi-hop relaying is beneficial when the communication burden of the WDC nodes can be shared by the relaying nodes. Traditional routing and relaying methods take into consideration the communication cost associated with different links and the availability of communication resources. However, in WDC networks that are comprised of SDRs, where the communication functionality is implemented in software, computational burden and the availability of computational resources have to be taken into consideration [8]. This is particularly true for relaying methods that require significant amount of computational resources in order to execute channel decoding algorithms. Hence, several communication and computational factors influence WDC routing decisions, such as connectivity (relaying between two disconnected processes), interference, joint scheduling of computational and communication processes, channel selection, and allocation of both communication and computational resources. This problem is more challenging in heterogeneous networks where the computational resource availability varies between nodes. 3) Buffering: Limited and variable data rates over wireless channels can pose a bottleneck to applications involving fast computations. In such cases, buffering between communication and computational subsystems can control message losses and delays. However, the limited buffering capacity of portable WDC nodes makes it challenging to address this problem. B. Synchronization Some distributed applications require the computational processes on different nodes in a WDC network to be synchronized with each other [3]. Synchronization is particularly important when the processes have to interact with each other while executing the application. Synchronization involves the establishment of a temporal relationship between these processes [9]. In WDC networks, synchronization is a challenge when the computational processes which have to be synchronized are heterogeneous in terms of their execution times and the execution time is uncertain, particularly when there are other computational processes contending for limited computational resources within each node. 1) Process Synchronization: Different computational events can occur during each round of the execution of a computational process. If event a q,r+1 in round r + 1 of process Q has to occur after event a p,r in process P , then there should be a way for the events to be synchronized. Upon completion of event ap,r , process P sends a message to process Q thereby triggering event a q,r+1 , as shown in Figure 4. The sequence of events in process P has to be executed such that the message sent after completion of event a p,r reaches process Q before event aq,r+1 occurs. This is the fundamental synchronization problem [9] which is prevalent in both traditional DC and WDC. The relative time difference, denoted by τ , between processes P and Q, can be a random variable. Channel uncertainty in WDC networks, particularly due to the presence of unknown interference, poses a challenge to the determination of message transit time bounds that are critical for synchronization. In addition to the synchronization of the individual computational

5

P

Q

Round 1

ap,1

Me ssa ge tim transit e m

Round 2

TP

C4 Round 1

Time Round 2

Space L1 N1 L2 N2 L3

b36

N4

b56 C6

Time

L4 L6

N3 L5 N5 Space

Communication graph

ap,3

Time at node P

Time at node Q

Global time

ap,i = Event representing occurrence of ith round of algorithm in node P c = time difference between clocks (time references) of nodes P and Q Ti = Round time in ith process

Fig. 4.

b25

Task graph

c

b13 C3

C5

b46

ap,2 aq,2

Round 3

C2

b24 aq,1

C1

b12

Space

Fundamental synchronization problem in distributed computing.

and communication processes on all the nodes, it is a challenge to synchronize the two intra-node processes within individual nodes in order to optimize buffering between them. 2) Clock Synchronization: The local clock on each node needs to be synchronized to the network’s global clock for various purposes such as time stamping of events, network re-synchronization, and fault tolerance [9]. The differences between local clocks and unbounded message transit times in WDC networks can pose a problem when clock synchronization is based on message time stamps. In addition, incorrect clock readings provided by erroneous or malicious nodes can affect synchronization. C. WDC Network Control Control of distributed application execution in the WDC network includes enabling cooperation between the processes, segmentation of computational task into sub-tasks, and optimal resource allocation (including scheduling and workload allocation of sub-tasks to various nodes). The different aspects of control go hand-in-hand and involve additional challenges in terms of flexible network protocol design. Control can be either centralized, distributed (possibly concurrent), or a combination of both. In the case of distributed control, the consensus among nodes is made through agreement protocols. The computational burden, communication overhead and reliability of control mechanisms in dynamic environments are of concern in WDC network design. 1) Leader Election: Many DC applications require a leader process that can perform centralized control functions. In a general WDC master-slave cluster, the leader process may reside either in the master node, a slave node or any node outside the cluster. In addition to existing challenges, WDC leader election must consider both computation and communication costs incurred by the potential leader. In distributed control, the control functions can be allocated to multiple leader nodes based on their ability to execute those functions. 2) Topology Control: Clustering and topology construction involve the selection of a subset of collaborative nodes based

Fig. 5. graph.

Scheduling sub-tasks in task graph to nodes in the communication

on link quality and resource availability in order to construct a topology (such as a spanning tree) that optimizes objectives such as power, connectivity, and network reliability. 3) Mutual Exclusion and Resource Access: The mutual exclusion problem involves the “allocation of a single, indivisible non-shareable resource that can support only one user at a time”, among n concurrent entities [10]. This problem is prominent in WDC environments where resources must be shared in an opportunistic manner. In general, a user process requests the resource’s manager process for access to the resource. Interactions between these processes, as well as interactions among various manager processes, are of equal importance in the exclusion problem. Each resource maintains a queue of indices of user processes that require access to that resource [10]. It is important to access the current queue state including resource occupancy and user position in the queue. Resource access in WDC faces two challenges, namely, queue access fairness and stacking of resource queues. Traditionally, a process can instantaneously enter a resource queue. In WDC networks, however, random variation in message transit delays between different links may not allow fair queue access. In addition, a user process would have to contend for intermediate network resources (such as spectrum access and relay node computing resources) before gaining access to the required resource’s queue. Moreover, any interaction between the resource process and the user process has to surpass the spectrum manager queue. 4) Resource Allocation - Scheduling and Workload Allocation: Scheduling in WDC networks mainly involves task segmentation, allocation of sub-tasks to nodes, temporal scheduling of sub-tasks within each node, and scheduling of communication between sub-tasks in different nodes. The application’s computational task is segmented into sub-tasks which are represented in the form of a task graph, as shown in Figure 5. In the task graph, C i represents an abstraction of the i th sub-task that includes resource requirements and computational cost. The interaction between sub-tasks i and j is represented by b ij which denotes the amount of communication and tolerable latency between respective sub-tasks. In traditional homogeneous DC networks with deterministic channels, efficient resource utilization can be achieved by simple uniform workload allocation. However, resource allocation in WDC networks must consider several factors with cross layer and cross subsystem influences in a

6

40 σ

T

i,0

35

σ

T

i,1

Variance of Execution Time (s)

dynamic environment, resulting in the search for an optimum scheduling strategy over a multi-dimensional design space. WDC task allocation can be viewed as a problem of mapping the task graph to the communication graph [11], as shown in Figure 5. The communication graph captures the heterogeneity in the WDC network and represents nodes and links, denoted by N i and Li , which are abstracted by their capacity, resource availability, and link quality. Resource allocation (includes scheduling and workload allocation) in WDC networks can be formulated as an optimization problem with the following factors, which span across the application and communication layers, as well as across the communication, computational and power subsystems: (a) application requirements expressed in terms of time-to-completion, power/energy consumption, reliability, and task precedence constraints; (b) current network state and resource constraints specified by the communication graph, which include channel conditions, current processor load, computing, and communication resource availability; and (c) cost functions that quantify both communication and computation aspects such as throughput, fairness, reliability, concurrency, and overhead (energy/power, communication, and time). Power and energy optimality are clearly distinguished in WDC design [1], [2]. Task allocation may also consider synchronization, fault tolerance, and the security of individual nodes. The presence of stochastic environmentally-harvested power sources, such as solar power cells, adds an additional challenge to the already complex resource allocation problem. Some WDC resource allocation problems are pointed out. Processing rate can be assigned based on task concurrency and execution time of sub-tasks in each node, in order to minimize time-to-completion or power consumption. Workload allocation can minimize energy consumption by assigning message traffic on a link based on its channel quality. Adaptive resource allocation to achieve equal energy consumption across all nodes, particularly when channel conditions vary between links and processor loads differ between nodes, can ensure graceful degradation of battery life on all network nodes which maximizes network lifetime. Scheduling should also take into consideration the dynamics of network connectivity and resource availability, and the resulting time-variant node availability. If a participating node gets disconnected from the cluster while the application execution is in progress after workload allocation, there are two options: (a) the node’s completed sub-task results can be relayed back to the cluster, or (b) service-handoff can occur from the disconnected node to appropriate nodes within the cluster. In a similar manner, the WDC control mechanisms have to take into consideration the addition of computational resources by nodes that are newly admitted into the WDC network. Robustness (in terms of application execution time variance) of different workload allocation schemes, such as uniform allocation, allocation considering mean channel condition, and allocation considering both channel mean and variance, has been analyzed for WDC networks [12] as shown in Figure 6. It is observed that the latter scheme can reduce execution time variance, improve robustness dramatically [12], and improve

σ

30

T

i,2

25 20 15 10 5 0 20

40

60

80

100 120 Number of Nodes

140

160

180

200

Fig. 6. WDC network computational robustness for various workload allocation schemes [12]. σ0 : even allocation, σ1 : allocation considering mean channel condition, and σ2 : allocation considering both channel mean and variance.

power availability for computing purposes by 14-22% [13]. The scheme simultaneously minimizes communication power consumption and maximizes computing rate. In this manner, nodes located in favorable channel conditions (smaller distances and channel variance) are allocated a higher computing rate. Due to the smaller communications cost in these nodes, the ratio between computing and total power consumption can be improved for the WDC network. 5) Network Information Management: An important research challenge is to determine the amount of information that each node should have about the WDC environment in order to perform control tasks. Network information is used by the leader node to perform critical operations such as stability analysis, failure recovery, and analysis of algorithm correctness [9]. In a distributed approach to network information gathering, the dynamic global WDC network state (which includes that of all computation processes, node resources, and links) can be sampled and quantized in a distributed manner before it is disseminated to leader node(s). IV. I MPLEMENTATION I SSUES Preliminary theoretical models have been developed for WDC and show potential advantages of the approach. However, realization and adoption of these capabilities on actual hardware and in operational systems involves overcoming significant obstacles. A. Software Architecture To realize WDC benefits, an underlying architecture [14] is necessary to support both SDR operations for physical layer connectivity and more general processing for collaborative computing tasks. Existing software architectures typically assume either role, but not both. Physical layer activities may be characterized by strict requirements on timing or bandwidth. On the other hand, DC tasks can vary from long-running background activities to real-time applications that may resemble

7

physical layer communications. In a SDR implementation, this spectrum of activities must be managed on a single platform with one or more shared processors, which creates conflicting interests between simultaneously operating computing tasks. Such conflicts over shared resources have not been accounted for in traditional physical layer models which assume a dedicated implementation composed of discrete processing components. In addition, opportunistic computing in SDR based WDC is premised on the availability of underutilized processing capability. In practice, processor overhead or memory barriers may affect the ability to remotely target spare processor cycles over a wireless channel in an efficient manner. Furthermore, if remote processing is readily accessible, distributed computing tasks must not preempt primary physical layer operations. In other words, software implementations of various components must provide the behavior of a discrete design, although the underlying structure may be composed of shared and limited processing resources. B. Power Consumption on Real Devices Real–world energy utilization may include effects not included in theoretical network power/energy consumption models. These models are understandably generic due to the multitude of differences that exist between different platform implementations. However, to achieve the desired management for prolonged functionality, whether at the individual node or network level, effects of implementation related critical factors on system design must be taken into account. For tractability, theoretical computational power models often focus on processor performance characteristics. For example, fundamental physical relationships between processor voltage and switching frequency are easily abstracted for mathematical analysis [2]. However, physical properties on modern processors are not subject to direct manipulation by software applications. Furthermore, basic physical processor parameters do not account for all the computational energy draw in a node. Modern processor management schemes include voltage scaling, frequency scaling, and, more recently, power gating at the processor level. These parameters are not directly manipulable but are accessed through processor interfaces which provide selectable discrete power states. In addition, power control mechanisms at a higher operating system level allow for larger scale manipulation of overall platform properties. For example, sleep and hibernation states may power down individual portions of the platform or the entire platform. Thus, static loads can be manipulated and enabled or shut down completely. Consequently, computational power management in real, modern processors is more appropriately viewed as state based rather than a combination of continuously variable parameters, which leads to a divergence from the approach of many theoretical models. These features of power management, while difficult to abstract, are critical in computational energy determination on real systems. Application of these features to existing models may be feasible, depending on specific application

requirements and this will need to be resolved for actual implementations. C. Synchronization, Determinism, and Power Constraints WDC implementations must address synchronization issues often implicitly assumed to be solved in theoretical models. The introduction of general purpose processors and software based communication stacks further complicates these issues as they add non-deterministic properties through contention over consolidated processor resources. A challenge in embedded real-time computing systems is the determinism required for synchronization and efficient usage of power management schemes are intertwined on multiple levels with numerous contradicting characteristics. For example, the most power efficient processor states generally have the longest wakeup and response times. In addition, within the operating system of a single node, the most deterministic behavior typically requires a higher degree of headroom, or unutilized excess processor capacity, in order not to jeopardize timing deadlines. V. C ONCLUSION WDC enables powerful computing and extended service execution capabilities with the help of a collaborative network of small form factor radio nodes that operate with reasonably low-capacity batteries. We have looked at some ways in which traditional DC approaches may fail or not operate optimally in dynamic wireless environments. In this regard, the paper has introduced the potential challenges of WDC and hopes to provide the motivation to bring about design changes that can address the challenges. On the implementation front, considerable challenges remain in order for practical WDC networks to become a reality. SDR-based WDC implementations need to be developed with an expanded view of software architecture that takes into account characteristics of both physical layer communications and network computing. ACKNOWLEDGMENT The authors would like to thank the Office of Naval Research for its sponsorship (grant N30001407010536) of our research; the Korean National Research Foundation (NRF) for its sponsorship of Dr. Jeong-Ho Kim (this research was supported by Basic Science Research Program through the NRF of Korea funded by the Ministry of Education, Science and Technology, grant number : 2010-0008916); Dr. Maleq Khan (Network Dynamics and Simulation Science Laboratory, Virginia Bioinformatics Institute, Virginia Tech) for his valuable comments; and Cyndy Graham (Wireless@Virginia Tech) for proof-reading this article. R EFERENCES [1] D. Datla, S. M. Hasan, J. H. Reed, and T. Bose, “Fundamental Issues of Wireless Distributed Computing in SDR Networks,” in SDR’10 Technical Conference and Product Exposition, (Washington, DC, USA), November 2010. [2] D. Datla, X. Chen, T. R. Newman, J. H. Reed, and T. Bose, “Power Efficiency in Wireless Network Distributed Computing,” in IEEE Vehicular Technology Conference, (Anchorage, AK, USA), September 2009.

8

[3] X. Chen, S. M. Hasan, T. Bose, and J. Reed, “Software Defined Radio Based Wireless Grids,” in SDR’10 Technical Conference and Product Exposition, (Washington, DC, USA), November 2010. [4] M. Conti and M. Kumar, “Opportunities in Opportunistic Computing,” IEEE Computer, vol. 43, pp. 42 – 50, January 2010. [5] S. Abdelhak, R. S. Chaudhuri, C. S. Gurram, S. Ghosh, and M. Bayoumi, “Energy-Aware Distributed QR Decomposition on Wireless Sensor Nodes,” The Computer Journal, 2010. [6] Y. Yu and V. K. Prasanna, “Energy-Balanced Task Allocation for Collaborative Processing in Wireless Sensor Networks,” Journal Mobile Networks and Applications, vol. 10, February 2005. [7] A. B. Olsen, F. H. P. Fitzek, and P. Koch, “Evaluation of Cooperative Task Computing for Energy Aware Wireless Networks,” in Proceedings of International Workshop on Wireless Ad-Hoc Networking (IWWAN), 2005. [8] D. Datla, T. Tsou, T. R. Newman, J. H. Reed, and T. Bose, “Waveform Level Computational Energy Management in Software Defined radios,” in SDR’09 Technical Conference and Product Exposition, (Washington, DC, USA), December 2009. [9] G. Tel, Topics in Distributed Algorithms. Cambridge International Series On Parallel Computation; Vol. 1, 1991. [10] N. A. Lynch, Distributed Algorithms. The Morgan Kaufmann Series in Data Management Systems, first ed., March 1996. [11] D. P. Vidyarthi, B. K. Sarker, A. K. Tripathi, and L. T. Yang, Scheduling in Distributed Computing Systems: Analysis, Design and Models. Springer, 2008. [12] X. Chen, T. R. Newman, D. Datla, T. Bose, and J. H. Reed, “The Impact of Channel Variations on Wireless Distributed Computing Networks,” in IEEE Globecom, (Honolulu, HI, USA), Dec. 2009. [13] X. Chen, S. M. Hasan, T. Bose, and J. H. Reed, “Cross-layer Resource Allocation for Wireless Distributed Computing Networks,” in IEEE Radio and Wireless Symposium, (New Orleans, LA, USA), Jan. 2010. [14] T. Tsou and J. H. Reed, “Software Architecture for Cooperative Applications,” in SDR’09 Technical Conference and Product Exposition, (Washington, DC, USA), December 2009.

Dinesh Datla ([email protected]) is currently pursuing his Ph.D. degree in the Bradley Department of Electrical and Computer Engineering at Virginia Tech (Blacksburg, VA, USA). He received his M.S. in Electrical Engineering from the University of Kansas (Lawrence, KS, USA) in 2007 and B.E. in Electronics and Communications from the University of Madras (Chennai, India) in 2004. His current research interests include wireless distributed computing in software defined radio networks, green communications, cognitive radios and dynamic spectrum access radios.

Xuetao Chen ([email protected]) received his B.S. and M.S. in electrical engineering in 2001 and 2004 respectively from Wuhan University, China. Currently, he is pursuing a Ph.D. degree of electrical engineering at Virginia Tech, Blacksburg. His research interests include Cognitive Radio, wireless distributed computing networks, high efficiency power amplifier design.

S. M. Hasan ([email protected]) is a research scientist at Virginia Tech. He received his Ph.D. degree from Virginia Tech, M.S. degree from the University of Tennessee, Knoxville, and his B.S. degree from Bangladesh University of Engineering and Technology, all in electrical engineering. His research interest is in the RF front end and system design, SDR, and cognitive radio areas. He is also an expert in the field of instrumentation and measurement, and applied digital signal processing.

Jeffrey H. Reed ([email protected]) is the Willis G. Worcester Professor in the Bradley Department of Electrical and Computer Engineering. From June 200 to June 2002, Dr. Reed served as the Director of the Mobile and Portable Radio Research Group (MPRG). He currently serves as Director of the newly formed umbrella wireless organization Wireless@ Virgina Tech, one of the largest and most comprehensive university wireless research groups in the US. Dr. Reed received his B.S. M.S., and Ph.D from the University of California, Davis in 79-87. Dr. Reed has provided significant contributions in the area of software radio and communications signal processing. Specific contributions include the development of hardware architectures for software radios based on reconfigurable computing, prototype hardware development, assistance in standardizing software architectures, development of software radio tools in education and research that are available for public domain download and creation of analysis procedures for cognitive software radios. His textbook, Software Radios: A Modern Approach to Radio Design, is based on his research experience with ONR, DARPA, IT, and Samsung, and is one of the first books devoted exclusively to software radios. Dr. Reed has served on several company advisory boards. He is co-founder of CRT. In 2005, Dr Reed became Fellow to the IEEE for contributions to software radio and communications signal processing and for leadership in engineering education.

Bruce Fette ([email protected]) is a Program Manager in the STO office at DARPA for programs involving Wireless Networking technologies as part of the DARPA’s strategic investment in advanced networking communications systems for the warfighter. His goal is to advance RF communications technology with programs such as Advanced Wireless Network Systems, Wireless Network After Next and Communications under Extreme Spectrum Conditions. These programs are part of the broader context of activities in support of the soldier communications and situation awareness networks. Dr. Fette has extensive experience in industry developing products and business synthesis in LSI design, advanced signal processing including speech and radio waveforms, communication security, and wireless communications. Dr. Fette was previously with Motorola and then General Dynamics as Chief Scientist General Dynamics C4 Systems in Scottsdale. Dr. Fette received his MSEE and PhD EE from Arizona State University, and BSEE from University of Cincinnati.

Thomas Y. Tsou ([email protected]) received his B.S. degree in computer engineering from the University of Illinois at Urbana-Champaign in 2002. He is currently pursuing a Ph.D. in electrical engineering at Virginia Tech and involved in open source software-defined radio platforms for cellular networks. His research interests include wireless standards, reconfigurable devices, and position location.

Sahana Raghunandan ([email protected]) is currently pursuing her PhD degree in the Bradley Department of Electrical and Computer Engineering at Virginia Tech. Her research interests include software defined radios, resource allocation in wireless networks, and hardware and software design of radar systems. Earlier she was a graduate student in electrical engineering at The University of Kansas, Lawrence.

Carl Dietrich ([email protected]) is a research associate professor at Virginia Tech, where he completed Ph.D. and M.S. degrees after graduating from Texas A&M University. He worked with the Defense Information Systems Agency, Arlington, Virginia and Bell Northern Research, Richardson, Texas and conducted research on antenna systems and radio wave propagation. He leads the OSSIE open source software defined radio (SDR) effort, chairs the Wireless Innovation Forum Educational SIG, and is a professional engineer in Virginia.

9

JeongHo Kim ([email protected]) received the B.S, M.S. and Ph.D. degrees in electrical engineering from the Korea Advanced Institute of Science and Technology (KAIST), in 1991, 1993, and 1999, respectively. From Feb. 1993 to May 1995, he was a research engineer with LG Electronics Korea. From February 1999 to June 2002, he was a senior research engineer at UMTS System Laboratory, LG Electronics, Korea where he had been engaged in various projects including the development of the 3G Base Station Modem and Test of W-CDMA Pre-commercial Systems and Integration of W-CDMA subsystem with Core Networks. He has been with the faculty of Ewha Womans University now as an associate professor since 2002. His research interests include Cognitive radio HW platform, Cognitive networking, and QoS control strategy for convergent cognitive networks.

Tamal Bose ([email protected]) received the Ph.D. degree in electrical engineering from Southern Illinois University in 1988. After a faculty position at the University of Colorado, he joined Utah State University in 2000, where he served as the Department Head and Professor of Electrical and Computer Engineering from 2003-2007. Currently, he is Professor in the Bradley Department of Electrical and Computer Engineering at Virginia Tech. He is the Associate Director of Wireless@VT and Director of the NSF center site WICAT@VT. The research interests of Dr. Bose include signal classification for cognitive radios, channel equalization, adaptive filtering algorithms, and nonlinear effects in digital filters. He is author of the text Digital Signal and Image Processing, John Wiley, 2004. He is also the author or co-author of over 120 technical papers. Dr. Bose served as the Associate Editor for the IEEE Transactions on Signal Processing from 1992 to 1996. He is currently on the editorial board of the IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences (Japan) and Research Letters in Signal Processing. He also served on the organizing committees of several international conferences and workshops. He is an IEEE EAC program evaluator and a member of the DSP Technical Committee for the IEEE Circuits and Systems society.