to maximize the spectrum efficiency of the CR operation. We show how optimal sensing/transmission scheduling policies can be obtained and prove some ...
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 58, NO. 1, JANUARY 2010
235
Adaptive Joint Scheduling of Spectrum Sensing and Data Transmission in Cognitive Radio Networks Anh Tuan Hoang, Member, IEEE, Ying-Chang Liang, Senior Member, IEEE, and Yonghong Zeng, Senior Member, IEEE
Abstract—We consider a cognitive radio (CR) network that makes opportunistic use of a set of channels licensed to a primary network. During operation, the CR network is required to carry out spectrum sensing to detect active primary users, thereby avoiding interfering with them. However, spectrum sensing may cause negative effect on the performance of the CR network, as all CR communications has to be postponed during channel sensing. This paper focuses on adaptively scheduling spectrum sensing and data transmission so that negative impacts to the performance of the CR network are minimized. We first consider the case when CR nodes always have data to transmit and experience time-varying channels. Based on knowledge of channel conditions, the sensing periods are adaptively scheduled to maximize the spectrum efficiency of the CR operation. We show how optimal sensing/transmission scheduling policies can be obtained and prove some important structural properties of such optimal policies. We then consider the case when CR nodes experience both stochastic data arrival and time-varying channels. By treating each sensing period as a ‘virtual sensing packet’, we convert the problem of joint spectrum-sensing/datatransmission scheduling into a standard queueing model. Based on that, an efficient scheduling algorithm that takes into account channel and queue conditions of the CR network is proposed. Index Terms—Adaptive joint scheduling, spectrum sensing, data transmission, cognitive radio networks.
I. I NTRODUCTION
T
HE traditional approach of fixed spectrum allocation to licensed networks leads to spectrum under-utilization. In recent studies by the FCC, it is reported that there are vast temporal and spatial variations in the usage of allocated spectrum [1]. This motivates the concepts of opportunistic spectrum access that allows cognitive radio (CR) networks to opportunistically exploit under-utilized spectrum. On the one hand, opportunistic spectrum access can improve the overall spectrum utilization. On the other hand, transmission from CR networks can cause harmful interference to primary users of the spectrum. To mitigate such a problem, CR networks can frequently carry out spectrum sensing to detect active primary users. Upon detecting an active cochannel primary user, a CR network can either change its operating parameters, e.g., reduce its transmit powers, or move to another channel to avoid interfering with the primary user. Paper approved by Y. Fang, the Editor for Wireless Networks of the IEEE Communications Society. Manuscript received March 17, 2008; revised November 11, 2008 and February 28, 2009. The authors are with the Department of Modulation and Coding, Institute for Infocomm Research (I2R), A-Star, Singapore 119613 (e-mail: {ycliang, yhzeng}@i2r.a-star.edu.sg). Digital Object Identifier 10.1109/TCOMM.2010.01.070270
In most cases, to achieve reliable spectrum sensing for a particular channel, a CR network has to postpone all of its transmissions on that channel, i.e., quiet sensing periods must be scheduled. Note that scheduling quiet sensing periods results in negative impacts to various performance metrics of CR networks, such as throughput and latency. One approach to reduce these negative impacts is to design efficient spectrum sensing algorithms that require short sensing time [2]–[6]. However, assuming that we are given a particular spectrum sensing algorithm with a required quiet sensing time, then the impacts of spectrum sensing depend on the state of a CR network when sensing is scheduled. For example, assuming a CR network in which transmission rate is varied based on the channel condition, then scheduling a quiet sensing period when the channel is good causes more throughput reduction than doing so when the channel is bad. Similarly, assuming a system with stochastic data packet arrival and deadline constraint, then scheduling a quiet period when there are many near-deadline packets causes more deadline violations than doing so when there are few packets queueing. This motivates us to study the problem of adaptive joint scheduling of spectrum sensing and data transmission to optimize the performance of CR networks. We note that most of the existing work on scheduling spectrum sensing in CR networks adopts a periodic control approach, i.e., sensing activities are carried out in fixed interval and are not adaptive to the channel or traffic load of CR networks [7]–[9]. The frameworks proposed in [10]–[12] do adapt the sensing activities, but in different contexts. In [10], Datla et al. propose to allocate different sensing durations to different portions of the spectrum so that different resolution can be achieved. In [11] and [12], sensing time and accuracy are adapted based on observation about the state of primary networks. We first consider a CR network in which nodes always have data to transmit and experience time-varying channels. Data can be transmitted at different rates for different channel conditions. Assuming that instantaneous channel conditions are available for making control decisions, we adaptively schedule quiet sensing periods so that the throughput of the CR network is maximized. This is based on the observation that transmitting data when the channels are good and carrying out spectrum sensing when the channels are in poorer conditions would increase the spectrum efficiency of CR operation. We term this approach CSI-based sensing/transmission, where CSI stands for channel state information. For a finite state Markov
c 2010 IEEE 0090-6778/10$25.00 ⃝
236
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 58, NO. 1, JANUARY 2010
channel (FSMC) model [13], we formulate the optimization problem as a finite-horizon Markov Decision Process and obtain optimal CSI-based sensing/transmission policies using dynamic programming. We also prove some important structural properties of the optimal scheduling policies. Based on that, we propose a suboptimal CSI-based sensing/transmission policy that performs close to optimal. Next, we consider a CR network in which users experience both stochastic data arrival and time-varying channels. Data packets of CR nodes have deadline constraints and the objective is to jointly schedule spectrum sensing and data transmission activities of the CR network so that the packet loss due to deadline violation is minimized. Here, joint sensing/transmission scheduling decisions are based upon both channel and queue conditions of the CR nodes. To tackle this problem, we introduce a novel idea of treating each quiet sensing period as a ‘virtual sensing packet’. Then, carrying out spectrum sensing on a particular channel is equivalent to transmitting a virtual sensing (VS) packet on that channel. This simple but useful abstraction enables us to transform the problem of joint sensing/transmission scheduling into a parallel server allocation problem. Based on this, we propose an efficient scheduling algorithm that takes into account the channel and traffic conditions of the CR network to achieve good quality of service. We term this approach QCIbased sensing/transmission, where QCI stands for queueing and channel information. Simulation results show QCI-based scheduling achieves significant reduction in packet loss due to deadline violation for the CR network. It should be noted that spectrum sensing is not merely a task, i.e., depending on the outcome of spectrum sensing, CR networks will have to take appropriate responses. In most cases, the responses of the CR network to sensing outcomes are strictly specified by either regulatory rules or agreements between operators of CR and primary networks. In this paper, we assume that when primary users are detected, the CR network must vacate the busy channel and move to a new one. Then, the same sensing scheduling operation can be applied to the new channel. Therefore, our optimization problem can be viewed as ‘optimizing the sensing schedule for periods when a channel is available’. The rest of this paper is organized as follows. In Section II, we discuss spectrum sensing operation in CR networks and describe our general system model. In Section III, the CSI-based sensing/transmission approach is studied. We then consider the case when CR nodes experience both timevarying channels and stochastic data arrival in Section V. The QCI-based sensing/transmission approach is subsequently presented. Numerical results and discussion are presented in Section VI. Finally, we conclude the paper in Section VII II. S YSTEM D ESCRIPTION A. Notational Convention Throughout this paper, we adopt the following notational convention: ∙
Scalars are denoted by upper-case or lower-case letters, for example 𝑁 and 𝑛.
∙
∙
∙
∙
Vectors and matrices are respectively denoted by lowercase boldface and upper-case boldface letters, with the exception that vectors of random variables are denoted by upper-case boldface letters. Let 𝒙 be a one dimensional vector and 𝑿 be a two dimensional matrix, then the 𝑖th element of 𝒙 and (𝑖, 𝑗)th element of 𝑿 are denoted by [𝒙]𝑖 and [𝑿]𝑖,𝑗 , respectively. Let 𝒙, 𝒚 be two row vectors and 𝑎 be a scalar, then [𝒙, 𝒚] denotes the concatenation of two vectors 𝒙 and 𝒚, while [𝑎, 𝒙] denotes the concatenation of scalar 𝑎 to the front of vector 𝒙. E𝑋 {𝑓 (𝑋)} denotes the expectation of function 𝑓 (𝑋) over the distribution of random variable 𝑋.
B. Spectrum Sensing in CR Networks Spectrum sensing is crucial for CR networks to detect active primary users and avoid causing interference. Let us briefly go through important parameters that characterize spectrum sensing in CR networks. 1) Signal to Noise Ratio (SNR): When a primary user is active, the higher the SNR of the primary user’s signal at the receiver of a CR device, the easier it is to detect. We denote this SNR by 𝛾. 2) Probability of Detection 𝑃𝑑 : This is the probability that a CR network accurately detects the presence of an active primary user. The higher the value of 𝑃𝑑 , the better the protection for primary operation. 3) Probability of False Alarm 𝑃𝑓 : This is the probability that a CR network falsely detects the presence of primary users when in fact none of them are active at the sensing time. From the CR network point of view, the lower the value of 𝑃𝑓 , the higher the spectrum utilization. 4) Sensing Time 𝑇𝑠 : This is the time that a CR network needs to postpone all communications to sense a channel. In general, the longer the value of 𝑇𝑠 , the more accurate the sensing outcome. As an example, assume that: i) the primary signal is independent and identically distributed (i.i.d), complex PSK modulated, with zero mean; ii) the noise at a CR receiver is circular symmetric complex Gaussian with zero mean; and iii) the primary signal and noise are independent. Using energy detection, the minimum sensing time to achieve probabilities of detection and false alarm of 𝑃𝑑 and 𝑃𝑓 , respectively, is approximately [14]: )2 √ 𝜏 ( (1) 𝑇𝑠 = 2 𝑄−1 (𝑃𝑓 ) − 𝑄−1 (𝑃𝑑 ) 2𝛾 + 1 , 𝛾 where 𝜏 is the channel sampling interval and 𝑄(.) is the complementary distribution function of a standard Gaussian variable. 5) Detection Time 𝑇𝑑 : This is the time taken to detect a primary user since it first turns on. C. General System Model We consider a CR deployment on licensed band, as depicted in Fig. 1. Each CR network consists of a set of nodes that are supported by a base station (BS). Each CR network operates based on making opportunistic use of the channels
HOANG et al.: ADAPTIVE JOINT SCHEDULING OF SPECTRUM SENSING AND DATA TRANSMISSION IN COGNITIVE RADIO NETWORKS
Secondary BS
Primary BS
CH1
Comms.
Sensing
Comms.
Comms.
237
Sensing Comms.
Cognitive Radio Network 1
CH2
CH3
Comms.
Comms.
Sensing
Sensing
Comms.
Comms.
Comms.
Comms.
Sensing
Sensing
Comms.
Comms.
Secondary user
Frame n
Primary user Secondary BS
Frame n+1
Time
Fig. 2. An example of a CR network operating on multiple channels. Time is divided into frames of fixed length. During each frame, a fixed amount of time must be schedule for quiet spectrum sensing on each channel.
Primary Network Secondary user Cognitive Radio Network 2
Fig. 1.
General system model.
that belong to a primary network. This is one of the CR network architectures similar to the future deployment of IEEE 802.22 technologies [15]. We focus on the operation of one of the CR networks in Fig. 1. The CR network consists of 𝑁 secondary nodes and a BS that carry out opportunistic transmissions on a set of 𝐾 channels licensed to the primary network. The 𝐾 channels are assumed to be orthogonal to each other. Our formulation applied to both uplink and downlink scenarios. The operation of the CR network is depicted in Fig. 2. Time is divided into frames of duration 𝑇𝑓 seconds. During each frame, 𝑇𝑠 seconds must be scheduled for spectrum sensing on each of the 𝐾 channels. We further assume that when primary operation is active, the primary signal strength at CR receivers could be significantly lower than the noise floor. An example of such scenario is the future deployment of 802.22 CR networks operating on the TV bands, where the TV signal strength at 802.22 receivers can be as low as −116 dBm, i.e., around 20 dB below the noise floor, and must be detected with minimum probability of detection of 90% and maximum probability of false alarm of 10% (see [15], Section 15.1.1.7 and Table 15.1.2). Because the primary signal strength at CR receivers is much lower than the noise floor, interference caused by primary transmission to CR operation is negligible. Also, for reliable sensing, all CR devices must stay quiet, otherwise, self-interference from CR transmission would greatly increase false alarm probability. The requirement of quiet sensing means centralized control is assumed for our CR network. The primary network is protected from interference based on the following premises: ∙
∙
∙
Frequent Sensing: Spectrum sensing is carried out frequently, with the interval between any two consecutive sensings not longer than 𝑇𝑑 . This condition translates into 2𝑇𝑓 ≤ 𝑇𝑑 . Reliable Sensing: Given the sensing SNR 𝛾, the sensing time 𝑇𝑠 is calculated based on (1) so that the target detection probability 𝑃𝑑 and false alarm probability 𝑃𝑓 are met. Quick Vacation: Upon detecting active primary users on a particular channel, the CR network will immediately vacate the channel.
We note that the above mechanisms to protect primary network operation is similar to what have been specified for the currently being developed 802.22 standard [15]. The parts of each frame that are not scheduled for channel sensing can be used for CR communications. We assume that, at each time instance, each channel can only be used by at most one CR node of the network. Note that the same channel can be reused in other CR networks. For CR operation, the set of available channels will change if either of the following two events occur: ∙ E1: A primary user starts operation on a particular channel and is detected by the CR network. ∙ E2: No new primary user starts operation, however, due to false alarm in spectrum sensing, the CR network mistakenly concludes that some channel is not available anymore. We assume that the average durations of idle/active periods of primary operation are much longer than the time-scale at which data transmission is carried out, and CR spectrum sensing is highly accurate, so that CR users observe a quasi-static set of 𝐾 available channels. This means the availability of 𝐾 channels changes much slower than the sensing/transmission activities of the CR users so that the time-average performance metrics in Sections III and V are applicable. This is not to be mistaken as the fading process is quasi-static; the dynamics of the fading process. As an example, 802.22 [15] standard will allow CR networks to operate on unused VHF/UHF TV channels, which rarely change the availability status. When event E1 or E2 happens, the CR network will vacate the channel and move to a new one and our control processes can be directly applied on the new channel. III. CSI- BASED S ENSING /T RANSMISSION A PPROACH In this section, we consider the case when CR nodes always have data to transmit and experience time-varying channels. Based on knowledge of channel conditions, we adaptively schedule the spectrum sensing and data transmission activities in the CR network. We term this approach CSI-based sensing/transmission. A. Problem Formulation We assume that each frame of duration 𝑇𝑓 is further divided into 𝐿 slots of size 𝑇𝑐 = 𝑇𝑓 /𝐿. Furthermore, during each frame, one slot must be scheduled for quiet spectrum sensing, i.e., 𝑇𝑠 = 𝑇𝑐 .
238
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 58, NO. 1, JANUARY 2010
1) Channel Model: For each pair of user 𝑛 and channel 𝑘, 1 ≤ 𝑛 ≤ 𝑁, 1 ≤ 𝑘 ≤ 𝐾, the channel model is represented by a finite state Markov chain (FSMC) [13]. In particular, the channel can take 𝑀 possible states and when the channel is in state 𝑚 (1 ≤ 𝑚 ≤ 𝑀 ), user 𝑛 can transmit at rate 𝑟𝑛 (𝑚). It is assumed that the channel state stays unchanged during each time slot and the probability of transitioning from state 𝑚 into state 𝑚′ after each time slot is given by 𝑝𝑛𝑀 (𝑚, 𝑚′ ). During time slot 𝑖 (1 ≤ 𝑖 ≤ 𝐿) of a particular frame, let 𝑮𝑘 (𝑖) be the 1 × 𝑁 vector that denotes the channel states of 𝑁 CR nodes on channel 𝑘. In particular, [𝑮𝑘 (𝑖)]𝑛 is the channel state of CR node 𝑛 on channel 𝑘 during time slot 𝑖. Let Γ(𝑖) be the 𝐾 × 𝑁 matrix where [Γ(𝑖)]𝑘,𝑛 = [𝑮𝑘 (𝑖)]𝑛 . In this paper, we assume that the channels of 𝑁 users vary independently, nevertheless, this assumption can be relaxed, i.e., as long as the joint channel transitioning probabilities of 𝑁 users are available, a similar optimization framework can be carried out. Let 𝑝𝐺 (𝒈, 𝒈 ′ ) be the probability of transitioning from a vector of channel states 𝒈 into a vector of channel states 𝒈 ′ after each time slot, then
K channels must now be carried out jointly. Then, as shown in [16], optimal allocating multiple channels to multiple users can be formulated and solved as a maximum weighted bipartite matching problem.
𝑝𝐺 (𝒈, 𝒈 ′ ) = Pr{𝑮𝑘 (𝑖 + 1) = 𝒈 ′ ∣ 𝑮𝑘 (𝑖) = 𝒈} =
𝑁 ∏
𝑝𝑛𝑀 ([𝒈]𝑛 , [𝒈 ′ ]𝑛 ).
(2)
𝑛=1
2) Control and Objective: We define the following CSIbased sensing/transmission problem. Definition 1: CSI-based Sensing/Transmission Problem For each slot 𝑖 within a frame, given that the channel state information Γ(𝑖) is available, on each channel 𝑘, make a decision on whether to carry out sensing or to allow a user to carry out data transmission so that the achievable throughput is maximized, subject to the constraint that, for each channel, one slot within the frame must be scheduled for quiet sensing. A scheduling policy that maximizes the achievable throughput subject to the sensing constraint is termed an ‘optimal scheduling policy’. For the above problem, the following observations are true for an optimal scheduling policy. First, if sensing is not carried out on a channel, then the channel should be assigned to a CR node that can transmit at the highest rate. This is clearly necessary in order to maximize the achievable throughput. Second, the problem can be decoupled into 𝐾 control problems, one for each channel, and for channel 𝑘, the sensing scheduling decision for slot 𝑖 is made based on the vector of channel states 𝑮𝑘 (𝑖). This decoupling approach is possible due to the fact that 𝐾 channels are orthogonal to each other and each CR user can transmit on multiple channels in a single time slot. We noted that the two conditions observed above are not sufficient for a scheduling policy to be optimal. In fact, there are many scheduling policies satisfying these conditions and an optimal policy can be obtained via the approach described next. The assumption that each CR user can transmit on multiple channels in a single time slots simplifies our channel allocation task. If we are to enforce the condition that each user can only transmit on one channel during a given time slot, the problem would become much more complicated, i.e., the control of all
B. Optimal CSI-based Sensing/Transmission Policies The CSI-based sensing/transmission problem for channel 𝑘 can be formulated as a finite-horizon Markov decision process (FH-MDP) [17]. In particular, each FH-MDP models a sequential decision-making problem over a finite time period. At each specified point within this time period, a decision maker observes the state of a system and chooses an action which produces two results: an immediate reward (or an immediate cost), and a new state at a subsequent discrete point in time. Furthermore, the probability of transitioning into a new state only depends on the current state and the chosen action. Our CSI-based sensing/transmission problem can be modeled by a FH-MDP with the following components. 1) Control Horizon: The control horizon is 𝐿 slots and decisions are made in a slot-by-slot basis. 2) System States: For each frame, let 𝐹 𝑘 (𝑖) be the sensing flag for slot 𝑖, where 𝐹 𝑘 (𝑖) = 1 indicates that sensing has been carried out prior to slot 𝑖 and 𝐹 𝑘 (𝑖) = 0 indicates otherwise. Obviously, for the first slot of each frame, we should set 𝐹 𝑘 (1) = 0. The system state of our control problem is a 𝑘 1 × (𝑁 + 1) vector formed by concatenating [ 𝑘𝐹 (𝑖) 𝑘and]the 𝑘 𝑘 state vector 𝑮 (𝑖), i.e., 𝑺 (𝑖) ]= 𝐹 (𝑖), 𝑮 (𝑖) = [channel 𝐹 𝑘 (𝑖), [𝑮𝑘 (𝑖)]1 , [𝑮𝑘 (𝑖)]2 , . . . [𝑮𝑘 (𝑖)]𝑁 . 3) Actions and Rewards: Let 𝐴𝑘 (𝑖) denote the control action taken for slot 𝑖, where 𝐴𝑘 (𝑖) = 1 means allowing the user with the best channel condition to carry out transmission on channel 𝑘 while 𝐴𝑘 (𝑖) = 0 indicates slot 𝑖 is scheduled for spectrum sensing. Corresponding to system state 𝑺 𝑘 (𝑖) and action 𝐴𝑘 (𝑖), we have the following immediate reward function which specifies the throughput gain: { max𝑛 {𝑟𝑛 ([𝑮𝑘 (𝑖)]𝑛 )}, if 𝐴𝑘 (𝑖) = 1 𝑘 𝑘 𝑅(𝑺 (𝑖), 𝐴 (𝑖)) = 0, if 𝐴𝑘 (𝑖) = 0, (3) where 𝑟𝑛 ([𝑮𝑘 (𝑖)]𝑛 ) is the transmission rate of user 𝑛 on channel 𝑘 during slot 𝑖. We would like to point out that the time slot being considered is at the control level and contains several modulated symbols. Therefore, the data rate can be varied based on channel condition using adaptive transmission schemes similar to those proposed in [18]. It can also be noted that (3) does not ensure fairness among users. This will be handled by another control approach in Section V. As one sensing slot must be carried out during each frame, if at slot 𝐿, the sensing flag 𝐹 𝑘 (𝐿) = 0 then only the sensing action 𝐴𝑘 (𝐿) = 0 is allowed. Also, if at slot 𝑖, 𝐹 𝑘 (𝑖) = 1, i.e., sensing has been carried out, then in order to maximize the achievable throughput, we should set 𝐴𝑘 (𝑖) = 1. Based on this, we define the following CSI-based sensing/transmission policy. Definition 2: A CSI-based sensing/transmission policy 𝝅 is defined as a set {𝜓1 , 𝜓2 , . . . 𝜓𝐿 }, where, for slot 𝑖, 𝜓𝑖 maps the system state 𝑺 𝑘 (𝑖) into control action 𝐴𝑘 (𝑖) = 𝜓𝑖 (𝑺 𝑘 (𝑖)) and 𝜓𝑖 satisfies the following equations (4) and (5): 𝜓𝑖 (𝑺 𝑘 (𝑖)) ∈ {0, 1},
(4)
HOANG et al.: ADAPTIVE JOINT SCHEDULING OF SPECTRUM SENSING AND DATA TRANSMISSION IN COGNITIVE RADIO NETWORKS
{
if 𝐹 𝑘 (𝑖) = [𝑺 𝑘 (𝑖)]1 = 1 if 𝑖 = 𝐿 and 𝐹 𝑘 (𝑖) = [𝑺 𝑘 (𝑖)]1 = 0. (5) 4) State Transition Probabilities: Given a CSI-based policy 𝝅, the transitioning probabilities of the system state, i.e., { } 𝑝𝑆 (𝒔, 𝒔′ , 𝑎) = Pr 𝑺 𝑘 (𝑖 + 1) = 𝒔′ ∣ 𝑺 𝑘 (𝑖) = 𝒔 & 𝐴𝑘 (𝑖) = 𝑎 (6) can be readily derived. In particular, we can list the following cases: 𝜓𝑖 (𝑺 𝑘 (𝑖)) =
1, 0,
𝑝𝑆 ([0, 𝒈], [0, 𝒈 ′ ], 0) = 0 𝑝𝑆 ([0, 𝒈], [1, 𝒈 ′ ], 1) = 0 𝑝𝑆 ([0, 𝒈], [0, 𝒈 ′ ], 1) = 𝑝𝐺 (𝒈, 𝒈 ′ ) ′
𝑝𝑆 ([0, 𝒈], [1, 𝒈 ], 0) = 𝑝𝐺 (𝒈, 𝒈 ) 𝑝𝑆 ([1, 𝒈], [0, 𝒈 ′ ], 1) = 0
where 𝒈 and 𝒈 ′ are two possible 1 × 𝑁 vectors of channel states of 𝑁 users on channel 𝑘. 5) Obtaining Optimal Solutions: The problem of finding CSI-based sensing/transmission policy 𝝅 that maximizes the total system throughput for each frame can be written as: arg max
𝐿 {∑ ( )} E𝑺 𝑘 (𝑖) 𝑅 𝑺 𝑘 (𝑖), 𝜓𝑖 (𝑺 𝑘 (𝑖)) .
(8)
𝑖=1
The above FH-MDP can be solved effectively using dynamic programming techniques. We refer the readers to [17] for an in-depth study of these techniques. The complexity of FHMDP is of the order 𝑂(𝐿(2𝑁 𝑀 )2 ), where 𝐿 is the control horizon and 2𝑁 𝑀 is the number of possible states for 𝑺 𝑘 (𝑖). It should be pointed out that the optimal CSI-based policy obtained by solving (8) does not require knowledge of the future channel states. What required to obtain a solution to (8) is the channel transitioning statistics, 𝑝𝑆 (), as defined in (6) and (7). The resultant CSI-based policy is a look-up table, as illustrated in Table I (discussed in Section IV). From this look up table, if we are at a particular time slot and know the present channel state, we will know whether sensing or transmission should be carried out. C. Structural Properties Sensing/Transmission Policies
of
Optimal
CSI-based
∗ } be an optimal CSI-based policy Let 𝝅∗ = {𝜓1∗ , 𝜓2∗ , . . . 𝜓𝐿 obtained by solving the FH-MDP in (8). Letting 𝐽 ∗ (𝑗, 𝒔) be the expected throughput that can be obtained in slots 𝑗, 𝑗 + 1, . . . 𝐿 given that the system state at time slot 𝑗 is 𝒔 and the ∗ ∗ , . . . 𝜓𝐿 are applied, i.e., maps 𝜓𝑖∗ , 𝜓𝑖+1 𝐿 {∑ } ( ) 𝑅 𝑺 𝑘 (𝑖), 𝜓𝑖∗ (𝑺 𝑘 (𝑖)) ∣ 𝑺 𝑘 (𝑗) = 𝒔 , 𝐽 ∗ (𝑗, 𝒔) = E𝑺 𝑘 (𝑖) 𝑖=𝑗
(9)
the following Bellman’s equation always holds ([17]): { } ∑ 𝜓𝑖∗ (𝒔) ∈ arg max 𝑅(𝒔, 𝑎) + 𝑝𝑆 (𝒔, 𝒔′ , 𝑎)𝐽 ∗ (𝑖 + 1, 𝒔′ ) . 𝑎∈{0,1}
∀𝒔′
𝐽 ∗ (𝑖, [1, 𝒈]) − 𝐽 ∗ (𝑖, [0, 𝒈]) ≤ 𝐽 ∗ (𝑗, [1, 𝒈]) − 𝐽 ∗ (𝑗, [0, 𝒈]). (11) Proof: Please refer to the Appendix. Using Lemma 1, have can prove the following proposition. Proposition 1: Let 𝒔 = [0, 𝒈] be a particular system state, if 𝜓𝑖∗ (𝒔) = 0 then 𝜓𝑗∗ (𝒔) = 0 for all 1 ≤ 𝑖 < 𝑗 ≤ 𝐿. Proof: From the Bellman’s equation in (10) and the assumption that 𝜓𝑖∗ (𝒔) = 0, we have: ∑ 𝑝𝑆 (𝒔, 𝒔′ , 0)𝐽 ∗ (𝑖 + 1, 𝒔′ ) 𝑅(𝒔, 0) + ∀𝒔′
≥ 𝑅(𝒔, 1) +
∑
𝑝𝑆 (𝒔, 𝒔′ , 1)𝐽 ∗ (𝑖 + 1, 𝒔′ ),
(12)
∀𝒔′
𝑝𝑆 ([1, 𝒈], [1, 𝒈 ′ ], 1) = 𝑝𝐺 (𝒈, 𝒈 ′ ),
𝝅={𝜓1 ,...𝜓𝐿 }
Lemma 1: For every vector of channel states 𝒈 and 1 ≤ 𝑖 < 𝑗 ≤ 𝐿,
(7)
′
239
(10) Equation (10) is useful in proving the following structural properties of the optimal policy 𝝅 ∗ .
i.e., ∑
𝑝𝑆 (𝒔, 𝒔′ , 0)𝐽 ∗ (𝑖 + 1, 𝒔′ )
∀𝒔′
≥ max{𝑟𝑛 ([𝒈]𝑛 )} + 𝑛
∑
𝑝𝑆 (𝒔, 𝒔′ , 1)𝐽 ∗ (𝑖 + 1, 𝒔′ ),
(13)
∀𝒔′
where ∑ ∑ 𝑝𝑆 (𝒔, 𝒔′ , 0)𝐽 ∗ (𝑖 + 1, 𝒔′ ) = 𝑝𝐺 (𝒈, 𝒈 ′ )𝐽 ∗ (𝑖 + 1, [1, 𝒈′ ]) ∀𝒔′
and ∑
∀𝒈 ′
𝑝𝑆 (𝒔, 𝒔′ , 1)𝐽 ∗ (𝑖 + 1, 𝒔′ ) =
∀𝒔′
∑
(14) 𝑝𝐺 (𝒈, 𝒈 ′ )𝐽 ∗ (𝑖 + 1, [0, 𝒈′ ]).
∀𝒈′
(15)
From (13), (14), (15), it follows that ( ) ∑ 𝑝𝐺 (𝒈, 𝒈 ′ ) 𝐽 ∗ (𝑖 + 1, [1, 𝒈′ ]) − 𝐽 ∗ (𝑖 + 1, [0, 𝒈′ ]) (16)
∀𝒈′
≥ max{𝑟𝑛 ([𝒈]𝑛 )}. 𝑛
Now, from Lemma 1 and 𝑗 > 𝑖, we have: 𝐽 ∗ (𝑗 + 1, [1,𝒈 ′ ]) − 𝐽 ∗ (𝑗 + 1, [0, 𝒈′ ]) ≥ 𝐽 ∗ (𝑖 + 1, [1, 𝒈′ ]) − 𝐽 ∗ (𝑖 + 1, [0, 𝒈′ ]).
(17)
Combining (16) and (17) yields: ( ) ∑ 𝑝𝐺 (𝒈, 𝒈 ′ ) 𝐽 ∗ (𝑗 + 1, [1, 𝒈′ ]) − 𝐽 ∗ (𝑗 + 1, [0, 𝒈′ ]) ∀𝒈 ′
≥ max{𝑟𝑛 ([𝒈]𝑛 )}
(18)
𝑛
or ∑
𝑝𝑆 (𝒔, 𝒔′ , 0)𝐽 ∗ (𝑗 + 1, 𝒔′ )
∀𝒔′
≥ max{𝑟𝑛 ([𝒈]𝑛 )} + 𝑛
∑
𝑝𝑆 (𝒔, 𝒔′ , 1)𝐽 ∗ (𝑗 + 1, 𝒔′ ).
(19)
∀𝒔′
From (19) and the Bellman’s equation (10), it follows that 𝜓𝑗∗ (𝒔) = 0. This completes the proof. Remark 1: Proposition 1 tells us that, for each channel state, the optimal CSI-based policy tends to delay spectrum sensing toward the end of each frame. For each channel state, scheduling a data transmission in a particular slot will be throughput-beneficial if spectrum sensing can be scheduled in some later slot with a worse channel state. So the optimal
240
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 58, NO. 1, JANUARY 2010
policy tends to follow the ‘wait for a worse channel state’ strategy in scheduling spectrum sensing. However, as channel states for future time slots are not known when the the decision is made, the policy can only estimate the probability that such a worse channel state will occur. The earlier in the frame, the more likely that there will be a worse channel state in the subsequent time slots. That is why the optimal policy tends to delay sensing toward the later part of each frame. Proposition 2: Considering the case when the channel states vary independently across users and i.i.d. over time, for each slot 𝑖, let 𝒔1 = [0, 𝒈 1 ] and 𝒔2 = [0, 𝒈 2 ] be two possible system states such that max𝑛 {𝑟𝑛 ([𝒈 1 ]𝑛 )} > max𝑛 {𝑟𝑛 ([𝒈 2 ]𝑛 )}, if 𝜓𝑖∗ (𝒔1 ) = 0 then 𝜓𝑖∗ (𝒔2 ) = 0. Proof: From the Bellman’s equation in (10) and the assumption that 𝜓 ∗ (𝒔1 ) = 0, we have: ∑ 𝑝𝑆 (𝒔1 , 𝒔′ , 0)𝐽 ∗ (𝑖 + 1, 𝒔′ )
for short), Greedy CSI-based (Greedy for short), and Fixed sensing scheduling (Fixed for short). As has been noted, CSIbased control is independent over different channels, hence we focus on the performance of a single channel scenario while varying the number of users 𝑁 from 1 to 20. For each user, the channel is modeled by a 10-state FSMC (𝑀 = 10) with the corresponding transmission rates of 1, 2, 6, 9, 12, 18, 24, 36, 48, 54 Mbps (similar to the data rates of 802.11g). Each channel is i.i.d. overtime, with every state equally likely. For Greedy scheme, we consider two approaches to set the sensing threshold 𝑚∗ , i.e., Load-oblivious and Load-adaptive. In the Load-oblivious approach, regardless of the number of CR users, we always set the sensing threshold 𝑚∗ = 6, i.e., at the median channel condition. In the Load-oblivious approach, 𝑚∗ is chosen as: 𝑚 {∑ } 𝑚∗ = arg min 𝜋(𝑚) > 0.5 , (22)
∀𝒔′
≥ max{𝑟𝑛 ([𝒈 1 ]𝑛 )} + 𝑛
∑
𝑝𝑆 (𝒔1 , 𝒔′ , 1)𝐽 ∗ (𝑖 + 1, 𝒔′ ).
(20)
𝑚
∀𝒔′
As the channel state transitions are independent across users and i.i.d. over time, we have 𝑝𝑆 (𝒔1 , 𝒔′ , 0) = 𝑝𝑆 (𝒔2 , 𝒔′ , 0), this, together with the assumption that max𝑛 {𝑟𝑛 ([𝒈 1 ]𝑛 )} > max𝑛 {𝑟𝑛 ([𝒈 2 ]𝑛 )} and (20), yields ∑ 𝑝𝑆 (𝒔2 , 𝒔′ , 0)𝐽 ∗ (𝑖 + 1, 𝒔′ ) ∀𝒔′
≥ max{𝑟𝑛 ([𝒈 2 ]𝑛 )} + 𝑛
∑
𝑝𝑆 (𝒔𝑠 , 𝒔′ , 1)𝐽 ∗ (𝑖 + 1, 𝒔′ ).
(21)
∀𝒔′
From (10) and (20), it follows that 𝜓𝑖∗ (𝒔2 ) = 0. This completes the proof. Remark 2: Proposition 2 tells us that, when the channel is i.i.d. over time, for each slot within a frame, there exist a threshold such that: when the channel state is worse than the threshold, then sensing should be carried out in the time slot; and when the channel state is better than the threshold, data transmission should be carried out. The explanation for this is similar to that given in Remark 1, i.e., the optimal policy tends to follow the ‘wait for a worse channel state’ strategy in scheduling spectrum sensing. D. Other Scheduling Rules We also consider the following sensing/transmission scheduling schemes. 1) Greedy CSI-based Scheduling: Based on Remarks 1 and 2 on the structural properties of the optimal CSI-based scheduling policy, we propose a simple scheduling policy called ‘Greedy CSI-based Scheduling’. This greedy policy follows a simple rule: schedule spectrum sensing only when the channel states of all users are below some threshold state 𝑚∗ or when we are already at the last slot of each frame. 2) Fixed Sensing Scheduling: The first slot of each frame is always scheduled for spectrum sensing. IV. N UMERICAL R ESULTS FOR CSI- BASED S ENSING /T RANSMISSION P OLICIES In this section, we evaluate the performance of the three sensing/transmission scheduling algorithms described in Section III, i.e., Optimal CSI-based sensing/transmission (Optimal
𝑁
𝑗=1 𝑁
where 𝜋(𝑚) = 𝑚 −(𝑚−1) is the steady-state probability that 𝑀𝑁 the best channel state among 𝑁 users is 𝑚. Note that when 𝑁 = 1, Load-oblivious and Load-adaptive Greedy policies are the same. In Fig. 3, we plot the throughput of Greedy and Fixed policies, normalized by that of the corresponding Optimal policy, for different numbers of users and frame sizes. As can be observed, Load-adaptive Greedy policy performs very closed to optimal while Fixed-scheduling policy always performs worst. When the number of user is 1, Load-oblivious and Load-adaptive Greedy policies achieve the same performance, however, when the load (number of CR users) increases, the performance of Load-oblivious Greedy degrades to that of Fixed-scheduling. This show the importance of taking the number of CR users into account when setting the sensing threshold 𝑚∗ of Greedy policies. The performance advantage of Optimal and Load-adaptive Greedy, relative to Fixed scheduling scheme is more prominent when sensing time takes more percentage per each frame duration (i.e., when the frame size is shorten). This is because the gain of CSI-based sensing/transmission scheduling is limited to the time when sensing is carried out, while what we plot here is average over the whole frame duration. It is also evident that when the number of users increases, the gain of CSI-based sensing scheduling compared to Fixed sensing scheduling is less prominent. This is because from the system point of view, the effective channel state in each time slot is equal to the best channel state over all 𝑁 users. This effective channel state exhibit less variation when the number of users increases and therefore, there is less channel dynamic for CSI-based sensing scheduling to exploit. To further illustrate the dependent of Greedy policy on the sensing threshold 𝑚∗ , in Fig.‘4, we plot the throughput achieved by Greedy scheduling, normalized by that of Optimal policy, when the threshold 𝑚∗ is varied from 1 to 𝑀 = 10. As can be seen, there is an optimal value of 𝑚∗ at which Greedy performs very closed to optimal. This optimal value increases with the number of users 𝑁 and, as evident from Fig. 3, can be approximated by (22).
1
Normalized throughput
0.95
0.9
0.85
0.8 Load−adaptive Greedy (Frame size = 2) Load−oblivous Greedy (Frame size = 2) Fixed scheduling (Frame size = 2) Load−adaptive Greedy (Frame size = 5) Load−oblivious Greedy (Frame size = 5) Fixed scheduling (Frame size = 5)
0.75
0.7
1
2
3
4
5
6
7
8
Normalized throughput of Greedy scheme
HOANG et al.: ADAPTIVE JOINT SCHEDULING OF SPECTRUM SENSING AND DATA TRANSMISSION IN COGNITIVE RADIO NETWORKS
1
0.95
0.9
0.85
0.8
0.75
0.7
9 10 11 12 13 14 15 16 17 18 19 20
Fig. 3. Throughput of Greedy and Fixed scheduling policies, normalized by that of the Optimal policy. TABLE I L OOK - UP TABLE TO IMPLEMENT O PTIMAL CSI- BASED SENSING / TRANSMISSION POLICY. S = SENSING DECISION ; T = TRANSMISSION DECISION . S LOT 1 IS THE FIRST SLOT IN THE FRAME . State State State State State State State State State State
1 (1Mbps) 2 (2Mbps) 3 (6Mbps) 4 (9Mbps) 5 (12Mbps) 6 (18Mbps) 7 (24Mbps) 8 (36Mbps) 9 (48Mbps) 10 (54Mbps)
Slot 1 S S S T T T T T T T
Slot 2 S S S S T T T T T T
Slot 3 S S S S S T T T T T
1 user 5 users 10 users 1
2
3
4
5
6
7
8
9
10
Sensing threshold
Number of users
Ch. Ch. Ch. Ch. Ch. Ch. Ch. Ch. Ch. Ch.
241
Slot 4 S S S S S S T T T T
Slot 5 S S S S S S S S S S
To get a better understanding of the behavior of Optimal CSI-based sensing/transmission policies, in Table. I, we show the decisions made by such an optimal policy under different channel states and time slots within each frame. In the entries of this table, an ‘S’ indicates the decision to carry out sensing (if sensing has not been done in the frame) and a ‘T’ indicates the decision to carry out transmission and delay sensing to future slots. It is clear from Table. I that the optimal policy tends to delay spectrum sensing when the channel conditions are good and when there are still multiple slots left in the frame for scheduling sensing later on. This again confirms our observations in Remarks 1 and 2.
V. QCI- BASED S ENSING /T RANSMISSION A PPROACH In this section, we consider the case when CR nodes experience time-varying channels, stochastic data arrival, and have deadline constrained traffic. For that, we take a cross-layer control approach and adaptively schedule spectrum sensing and data transmission activities based on both the channel and queue condition to achieve good performance. We term this approach QCI-based sensing/transmission.
Fig. 4. Effect of choosing different sensing threshold 𝑚∗ on the performance of Greedy scheduling scheme.
A. Problem Formulation 1) Traffic Model: We assume that data packets randomly arrive to the buffers of 𝑁 CR nodes. All data packets of node 𝑛 have the same size of 𝑃𝑛 bits and the same deadline of 𝐷𝑛 seconds. If a packet is not transmitted by its deadline, it is dropped and considered lost. We also assume that the buffers are long enough so that overflow is negligible. 2) Channel Model: The channel model considered in this section is similar to that used in Section III, i.e., for each pair of user 𝑛 and channel 𝑘, the channel condition can take one of the 𝑀 possible states. We denote by 𝜇𝑘𝑛 (𝑡) the rate (in packets per second) at which user 𝑛 can transmit on channel 𝑘 at time 𝑡. We further assume that this rate does not change during the transmission of each packet. 3) Control and Objective: The operation of the CR network can be described as follows. Data transmission and spectrum sensing are carried out independently on different channels. For channel 𝑘, when spectrum sensing is not carried out, one of the 𝑁 CR nodes will be scheduled to transmit one packet. After the transmission of this packet, channel 𝑘 becomes available again and if sensing is not scheduled, some node will be allowed to transmit another packet on the channel. At time 𝑡, let 𝑤𝑛 (𝑡) denote the total queueing time (sojourn time) of the packet at the front of the buffer of CR node 𝑛. We define the following QCI-based sensing/transmission problem. Definition 3: QCI-based Sensing/Transmission Problem For each channel 𝑘, 1 ≤ 𝑘 ≤ 𝐾, supposing that the channel becomes available at time 𝑡 and assuming that 𝜇𝑘𝑛 (𝑡) and 𝑤𝑛 (𝑡) are known for all 𝑛 ∈ {1, 2, . . . 𝑁 }, select one of the following decisions: ∙ ∙
carrying out sensing on channel 𝑘, or allowing one of the 𝑁 CR nodes to transmit one packet on channel 𝑘,
so that the total packet loss rate due to deadline violation of 𝑁 CR nodes is minimized. This is subject to the constraint that 𝑇𝑠 seconds must be scheduled for sensing each channel during each frame.
242
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 58, NO. 1, JANUARY 2010
Data packets
Channels Data packets
1
Data packets VS packets of Channel 1
2
VS packets of Channel 2
Fig. 5. Representing the joint transmission/sensing problem as a parallel queueing system. Quiet sensing periods are represented by virtual sensing (VS) packets.
In Definition 3, the constraint that 𝑇𝑠 seconds must be scheduled for sensing during each frame does not require sensing to be carried out continuously. The sensing time can be broken into multiple non-contiguous sensing sub-periods within a frame. We further assume that energy detection is employed for spectrum sensing, with data fusion being carried out at the sensing node based on the samples collected from all the sensing sub-periods. The data fusion rule we employ is the equal gain combination (EGC) [19], [20] of signal powers at all time intervals. In particular, we simply compute the average value of the average received signal power at different sensing intervals and then compare this to the noise power times a threshold to make a decision. We choose EGC because it is simple and has relatively good performance [19], [20].
C. QCI-based Scheduling Disciplines With the introduction of VS users and VS packets, our QCI-based sensing/transmission problem can be abstracted as a parallel queueing network as in Fig. 5. We aim to schedule the traffic in this parallel queueing system so that the packet loss due to deadline violation for 𝑁 CR nodes is minimized while all VS packets meet their deadlines. However, for such a parallel queueing system, to the best of our knowledge, there has not been any scheduling scheme identified as optimal. We therefore select a suboptimal scheduling scheme that gives good performance. Let us start by reviewing some known efficient scheduling schemes. 1) G-𝑐𝜇 Rule: The Generalized-𝑐𝜇 rule (G-𝑐𝜇) introduced in [21]–[23] aims to minimize the queueing cost under heavy traffic. The queueing cost is represented by convex functions that increase with the packet queueing time. For deadlineconstrained traffic, the following cost function is proposed in [22]: (24) 𝐶𝑛 (𝑤) = (𝑤/𝐷𝑛 )𝛼 , where 𝑤 is the sojourn time of the oldest packet in the buffer of user 𝑛, 𝐷𝑛 is the deadline constraint, and 𝛼 > 1 is a selected value. The intuition behind this cost function is that, if the sojourn time 𝑤 is kept smaller than the constraint 𝐷𝑛 , then the queueing cost incured is mall, while this cost increases rapidly when 𝑤 > 𝐷𝑛 (as 𝛼 > 1). Applying to our queueing model in Fig.‘5, when channel 𝑘 is available at time 𝑡, the G-𝑐𝜇 rule schedules a packet of node 𝑢 that satisfies: 𝑢 ∈ arg max {𝐶𝑛′ (𝑤𝑛 (𝑡))𝜈𝑛𝑘 (𝑡)}, 𝑛∈퓝 ∪퓥
B. Spectrum Sensing as Virtual Transmission For each channel 𝑘, the effects of scheduling a quiet sensing period for 𝑞 seconds on the performance of 𝑁 CR nodes are the same as the effects we would have if the channel was assigned to some ‘virtual transmission’ for the same 𝑞 seconds. Therefore, we can abstract the sensing activities on the 𝐾 channels by introducing the following specifications. 1) Virtual Sensing Nodes: 𝐾 virtual sensing (VS) nodes are introduced, one for each of the 𝐾 channels. When sensing is carried out on channel 𝑘, we say VS node 𝑘 ‘transmits’ a VS packet. 2) Virtual Sensing Packets: The arriving patterns, lengths, and deadlines of VS packets are specified to match the sensing requirements. Suppose that, for each channel 𝑘, the sensing duration 𝑇𝑠 is divided into 𝑧 equal sub-periods of duration 𝑡𝑠 = 𝑇𝑠 /𝑧. Then, at the beginning of each frame, 𝑧 VS packets arrive to the buffer of VS node 𝑘. All of these VS packets have deadline of 𝑇𝑓 seconds. 3) Virtual Transmission Rates: VS node 𝑘 has VS packets for channel 𝑘, each packet take 𝑡𝑠 seconds to ‘transmit’. Therefore, the virtual transmission rate for VS user 𝑘 on channel 𝑙 (1 ≤ 𝑘, 𝑙 ≤ 𝐾) is: { 1/𝑡𝑠 , if 𝑘 = 𝑙, 𝑙 (23) 𝜙𝑘 (𝑡) = 0, otherwise. Note that the virtual transmission rates for VS user 𝑘 does not change over time.
(25)
where 퓝 and 퓥 are the set of CR nodes and the set of VS nodes, respectively, 𝐶𝑛′ (.) is the first order derivative of function 𝐶𝑛 (.), and { 𝑘 𝜇𝑛 (𝑡), if 𝑛 ∈ 퓝 , 𝜈𝑛𝑘 (𝑡) = (26) 𝜙𝑘𝑛 (𝑡), if 𝑛 ∈ 퓥. 2) M-LWDF Rule: In [24], Andrews et al. introduced the Modified Largest Weighted Delay First (M-LWDF) scheduling rule that, when applied to the queueing system in Fig. 5, assigns channel 𝑘 to node 𝑢 that satisfies: 𝑢 ∈ arg max {𝛾𝑛 (𝑤𝑛 (𝑡))𝛽 𝜈𝑛𝑘 (𝑡)}, 𝑛∈퓝 ∪퓥
(27)
where 𝛾𝑛 is a weighting factor representing the delay constraint of node 𝑛 and 𝛽 is some selected constant. In [24], it is shown that M-LWDF rule is throughput optimal, i.e., MLWDF stabilizes the queueing system if it is possible to do so with any other queueing discipline. It is further argued that by stabilizing the queues, the queueing latency experienced by packets is also kept bounded. 3) Proposed QCI-based Sensing/Transmission Scheduling Rule: Note that both G-𝑐𝜇 and M-LWDF scheduling rules are only optimal under heavy traffic conditions. For our system, taking these two scheduling rules as guidelines, we have tested different dynamic scheduling rules that take 𝑤𝑛 (𝑡), 𝜇𝑘𝑛 (𝑡), and 𝐷𝑛 into account and found that selecting user 𝑢 to transmit on channel 𝑘 according to the following rule gives the most
HOANG et al.: ADAPTIVE JOINT SCHEDULING OF SPECTRUM SENSING AND DATA TRANSMISSION IN COGNITIVE RADIO NETWORKS
favorable performance, in terms of reducing packet deadline violation: } { 𝑤 (𝑡) 𝑛 𝜈𝑛𝑘 (𝑡) . (28) 𝑢 ∈ arg max 𝐷𝑛 𝑛∈퓝 ∪퓥 As can be seen, (28) is the same as (25) when 𝐶𝑛 (𝑤) = 𝑤2 /𝐷𝑛 and the same as (27) when 𝛾𝑛 = 1/𝐷𝑛 and 𝛽 = 1. The intuition behind (28) is that, in order to reduce packet loss due to deadline violation, higher priority should be given to packets with more stringent deadlines, longer sojourn time, and that have faster transmission rate. Furthermore, the factor 𝑤2 in the cost function ensures that the cost quickly increases with the sojourn time, which gives extra incentive to schedule long-waiting packets. It can be noted that, when VS packets are scheduled together with data packets based on (28), there is no guarantee that every VS packet will be transmitted by its deadline. To meet such requirement, we introduce the following QCI-based sensing/transmission scheduling policy. First, we find a user 𝑢 according to (28). If 𝑢 is a VS user, then transmit a VS packet of 𝑢 on channel 𝑘, i.e., sensing is carried out on channel 𝑘. If 𝑢 is one of the 𝑁 CR nodes, we need to calculate the time that channel 𝑘 will be occupied if a packet of 𝑢 is transmitted. If this transmission time leads to deadline violation for any VS packets of channel 𝑘, then we do not allow 𝑢 to transmit on the channel and schedule a sensing packet instead. If there is no deadline violation for any VS packets of channel 𝑘, then we allow data user 𝑢 to transmit one packet. The pseudo codes for the above algorithm is given in Algorithm 1. Algorithm 1 QCI-based Sensing/Transmission Scheduling 1: while ‘channel 𝑘 is available’ do 2: 𝑢 ← arg max { 𝑤𝐷𝑛 (𝑡) 𝜈𝑛𝑘 (𝑡)} 𝑛 3: 4: 5: 6: 7: 8: 9: 10: 11: 12:
𝑛∈퓝 ∪퓥
if 𝑢 is VS user 𝑘 then carry out sensing on channel 𝑘 else if ‘there is enough time to sense channel 𝑘 after user 𝑢 transmits one packet’ then allow user 𝑢 to transmit one data packet else carry out sensing on channel 𝑘 end if end if end while
D. Other Scheduling Rules To evaluate the performance of the proposed QCI-based sensing/transmission scheme, we compare that to the performance of some other scheduling schemes described below. 1) Fixed Sensing, Random Transmission: In this simple scheduling scheme, the sensing time is always fixed at the beginning of each frame. For data transmission, a backlogged user is picked randomly from the set of 𝑁 users. By comparing the performance of this scheme to that of the QCI-based sensing/transmission scheme proposed above, we can determine the gain of jointly scheduling the sensing and transmission activities in a dynamic manner.
243
2) Fixed Sensing, QCI-based Transmission: In this scheme, the sensing time is always fixed at the beginning of each frame. For data transmission, we applied the same QCI-based scheduling rule as given in (28). In particular, after sensing has been carried out and when channel 𝑘 is available, a user is picked according to the following rule: { 𝑤 (𝑡) } 𝑛 𝜇𝑘𝑛 (𝑡) . (29) 𝑢 ∈ arg max 𝐷𝑛 𝑛∈퓝 By comparing the performance of this scheme to that of the QCI-based sensing/transmission scheme, we can determine the gain obtained by dynamically scheduling the sensing period within each frame. VI. S IMULATION R ESULTS OF QCI- BASED S ENSING /T RANSMISSION P OLICIES In this section, we obtain simulation results for the performance of different scheduling rules described in Section V. We abbreviate the QCI-based sensing/transmission scheme as QCST, the Fixed Sensing QCI-based Transmission scheme as FSQCT, and the Fixed Sensing Random Transmission scheme as FSRT. A. Simulation Model The number of channels in the CR network is fixed at 𝐾 = 2. Each channel is time varying and can take 𝑀 = 5 possible states, corresponding to the transmission rates of 0.5, 1, 2.75, and 5.5 Mbps (about half of the rates of IEEE 802.11b). We assume that each channel between CR users varies i.i.d. overtime and takes all possible states with equal probability. The duration of each channel state is exponentially distributed with average of either 5 ms (in Fig. 6, 7, 9, 10) or 10 ms (in Fig. 8). Time is divided into frames, each of duration 𝑇𝑓 = 200 ms. The required sensing time 𝑇𝑠 is set at either 10 ms (in Fig. 7) or 20 ms (in Fig. 6, 8, 9, 10). We consider two traffic models, i.e., data traffic and voice over IP (VoIP) traffic. For a user with data traffic, data packets arrive to his buffer according to a Poisson distribution. All data packets have the same length of 1000 bytes and deadline of 40 ms. For a user with VoIP traffic, we assume that the G.729 vocoder is employed with the following specifications [25]: i) each VoIP user switches between talking and silent modes, with the durations of talking and silent periods exponentially distributed with average of 352 ms and 650 ms, respectively; ii) during each talking period, a VoIP packet of size 206 bytes is generated every 20 ms. We assume that the deadline for each VoIP packet is 20 ms. B. Results and Discussion Consider the case when only data users are present in the network. In Fig. 6, we plot the packet loss rate (due to deadline violation) versus the Poisson arrival rate of data users. As can be seen, when the location of the sensing period within each frame is fixed and data users are scheduled in a random manner (using FSRT scheme), the performance is much worse than other dynamic scheduling schemes. Comparing the performance of FSQCT scheme to that of our proposed
244
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 58, NO. 1, JANUARY 2010
0.16
0.2
FSQCT QCST (1 sensing slot) QCST (2 sensing slots) QCST (4 sensing slots) QCST (8 sensing slots)
Packet loss rate (deadline violation)
Packet loss rate (deadline violation)
0.25
0.15
0.1
0.05
0 0.05
0.06
0.07
0.08
0.09
0.1
0.11
0.14 0.12 0.1 0.08 0.06 0.04 0.02 0 0.05
0.12
FSQCT QCST (1 sensing slot) QCST (2 sensing slots) QCST (4 sensing slots)
0.06
Arrival rate (packets/ms/user) Fig. 6. Performance of different sensing/transmission scheduling schemes. Number of CR nodes 𝑁 = 10, all users are data users with Poisson arrival traffic. Required sensing time 𝑇𝑠 = 20 ms. Average duration of each channel state is 5 ms.
0.07
0.08
0.09
0.1
0.11
0.12
Arrival rate (packets/ms/user) Fig. 7. Performance of different sensing/transmission scheduling schemes. Number of CR nodes 𝑁 = 10, all users are data users with Poisson arrival traffic. Required sensing time 𝑇𝑠 = 10 ms. Average duration of each channel state is 5 ms.
QCST scheme, we can observed a significant gain of the QCST scheme when the system is lightly loaded. At higher traffic load, the performance gain is less significant. This is because when the system is highly loaded, there is not much freedom to exercise dynamic scheduling. However, we can further improve the performance of the QCST scheme by dividing the sensing period into multiple shorter sub-periods and scheduling these sub-periods dynamically. For example, if the sensing duration (of 20 ms) is divided into 8 sensing sub-periods (of 250 us), significant improvement in the packet loss rate can be achieved at high traffic load range. The results in Fig. 6 are obtained when the required sensing duration is 20 ms for every frame of 200 ms. In Fig. 7, we look at the performance of different scheduling schemes when the required sensing time is reduced from 20 ms to 10 ms. As expected, when the required sensing time is reduced, the packet loss rate also reduced, relative to what shown in Fig. 6. When comparing Fig. 6 to Fig. 7, it can also be noted that the performance gain of QCST scheme, relative to that of FSQCT scheme, is less prominent when the required sensing time is reduced. This is because with less required sensing time, the need to dynamically schedule the sensing period is less prominent. In the limit when the required sensing time approaches zero, our proposed QCST rule approaches FSQCT rule. In Fig. 6, we assume that, on average, the channels stay unchanged for 5 ms. In Fig. 8, we consider the case when, on average, the channels stay unchanged for 10 ms. As can be observed, the gain of QCST scheme, relative to that of FSQCT scheme, is still significant. However, when comparing Fig. 6 to Fig. 8, it can also be noted that the performance of all scheduling schemes degrade when the channels vary slower in time. This is because all schemes considered in Figs 6 and 8 benefit from multiuser diversity and this diversity is reduced when the channels are less dynamic in time. Next, we consider the case when both data users and VoIP users are present. We fixed the number of VoIP users at 20
Packet loss rate (deadline violation)
0.35
0.3
FSQCT QCST (1 sensing slot) QCST (4 sensing slots) QCST (8 sensing slots)
0.25
0.2
0.15
0.1
0.05
0 0.05
0.06
0.07
0.08
0.09
0.1
0.11
0.12
Arrival rate (packets/ms/user) Fig. 8. Performance of different sensing/transmission scheduling schemes. Number of CR nodes 𝑁 = 10, all users are data users with Poisson arrival traffic. Required sensing time 𝑇𝑠 = 20 ms. Average duration of each channel state is 10 ms.
and vary the number of data users from 4 to 16. Packets arrive to each data user’s buffer according to a Poisson distribution with rate of 0.08 packets/ms. In Figs. 9 and 10, we plot the packet loss rates for data and VoIP users versus the number of data users, for the FSRT, FSQCT, and our proposed QCST schemes. As can be observed, Fixed sensing, random transmission scheme always has worst performance. When comparing the performance of our proposed QCST scheme to that of the FSQCT scheme, it can be noted that the gain of QCST is very significant, especially for the VoIP users. This can be explained by the fact that VoIP users have more stringent deadline constraint and therefore, benefit more from dynamic scheduling of the spectrum sensing periods. VII. C ONCLUSION This paper addresses an important problem in cognitive radio (CR) networks, i.e., to minimize the negative effects
HOANG et al.: ADAPTIVE JOINT SCHEDULING OF SPECTRUM SENSING AND DATA TRANSMISSION IN COGNITIVE RADIO NETWORKS
0
0
10
Packet loss rate (VoIP users)
Packet loss rate (data users)
10
−1
10
−2
10
−3
10
FSRT FSQCT QCST(1 sensing slot) QCST (4 sensing slots)
−4
10
245
−1
10
−2
10
−3
10
−4
10
−5
10
4
6
8
10
12
14
FSRT FSQCT QCST (1 sensing slot) QCST (4 sensing slots)
16
4
6
8
Number of data users
10
12
14
16
Number of data users
Fig. 9. Performance of different sensing/transmission scheduling schemes, in terms of packet loss rate for data users. Number of VoIP users = 20. Required sensing time 𝑇𝑠 = 20 ms. Average duration of each channel state is 5 ms.
Fig. 10. Performance of different sensing/transmission scheduling schemes, in terms of packet loss rate for VoIP users. Number of VoIP users = 20. Required sensing time 𝑇𝑠 = 20 ms. Average duration of each channel state is 5 ms.
of postponing CR communications for spectrum sensing, i.e., primary user detection. Our novelty is to jointly schedule the spectrum-sensing and data-transmission activities in CR networks so that all the sensing requirements are met while the throughput loss or transmission delay is minimized. We consider two closely related scenarios: i) when CR nodes always have data to transmit and experience time-varying channels; and ii) when CR nodes experience both stochastic data traffic and time-varying channels. For the first scenario, a CSIbased sensing/transmission approach that adaptively schedules sensing/transmission activities based on the channel condition is proposed. For the second scenario, a QCI-based approach that takes both channel and queue conditions into account is presented. We obtain theoretical analysis and numerical results that highlight the effectiveness of our proposed approaches.
Therefore,
∗
(32) For time slot 𝐿 − 1 and state [0, 𝒈], there are two possible control actions, i.e., to transmit or to sense the channel. If we choose the sensing action at time slot 𝐿 − 1 and state [0, 𝒈] and then follow ∑ the optimal control decision in slot 𝐿, the total reward is 𝒈′ 𝑝𝐺 (𝒈, 𝒈 ′ )𝐽 ∗ (𝐿, [1, 𝒈′ ]). However, as 𝐽 ∗ (𝐿 − 1, [0, 𝒈]) is the optimal reward that can be achieved starting at time slot 𝐿 − 1 and state [0, 𝒈], we must have: ∑ 𝑝𝐺 (𝒈, 𝒈 ′ )𝐽 ∗ (𝐿, [1, 𝒈 ′ ]). (33) 𝐽 ∗ (𝐿 − 1, [0, 𝒈]) ≥ 𝒈′
From (32) and (33), it follows that (34)
𝑛
∗
𝐽 (𝑖, [1, 𝒈])−𝐽 (𝑖, [0, 𝒈]) ≤ 𝐽 (𝑖+1, [1, 𝒈])−𝐽 (𝑖+1, [0, 𝒈]). (30) We prove (30) by induction. At the last time slot of each frame, i.e., time slot 𝐿, the optimal control action for state ∗ ([1, 𝒈]) = 1, i.e., a transmission decision, while [1, 𝒈] is 𝜓𝐿 ∗ the optimal control action for state [0, 𝒈] is 𝜓𝐿 ([0, 𝒈]) = 0, i.e., a sensing decision. Therefore, 𝐽 ∗ (𝐿, [1, 𝒈]) − 𝐽 ∗ (𝐿, [0, 𝒈]) = 𝑅([1, 𝒈], 1) − 𝑅([0, 𝒈], 0) = max{𝑟𝑛 ([𝒈]𝑛 )}. 𝑛
𝑝𝐺 (𝒈, 𝒈 ′ )𝐽 ∗ (𝐿, [1, 𝒈 ′ ]).
𝒈′
≤ max{𝑟𝑛 ([𝒈]𝑛 )} = 𝐽 ∗ (𝐿, [1, 𝒈]) − 𝐽 ∗ (𝐿, [0, 𝒈]),
Proof: It can be noted that proving Lemma 1 is equivalent to proving that for every vector of channel stage 𝒈 and 1 ≤ 𝑖 < 𝐿, the following inequality holds: ∗
𝑛
∑
𝐽 ∗ (𝐿 − 1, [1, 𝒈]) − 𝐽 ∗ (𝐿 − 1, [0, 𝒈])
A PPENDIX P ROOF OF L EMMA 1
∗
𝐽 ∗ (𝐿−1, [1, 𝒈]) = max{𝑟𝑛 ([𝒈]𝑛 )}+
(31)
For time slot 𝐿 − 1 and system state [1, 𝒈], the optimal con∗ trol action is 𝜓𝐿−1 ([1, 𝒈]) = 1, i.e., a transmission decision.
which means that (30) holds for 𝑖 = 𝐿 − 1. Now assuming that 𝐽 ∗ (𝑖, [1, 𝒈])−𝐽 ∗ (𝑖, [0, 𝒈]) ≤ 𝐽 ∗ (𝑖+1, [1, 𝒈])−𝐽 ∗ (𝑖+1, [0, 𝒈]), (35) for some value of 𝑖 where 1 < 𝑖 ≤ 𝐿 − 1, we need to prove that, 𝐽 ∗ (𝑖−1, [1, 𝒈])−𝐽 ∗ (𝑖−1, [0, 𝒈]) ≤ 𝐽 ∗ (𝑖, [1, 𝒈])−𝐽 ∗ (𝑖, [0, 𝒈]). (36) There are two cases to consider: ∙ Case 1: At slot 𝑖 − 1, the optimal decision for state [0, 𝒈] is a sensing decision. In this case, from the Bellman’s equation (10), it follows that max{𝑟𝑛 ([𝒈]𝑛 )} 𝑛 ( ) ∑ (37) ≤ 𝑝𝐺 (𝒈, 𝒈 ′ ) 𝐽 ∗ (𝑖, [1, 𝒈′ ]) − 𝐽 ∗ (𝑖, [0, 𝒈′ ]) . 𝒈′
246
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 58, NO. 1, JANUARY 2010
However, from (35), we have: ( ) ∑ 𝑝𝐺 (𝒈, 𝒈 ′ ) 𝐽 ∗ (𝑖, [1, 𝒈′ ]) − 𝐽 ∗ (𝑖, [0, 𝒈′ ]) 𝒈′
≤
∑
R EFERENCES
( ) 𝑝𝐺 (𝒈, 𝒈 ′ ) 𝐽 ∗ (𝑖 + 1, [1, 𝒈′ ]) − 𝐽 ∗ (𝑖 + 1, [0, 𝒈′ ]) .
𝒈′
(38)
Combining (37) and (38) yields max{𝑟𝑛 ([𝒈]𝑛 )} 𝑛 ( ) ∑ ≤ 𝑝𝐺 (𝒈, 𝒈 ′ ) 𝐽 ∗ (𝑖 + 1, [1, 𝒈′ ]) − 𝐽 ∗ (𝑖 + 1, [0, 𝒈′ ]) . 𝒈′
(39)
From (39) and the Bellman’s equation (10), it follows that, at slot 𝑖, the optimal control action for state [0, 𝒈] is also a sensing action. Therefore 𝐽 ∗ (𝑖, [1, 𝒈]) − 𝐽 ∗ (𝑖, [0, 𝒈]) = 𝐽 ∗ (𝑖 − 1, [1, 𝒈]) − 𝐽 ∗ (𝑖 − 1, [0, 𝒈]) = max{𝑟𝑛 ([𝒈]𝑛 )}.
(40)
𝑛
∙
That means (36) holds for this case. Case 2: At slot 𝑖−1, the optimal decision for state [0, 𝒈] is a transmission decision. In this case, from the Bellman’s equation (10), it follows that 𝐽 ∗ (𝑖 − 1, [1, 𝒈]) − 𝐽 ∗ (𝑖 − 1, [0, 𝒈]) ≤ max{𝑟𝑛 ([𝒈]𝑛 )}. 𝑛 (41) Now if at time slot 𝑖, the optimal decision is a sensing decision, we have: 𝐽 ∗ (𝑖, [1, 𝒈]) − 𝐽 ∗ (𝑖, [0, 𝒈]) = max{𝑟𝑛 ([𝒈]𝑛 )} 𝑛
≥ 𝐽 ∗ (𝑖 − 1, [1, 𝒈]) − 𝐽 ∗ (𝑖 − 1, [0, 𝒈]).
(42)
Else, if at time slot 𝑖, the optimal decision is a transmission decision then: 𝐽 ∗ (𝑖, [1, 𝒈]) − 𝐽 ∗ (𝑖, [0, 𝒈]) ( ) ∑ 𝑝𝐺 (𝒈, 𝒈 ′ ) 𝐽 ∗ (𝑖 + 1, [1, 𝒈′ ]) − 𝐽 ∗ (𝑖 + 1, [0, 𝒈′ ]) . = 𝒈′
(43)
However 𝐽 ∗ (𝑖 − 1, [1, 𝒈]) − 𝐽 ∗ (𝑖 − 1, [0, 𝒈]) ( ) ∑ 𝑝𝐺 (𝒈, 𝒈 ′ ) 𝐽 ∗ (𝑖, [1, 𝒈′ ]) − 𝐽 ∗ (𝑖, [0, 𝒈 ′ ]) = 𝒈′
≤
∑
( ) 𝑝𝐺 (𝒈, 𝒈 ′ ) 𝐽 ∗ (𝑖 + 1, [1, 𝒈′ ]) − 𝐽 ∗ (𝑖 + 1, [0, 𝒈′ ]) .
𝒈′
(44)
From (43) and (44), it follows that 𝐽 ∗ (𝑖, [1, 𝒈]) − 𝐽 ∗ (𝑖, [0, 𝒈]) ≥ 𝐽 ∗ (𝑖 − 1, [1, 𝒈]) − 𝐽 ∗ (𝑖 − 1, [0, 𝒈]).
(45)
Thus, from (42) and (45), it follows that (36) also holds in this Case 2. That completes the proof.
[1] FCC, “Spectrum policy task force report, FCC 02-155,” Nov. 2002. [2] W. A. Gardner, “Exploitation of spectral redundancy in cyclostationary signals,” IEEE Signal Process. Mag., vol. 8, no. 2, pp. 14–36, Apr. 1991. [3] D. Cabric, S. Mishra, and R. W. Brodersen, “Implementation issues in spectrum sensing for cognitive radio,” in Proc. Asilomar Conf. Signals, Systems Computers, Oct. 2004. [4] A. Sahai and D. Cabric, “Spectrum sensing: fundamental limits and practical challenges,” in Proc. IEEE International Symposium New Frontiers Dynamic Spectrum Access Networks (DySPAN), Nov. 2005. [5] N. Han, S. H. Shon, J. O. Joo, and J. M. Kim, “Spectral correlation based signal detection method for spectrum sensing in IEEE 802.22 WRAN systems,” in Proc. Int. Conf. Advanced Commun. Technol., Feb. 2006. [6] Y. H. Zeng and Y. C. Liang, “Eigenvalue based sensing algorithms for cognitive radio,” IEEE Trans. Commun., vol. 57, no. 6, pp. 1784–1793, June 2009. [7] C. Cordeiro, M. Ghosh, D. Cavalcanti, and K. Challapali, “Spectrum sensing for dynamic spectrum access of TV bands,” in Proc. 3rd International Conf. Cognitive Radio Oriented Wireless Networks Commun. (CrownCom), May 2008. [8] Q. Zhao, S. Geirhofer, L. Tong, and B. M. Sadler, “Opportunistic spectrum access via periodic channel sensing,” IEEE Trans. Signal Process., vol. 58, no. 2, pp. 785–796, Feb. 2008. [9] H. Kim and K. G. Shin, “In-band spectrum sensing in cognitive radio networks: energy detection or feature detection?” in Proc. 14th ACM International Conf. Mobile Computing Networking, 2008. [10] D. Datla, R. .Rajbanshi, A. M. Wyglinski, and G. J. Minden, “Parametric adaptive spectrum sensing framework for dynamic spectrum access networks,” in Proc. IEEE DySPAN, Apr. 2007. [11] Y. Chen, Q. Zhao, and A. Swami, “Joint design and separation principle for opportunistic spectrum access in the presence of sensing errors,” IEEE Trans. Inf. Theory, vol. 54, no. 5, pp. 2053–2071, May 2008. [12] A. T. Hoang, Y.-C. Liang, D. T. C. Wong, Y. Zeng, and R. Zhang, “Opportunistic spectrum access for energy-constrained cognitive radios,” IEEE Trans. Wireless Commun., vol. 8, no. 3, pp. 1206–1211, Mar. 2009. [13] H. S. Wang and N. Moayeri, “Finite-state Markov channel—a useful model for radio communication channels,” IEEE Trans. Veh. Technol., vol. 44, no. 1, pp. 163–171, Feb. 1995. [14] Y. C. Liang, Y. H. Zeng, E. Peh, and A. T. Hoang, “Sensing-throughput tradeoff for cognitive radio networkss,” IEEE Trans. Wireless Commun., vol. 7, no. 4, pp. 1326–1337, Apr. 2008. [15] IEEE 802.22 Wireless RAN, “Functional requirements for the 802.22 WRAN standard, IEEE 802.22- 05/0007r46,” Oct. 2005. [16] A. T. Hoang, Y.-C. Liang, and M. H. Islam, “Maximizing throughput of cognitive radio networks with limited primary users’ cooperation,” in Proc. IEEE ICC, June 2007. [17] D. P. Bertsekas, Dynamic Programming and Optimal Control, 2nd ed., vols. 1 and 2. Athena Scientific, 2001. [18] A. J. Goldsmith and S. G. Chua, “Variable-rate variable-power MQAM for fading channels,” IEEE Trans. Commun., vol. 45, no. 10, pp. 1218– 1230, Oct. 1997. [19] Z. Quan, S. Cui, and A. H. Sayed, “Optimal linear cooperation for spectrum sensing in cognitive radio networks,” IEEE J. Sel. Topics Signal Process., vol. 2, no. 1, pp. 28–40, Feb. 2007. [20] J. Ma and Y. Li, “Soft combination and detection for cooperative spectrum sensing in cognitive radio networks,” in Proc. IEEE GlobeCom, Nov. 2007. [21] J. A. V. Mieghem, “Dynamic scheduling with convex delay costs: the generalized 𝑐𝜇 rule,” Annals Applied Probability, vol. 5, no. 3, pp. 809– 833, Aug. 1995. [22] ——, “Due-date scheduling: asymptotic optimality of generalized longest queue and generalized largest delay rules,” Operations Research, vol. 51, no. 1, pp. 113–122, Jan.–Feb. 2003. [23] A. Maldelbaum and A. L. Stolyar, “Scheduling flexible servers with convex delay costs: heavy-traffic optimality of the generalized 𝑐𝜇-rule,” Operation Research, vol. 52, no. 6, pp. 836–855, Nov.–Dec. 2004. [24] M. Andrews, K. Kumaran, K. Ramanan, A. L. Stolyar, R. Vijayakumar, and P. Whitting, “Scheduling in a queueing system with asynchronously varying service rates,” Probability Engineering Inf. Sciences, vol. 18, pp. 191–217, 2004. [25] “Voice over IP—per call bandwidth consumption.” [Online]. Available: http://www.cisco.com/en/US/tech/tk652/tk698\\/technologies\_tech\ _note09186a0080094ae2.shtml\#formulae
