Novel Policy Reasoning Architecture for Cognitive Radio Environments

This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE Globecom 2010 proceedings.

Novel Policy Reasoning Architecture for Cognitive Radio Environments Daniel Denkovski, Valentina Pavlovska, Vladimir Atanasovski and Liljana Gavrilovska Ss. Cyril and Methodius University Faculty of Electrical Engineering and Information Technologies Skopje, Macedonia {danield, valenpav, vladimir, liljana}@feit.ukim.edu.mk Abstract— Policing a wireless network is an evolving and exciting paradigm, specially emphasized with the emergence of cognitive radio networks. The possibility to manage radio resources efficiently and achieve different goals imposed by different stakeholders with dynamically changing policies in a wireless network is the main advantage of the policy based cognitive system. Moreover, the ability to express a variety of policy types for different purposes allows for high system flexibility and adaptability. This paper presents novel policy reasoning architecture for cognitive radio networks and elaborates on its functionalities and applications. Additionally, the paper provides a proof-of-concept testing of spectrum usage management feature of the architecture using USRP2 nodes. The elaborated policy framework is general enough to handle wide range of resource management issues in the wireless domain. Index Terms— Policy Reasoning, Policy Architecture, Cognitive Radio Networks, Resource Management

I.

INTRODUCTION

The cognitive radios [1] aim to solve the problem of wireless spectrum scarcity by dynamically exploiting the spectrum holes and thus improving the spectrum utilization. They are envisioned as smart radios using the features of environmental awareness, optimization and learning [2] in order to maximize their performances and achieve better spectrum resources utilization. Policy based cognitive radio networks are a subset of cognitive radio networks employing policies to accomplish the previously described goals. Policies are set of rules that determine the radio behavior in the network. In current radio systems, policies are developed by regulatory bodies and hardcoded into the radio devices. However, this approach cannot cope with the rapidly changing spectrum needs. The aim of the policy based cognitive radio networks is to separate the policies from the radio firmware, thus achieving more efficient resource management by dynamically changing the policy set. Policies can be supplied from different sources, i.e. regulators can express their regulations in terms of spectrum usage, operators can enforce their preferences of network resources usage and even users are able to apply policies expressing their preferences and requirements. Many research efforts on policy based cognitive radios in recent years have been put by DARPA with its neXt

Generation (XG) radio program [3] and SRI (Stanford Reserach Institute) International. They provided the Cognitive Radio Language CoRaL [4], an ontology language used to express policies and XG Policy Reasoner (PR) used to reason upon CoRaL specific policies. Two versions of the PR are available: XG Prolog PR [5] and XG Maude PR [6]. However, these concepts lack generalization into a generic policy reasoning architecture and need ontology extensions in order to cover all possible use cases that may be governed by the policy approach. This paper proposes a novel and general policy based reasoning architecture for cognitive radio networks. It elaborates all architectural building blocks and their functionalities and interactions. The proposed architecture offers a great deal of flexibility in terms of covering different parameters in the area of cognitive resource management and possibilities for many extensions and adaptations. The architecture supports policies coming from regulator, operator and users allowing them to dynamically control the devices by changing the policy set. The paper is organized as follows. Section II describes the proposed policy architecture, the composite elements, interfaces and interactions. Section III covers the functionalities offered by the proposed policy framework. Section IV proposes several alternative use cases of the policy architecture describing a proof of functionality testing results in a spectrum usage management application scenario. Finally, section V concludes the paper. II.

NOVEL POLICY REASONING ARCHITECTURE

The policy architecture proposed in this section embraces policies coming from all stakeholders offering options for each of them to express their specific goals. Moreover, the architecture supports dynamic resource management through dynamic policy changes that reflect the different behavior of the terminals. The full set of policies is efficiently reasoned and the reasoning output is presented to the resource management system as an available solution set. Unlike the previous work in this area that mainly target a limited resource management solutions, the proposed approach defines a general policy architecture that can dynamically handle a large set of resource management requirements, allowing all the stakeholders to apply their preferences. This section describes the system in

978-1-4244-5637-6/10/$26.00 ©2010 IEEE


details along with the system components and interfaces, the policy protocol specifying the messages and interactions between the entities and the policy language and the Policy Reasoner (PR) used in the framework. A. Architectural Components and Interfaces The proposed policy system architecture comprises 3 main elements, i.e. a policy server, a Policy Engine (PE) and a Policy Handling Toolbox (PHT) which is part of a Cognitive Resource Manager (CRM). The first one is located at the network side, while the other two are terminal based policy elements (Fig 1).

capable of performing the previously mentioned assignments, the policy engine consists of 3 components. Policy Engine Database (PED) is the local storage of the operator and regulator policies dedicated to the host user as well as the locally derived user policies. Policy Engine Manager (PEM) is assigned to handle the communication of the PE with the other policy network entities. It relays policy requests from the CRM to the PR and the policy replies in the opposite direction. It also handles the communications with the policy server, sending user registration message to the server and receiving policies and policy change notifications from the server. Policy Reasoner (PR) is the main component of the policy engine in charge of performing the reasoning process on the set of policies in PED after every received policy query, thus providing the solution space to the CRM.

Figure 1. Policy System Architecture

Policy Server. The policy server is the central policy repository in the network storing policies coming from operator and regulator sides. The proposed policy architecture allows several policy servers to coexist in the network in a centralized or distributed manner. The Policy Server comprises several components elaborated in the following paragraphs. Policy Server Database (PSD) is a central storage of the policies and policy users in the network. It keeps trace on the active users and active policies in the network and the user– policies associations. To satisfy these requirements, the PSD is divided into 3 tables: Users, Policies and ActivePolicies table. The Users table is part of the database dedicated to user related information, i.e. the users’ class and device type, the users’ identification and IP address in the network. The Policies table stores all policies coming from the operator and the regulator stakeholders. It keeps information on the policy identification, policy type, users’ class and device type this policy is dedicated to and the policy rules (the actual policy written in policy language specific format). The ActivePolicies table is used to associate the active users with their policies, providing the option when a policy is changed to extract only the affected users from the database. Policy Server Database Handler (PSDH) is the component in charge of managing the database (storing policies and registering users into the database), dissemination of the policies to the users and informing about policy changes. Policy Manager (PM) is the policy server component responsible of making the policy changes in the network. It has the option to extract the policies from the database, make the appropriate changes – add/change/delete policies, and reflect the changes back in the PSD. The PM accepts the policies coming from the policy administrators, translates them into policy language valid format and sends them to the PSDH. Policy Engine (PE). The PE is the policy decision point in the proposed policy architecture. It is located in the terminal and is responsible of reasoning on the set of active policies and presenting the reasoned result to the CRM. In order to be

Policy Handling Toolbox (PHT). The PHT is an integral part of the CRM which is the policy enforcing point in the architecture. The CRM is responsible for optimization, learning and decision making. The PHT creates and sends policy language specific requests to the PE. In the opposite direction, the policy replies are received and provided in CRM understandable fashion. Interfaces. As illustrated on Fig. 1, the policy architecture yields two key interfaces, the policy interface and the control channel interface. Policy interface is the interface supporting the local communication between the PE and the CRM. The messages exchanged via this interface are policy queries and replies, as well as policy change and emergency notifications. The policy architecture also considers the case when the terminal is not PE equipped. In this case, a remote reasoning option to exchange the policy interface related messages over the control channel and utilize the reasoning potential of a remote node is feasible. Control channel is handling the communications between the policy server and the PEs of the nodes, established in a secure and reliable manner. This interface is used for user registration, policy and notification exchanges. As previously noted, it is also used for policy interface messages exchange in case of remote reasoning. B. Policy Protocol and Interactions The policy architecture also includes the specification of a policy protocol, which defines the communication between the policy components via the defined policy and control channel interfaces. The messages defined with the policy protocol and the interactions between the entities engaged by the messages are described in this subsection. PolicyMsg is the message distributing the policies from policy server to the local PEs. This message is exchanged between PSDH and PEM through the control channel and can contain multiple policies. The PolicyMsg sending is initiated by a user registering process or a policy change situations. PolicyQuery is the message carrying the policy query (transmission request) in policy language specific format from the PHT to PEM, transmitted via policy interface. PolicyReply

978-1-4244-5637-6/10/$26.00 ©2010 IEEE


returns the reply from the PR as a result of the reasoning process caused by previously received PolicyQuery message. This message is sent in PEM to PHT direction over the policy interface. PolicyChangeMsg reports policy changes in the policy server that might require re-evaluation on the established links of the affected users. It triggers the terminals of interest to stop transmitting and request fresh policy conformance checking. It is exchanged through the control channel, between PSDH and PEM, and forwarded to PHT through the policy interface to notify the referred for the policy changes. EmergencyTrigger is a message that informs the policy server for possible emergency situations detected by the CRM. The message is transmitted in the direction PHT to PM (relayed by PEM and PSDH), and might impose the PM to change the policies. UserRegisterMsg serves for registration of users in the policy server. It carries information about user class and device type, as well as user identification and network address. It is sent from the PEM to the PSDH through the control channel. It is also used by the PM to extract policies from the policy server. NewPolicyMsg is the message used by the PM for uploading new policies or updating/deleting existing policies in the PSD. It is sent locally in the policy server, in the direction PM to PSDH. This message imposes sending of PolicyMsg’s to the users of interest to distribute the new/changed policies, and PolicyChangeMsg to notify about the changes.

permitted?” response from the reasoning process. This is important because it highly improves the conformance checking process and, as a result, the time required to converge to a permitted solution is minimized. The modular structure of the policy engine also offers the possibility to easily switch to a more efficient PR, if available. III.

POLICY ARCHITECTURE FUNCTIONALITIES

This section presents the key system functionalities describing several simple scenarios mapped into message sequence charts. User and/or terminal classification. The organization of the PSD provides the feature for policies classification and dissemination based on the users’ class and device type. Each terminal registers to the policy server at start up announcing its user class and device type. As a response, the respective policies are received from the policy server. Fig. 2 illustrates the registration process.

The described policy protocol messages and interactions introduce a negligible control traffic exchange in the network. C. Policy Language and Ontologies The policies are expressed using a specific policy language consisted of a set of clearly defined ontologies. Ontology language allows defining the meaning of terms in vocabularies and their relationships [7]. The proposed ontology language for expressing the policies in the policy architecture is CoRaL, since it is the only ontology language specifically designed for cognitive radios incorporating many useful radio related ontologies and being easily extendable. The CoRaL built-in ontologies along with additional ontology extensions make the following parameters available for expressing policies in the proposed policy framework: centerFrequency, bandwidth, meanEIRP (i.e. radiated power), rat (i.e. Radio Access Technology), application, priority, dayOfWeek and hourOfUsage. These parameters offer the possibility to express a variety of regulators’ and operators’ goals in terms of network and spectrum resources usage. Moreover, the set can be easily extended to support many new functionalities of the policy system. CoRaL specifies usage of 2 types of policies, i.e. allowing and disallowing policies. Allowing policy specifies conditions that must be satisfied in order to get approval by that policy, while disallowing policy specifies conditions that, if satisfied, will disallowed the request by the referred policy. In order to get transmission permission, the transmission parameters must satisfy the rules in at least one allowing policy, while not being disallowed by any of the disallowing ones. The PR used in the proposed policy framework is the XG Prolog PR with modified and extended functionalities. The crucial improvement to the XG PR is the support of “why not

Figure 2.

User registration and policy checking process

Efficient policy checking mechanism. The policy architecture has an efficient and flexible policy conformance checking. When the policy request is not permitted, a list of alternative solutions is formed utilizing the “why not permitted?” response, therefore avoiding significant time wasting checking process (Fig. 2).

Figure 3. Policy changes reporting and emergency situations triggering

Dynamic policy management. The proposed architecture offers framework for dynamic network resources management utilizing policies. When there is a policy change in the PSD, the changes are immediately distributed to the users (terminals) of interest, so the changes can be reflected in their behavior instantly (Fig. 3).

978-1-4244-5637-6/10/$26.00 ©2010 IEEE


Emergency Triggering. The architecture implements mechanism for reporting emergency situations to the PM (Fig. 3), so the referred can make appropriate policy changes in the PSD in order to recover and avoid that kind of situations in the future. Extensibility. The ontologies can be easily extended to support new functionalities of the system. Specific parameters coming from different levels of the protocol stack can be added in order to gain extended network management options [8]. Remote Reasoning. The policy architecture also offers the option to perform remote reasoning. This feature can be wisely utilized in scenarios that require separating the reasoning process from the users’ terminals and dedicating that task to a higher entity in the network, e.g. a base station or a point of attachment. User Preferences. This architecture provides the users with a feature to express their preferences and application priorities in forms of policies. These local policies can be directly input in the PE giving reasonable options for the users to manage their devices. IV.

POLICY ARCHITECTURE APPLICATIONS

The main purpose of the proposed policy architecture is to provide allowed solution space and constraints in the area of cognitive radio networks, so the CRM can perform decisions and optimization within the provided boundaries. A proof-ofconcept implementation of this application of the policy system is presented in the demo implementation in [9]. Additionally, the policy architecture can be also applied to handle a variety of resource management problems in the area of reconfigurable networks such as radio access control (including policies that manage the access to the network at different network conditions and assign specific user groups to specific access networks), load balancing (specifying policies that distribute users to the available networks avoiding network overload), traffic shaping (the supported ontologies provide options to support traffic shaping, deriving policies that assign specific application types to specific access networks), spectrum usage control (using policies specifying which portions of the spectrum are allowed under what conditions, distributing different user groups to different channel sets and shortening the search process), spectrum mobility management (the ontologies can be further extended to support the expression of policies that can indicate conditions under which the spectrum mobility should be performed), user and application prioritization (this feature can be supported by applying user and application prioritization policies which grant more resources to users and application with higher priority values) etc. A proof of functionality testing of the policy system spectrum management use case is presented in the following subsection. A. Functionality testing The simple demo setup targets throughout behavior and inter-channel handover analysis of 2 applications, i.e. RTP over UDP based streaming and FTP over TCP based file transfer, in an IEEE 802.11 based cognitive radio environment. The demo consists of a central policy server performing channel and

policy management for 2 USRP2 [10] enabled PCs aiming to establish communication. The central policy server is a desktop PC hosting policy server and policy engine functionalities. The USRP2 pair uses gmsk modulation and physical layer bitrate of 2 Mbps, while using basic CSMA-CA MAC layer functionalities. They are connected to the upper protocol layers through a virtual tun interface in the Linux platform. A local WLAN network is used for the control channel messages exchange. The demo behavior is observed on a spectrum analyzer that measures channel occupancy. The following steps explain the demo story: 1) The node pair obtains an allowed (channel) solution space from predefined policies (specifying certain rat and channel specific characteristics) from the policy server. 2) The source node forms a spectrum map (top-down power ranking of available channels) and chooses the best solution for connection establishment (in the demo the selected channel is WiFi channel 3, 2422MHz). Then, a RTP over UDP streaming is started. 3) After a certain amount of time, a policy that forbids the selected channel (channel 3) for the node pair is manually input in the PSD. 4) The policy change is propagated to the node pair enforcing reconfiguration of the USRP2 devices in order to change the channel and perform appropriate inter-channel handover. 5) New solution space is calculated at the PR and is passed to the USRP2s. This information is again combined with the new spectrum map formed by the source node to select the best channel solution. The chosen destination channel is channel 1 and results in seamless handover from channel 3 to channel 1.

Figure 4. RTP over UDP streaming throughput

The performances of interest are the system reaction time to policy changes as well as the application handover delay. Fig. 4 depicts the throughput of the streaming application while performing the channel switching. The inter-channel handover duration is around 1.5s taking into consideration all the actions performed during the channel switching. Around 200ms of this time are needed to exchange policy related messages and perform the reasoning process. Most of the time is wasted while switching between different modes of operation on the USRP2 (i.e. spectrum sensing mode, mode for connection

978-1-4244-5637-6/10/$26.00 ©2010 IEEE


establishment messages exchange and finally mode for communication). It should be noted that the system reaction time does not depend only on the reaction times on the policy components. It also depends on the actual hardware used in the implementation (USRP2 in this case), as well as the selected communication recovery mechanism. The same tests are performed with file transfer application type. Fig. 5 presents the results of the second test. Because of the nature of TCP, the application handover latency is higher (around 2s). In this scenario, the node pair is forced to make the handover in the opposite direction, i.e. from WiFi channel 1 to channel 3. Figures 6 and 7 depict the moments while performing the second channel handover (captured on a spectrum analyzer) proving the spectrum mobility functionality of the policy system.

V.

CONCLUSIONS AND FUTURE WORK

This paper presents a novel policy based reasoning architecture for resource management in cognitive radio environments. The main advantages of the proposed policy framework is its features to support dynamic policy management, the efficient and flexible policy engine, the ability to support different policies as well as the remote reasoning options. The policy architecture can be extended and adapted to solve different resource management issues [11] and a proof-of-concept implementation for the problem of spectrum usage management was conveyed and tested. The future work will focus on implementation of the proposed policy architecture in the other use cases and evaluation on the system performance in different and larger network scenarios. Future work will furthermore include a conflict resolution mechanism for case of multiple policy sources. The presented work in this paper is also open to some structural modification in order to adapt to ongoing standardization activities (e.g. IEEE P1900.4 [12]). ACKNOWLEDGMENT Parts of this work were funded by the EC through the FP7 projects ARAGORN (216856) [13] and FARAMIR (248351) [14]. The authors would like to thank everyone involved. REFERENCES [1]

Figure 5. FTP over TCP file transfer throughput [2]

[3] [4]

[5] [6] [7] [8] Figure 6. Stopping communication on WiFi channel 1 (2412MHz) and starting on channel 3 (2422MHz) [9]

[10] [11]

[12] [13] [14]

J. Mitola III,”Cognitive Radio An Integrated Agent Architecture for Software Defined Radio,” PhD thesis, KTH Royal Institute of Technology, Stockholm, Sweden, 2000. Y. Zhao et al., “Radio Environment Map Enabled Situations-Aware Cognitive Radio Learning Algorithms”, SDR06 Technical Conference and Product Exposition, SDR Forum, 2006 DARPA XG Working Group, ‘‘The XG Vision. Request for Comments, BBN Technologies, Cambridge, MA,’’ July 2003. D. Elenius et al., “CoRaL–Policy Language and Reasoning Techniques for Spectrum Policies,” 8th IEEE International Workshop on Policies for Distributed Systems and Networks, Washington, DC, USA, 2007. XG Prolog Policy Engine. Available at: http://xg.csl.sri.com/prolog.php. Maude XG Policy Reasoner. Available at: http://xg.csl.sri.com/maude.php B. Chandrasekaran et al., “What are ontologies, and why do we need them?”, IEEE Intelligent Systems, 14(1), January 1999. J. Riihijärvi, M. Petrova, V. Atanasovski and L. Gavrilovska, “Extending Policy Languages with Utility and Prioritization Knowledge: The CAPRI Approach,” IEEE DySPAN 2010, Singapore, April 2010. (accepted) V. Atanasovski, D. Denkovski, T. Farnham, L. Gavrilovska, A. Gefflaut, P. Mahonen, V. Pavlovska et al., “Cognitive Radio for Home Networking,” IEEE DySPAN 2010 demonstration, Singapore, April 2010. (accepted) Universal Software Radio Peripheral 2 (USRP2). Information available at: www.ettus.com. L. Gavrilovska and V. Atanasovski, “Resource Management in Wireless Heterogeneous Networks (WHNs),” IEEE TELSIKS2009, Nis, Serbia, October 2009. IEEE P1900.4. Information available at: www.scc41.org. EC FP7 project ARAGORN. Information available at: http://www.ictaragorn.eu. EC FP7 project FARAMIR, Information available at: http://www.ictfaramir.eu.

Figure 7. USRP2 communication continued on WiFi channel 3

978-1-4244-5637-6/10/$26.00 ©2010 IEEE