Embedded SDR System Design Case Study: An Implementation ...

Embedded SDR System Design Case Study: An Implementation Perspective Almohanad S. Fayez, Nicholas J. Kaminski, Alexander R. Young, Charles W. Bostian Bradley Department of Electrical and Computer Engineering Wireless@VT Virgina Tech, Blacksburg, VA 24061 {afayez, njkamin, alex.young, bostian}@vt.edu

Abstract—Cognitive Radios (CRs) and Software Defined Radios (SDRs) have ubiquitous applications ranging from handheld to base station devices. In order to meet the computational requirements of such radios, computing heterogeneity, the mixed usage of General Purpose Processors (GPPs), Digital Signal Processors (DSPs), and Field-Programmable Gate Arrays (FPGAs), is attractive. Developing SDR and CR applications already requires a diverse set of skills, and computing heterogeneity further complicates the process. This paper presents a developmental workflow used successfully by the authors for SDR and CR application running on a platform combining DSP and GPP based processors. The paper discusses tools used to set up the platform, create compilation environment, develop code for GPP/DSP communication, integrate the DSP into GNU Radio, and use the environment to develop SDR/CR applications. It presents a case study showing how computing heterogeneity can be used to address diverse application needs. Keywords-embedded sdr; system design; software defined radio; cognitive radio

I. I NTRODUCTION SDR has a wide domain, varying from handsets to base stations. In accommodating the processing needs of this vast application space, computational heterogeneity, the mixed use of GPPs, FPGAs, DSPs, is necessary. Development is further complicated by having to address issues like platform set up, multi toolset development, inter-processor communication, and realizing functional prototypes. This paper presents a case study of developing SDR and CR applications in such computing environments focusing on GPP/DSP processors and discusses some of the challenges and issues from an implementation perspective. We started investigating the implementation of SDR/CR on GPP/DSP processors in an effort to migrate the Virginia Tech Public Safety Cognitive Radio (PSCR) [1], a CR designed to address radio interoperability issues for public safety applications developed in our lab, from a GPP laptopbased implementation to an embedded platform. This paper discusses some existing technologies used in addition to new work developed. The paper is organized as follows. The first section discusses the platform selection process summarizing the general categories of platforms. The second section disc 978-1-4673-1239-4/12/$31.00 2012 IEEE

cusses the tools and software environments available to target GPP/DSP based SDR platforms, focusing on the Texas Instrument (TI) OMAP3530 processor. The third section discusses some aspects of developing and constructing applications on such platforms. II. P LATFORM S ELECTION An SDR depends on software to realize radio functionality and relies on some supporting hardware for transmitting and receiving the physical radio signals. The general components of a platform are as follows, and are summarized in Figure 1: 1) Programmable Radio Front-End: This contains analog RF components which have tunable RF parameters, an Analog-to-Digital Converter (ADC) to digitize analog RF signals, Digital-toAnalog Converter (DAC) to generate an RF signal from digital data, and a device to perform decimation and downconversion for receivers and interpolation and upconversion for transmitters, typically an FPGA. Manufacturers of such front-ends includes Ettus Reseach [2] and Lyrtech [3]. 2) Computational Platform: The computational element of the platform can include GPPs, DSPs, and/or FPGAs. The computational platform can be: a) homogeneous - composed of a single computational device or multiple computational devices of the same type e.g., one or more GPPs b) heterogeneous - composed of different computational devices, e.g., some mixture of GPP, DSP, and FPGA. 3) Integrated SDR Platform: An integrated SDR platform combines the computational elements and the programmable RF front-end into a single platform. Examples of such platforms includes the Universal Software Radio Peripheral (USRP) E100 [2], Lyrtech SFF SDR [3], and the Rice University WARP board [4]. An SDR platform can be constructed by combining a programmable RF front-end with a computational platform

Figure 1: Block Diagram of SDR System Breakdown. [5]

the target processor and platform devices, enabling it to build the necessary drivers. • Operating System: Probably some Linux based Distro. • Compiler: OE uses the GNU toolchain for cross compilation, and the user is able to specify which version to use. In developing code for the C64x+ DSP, the TI C6000 Code Generation Tools [9] DSP compiler was used. In addition, the DSP OS used was TI DSP BIOS, a lightweight real time library that provides basic run-time functionalities including thread scheduling, handling I/O, and capturing real-time information [10].

via an appropriate interface, as discussed in [6]. In selecting an SDR platform for the work in [7] and [5], we chose a first generation USRP as a programmable front-end and a Beagleboard [8] as the computational platform. They were connected via USB. The Beagleboard features a TI OMAP3530 processor which is composed of an ARMv7 GPP core and a TI C64x+ fixed point DSP. Eventually this platform was replaced with a USRP E100 which combines the programmable RF front-end and the computational platform in a single integrated system connected via the processor’s bus instead of a USB peripheral interface. III. T OOLS AND S OFTWARE E NVIRONMENTS The software needed to construct SDR applications on an embedded platform can be divided into three main categories. 1) Embedded Development Framework: The framework includes the development tools necessary to set up and compile applications. For example, this would include cross-compilation and generating a filesystem for a GPPs or a DSPs, or synthesis tools for FPGAs. 2) SDR Software Toolkit: The toolkit would include basic blocks to be used for constructing SDR and CR applications. 3) Processor Interface: SDR platforms which contain multiple computational devices require a mean for computational devices to communicate and transfer information amongst themselves. A. Embedded Development Framework The embedded framework used in [5] and [7] and discussed here is OpenEmbedded (OE), which provides a set of instructions or recipes, interpreted and executed through Bitbake and the necessary tools to generate final binaries and a filesystem to run on the ARMv7. The OE environment is summarized in Figure 2. After selecting a platform to be used, developers would need to set up the OE environment by specifying: • Embedded Platform, or Machine: This selection specifies developer’s platform and, as a result, OE knows

Figure 2: Description of the OE framework. [5]

B. SDR Software Toolkit While an application developer can choose to implement all application functionalities, it can be of great benefit to leverage an existing SDR software toolkit such as GNU Radio [11] and OSSIE [12]. Ultimately GNU Radio was used and mechanisms to integrate the C64x+ DSP were added to GNU Radio in addition to C64x+ based blocks. C. Processor Interface The main existing SDR software toolkits, GNU Radio and OSSIE, mostly target GPP based SDR and CR application development. The Software Communications Architecture (SCA), on which OSSIE is based, attempted to extend its standard to cover DSP and FPGA development by defining the Modem Hardware Abstraction Layer (MHAL), a layer targeted to enable compatibility and reuse of DSP and FPGA based components. However, MHAL became International Traffic in Arms Regulation (ITAR) restricted which makes it largely inaccessible to the development community. Looking into technologies such as Common Object Request Broker Architecture (CORBA), which is a component of the SCA architecture, to integrate GPP and DSP components without considering the associated overhead might be disadvantageous. In [13], the authors explored integrating an FPGA

based FIR filter with a GPP using CORBA and found that the overhead is potentially too large to justify. In terms of the OMAP3530 processor there are existing mechanisms and tools to enable cross GPP/DSP communication. These tools provide mechanisms for basic processor control, data transfer, memory sharing/synchronization, messaging, among other functionalities. In terms of TI GPP/DSP OMAP processors, there are three inter-processor communication tools available: • Codec Engine [14]: The Codec Engine relies on a server/client model for running DSP applications (or codecs) which adhere to the TI XDAIS standard as part of the Digital Video Software Development Kit (DVSDK) kit. The developer is able to instantiate and allocate system resources for DSP programs. Figure 3 summarizes this programming approach.

Figure 5: SysLink Multi-Core Communication Interface.

Syslink was not available when the work first started but it is worth investigating seeing that it can accommodate communication in a multi-core environment which may not have a shared memory interface. IV. D EVELOPMENT E NVIRONMENT The development environment for the GPP/DSP OMAP can be broken into two main domains: platform development and application development. The platform development domain deals with developing the necessary GPP/DSP communication support, DSP modules, and the interfaces for GNU Radio. While the application domain includes the physical applications development which utilizes the physical platform and the developed software support.

Figure 3: Codec Engine Programming Approach. [5] •

DSPLink [15]: Provides processor control and data transfer capabilities over the shared memory region between the GPP and DSP core. Figure 4 illustrates the communication interface.

A. Platform Development Figure 6 illustrates the overall platform development and interaction between components. Following is an explanation of the workflow from a bottom-up perspective.

Figure 4: DSPLink Processor Communication. [5] SysLink [16]: Provides processor control and data transfer over multi-core architectures with different combinations of GPP and DSP cores. SysLink is not limited to a shared memory interface but can accommodate different interconnection between the cores, e.g., bus connection. Figure 5 illustrates the SysLink interface. When the work in [7] and [5] first started, DSPLink was chosen as the backbone of the GPP/DSP interface. Codec Engine provides a client/server model, where DSP code would need to be written according to a TI standard and run within their server framework. While DSPLink provides data structure components running directly on the shared memory region between the GPP and DSP allowing developers to have more control over the GPP/DSP communication. •

Figure 6: Platform Development Workflow. [5] 1) DSP Program: The DSP program is a Finite State Machine (FSM) which waits to set up and service DSP processing block requests. Thus, it will receive messages from the GPP requesting blocks such as FIR filters blocks including necessary parameters such as filter coefficients in request messages. After a block is set up on the DSP, the GPP starts sending data to be processed using the configured block. Each GPP/DSP message has the following fields:

Block Type: Indicates if this packet is to set up a FIR, modulation, demodulation block, ... etc. Or it can be a data processing packet, where the block has already been set up and now the DSP needs to process the data. • Block ID: A unique identifier for a requested block. • Scaling Factor: The scaling factor used for the floating/fixed point conversion. • Interpolation Factor: Indicates user requested interpolation factor. • Decimation Factor: Indicates user requested decimation factor. • Payload Size: Indicates the payload size in a message. • Payload: The payload can be filter taps, for a FIR filter set up message, or IQ data for a data processing packet. Figure 7 illustrate the message format with the associated fields. •

Figure 7: GPP/DSP Message Format. [7] 2) easyCom Library: easyCom library was developed to allow developers to perform GPP and DSP interaction without having to deal with the specifics of the interprocessor communication. It was developed as a layer on top of DSPLink while hiding the details of the underlying function calls. This also enables platform developers to change the underlying inter-processor communication tools used without having to modify the applications running on the platform, e.g., switching from DSPLink to SysLink would be hidden from applications using the easyCom library. The following is a list of the function calls available in the easyCom library: • Initialize DSP: Initializes the DSP and loads a specified DSP executable. • Transmit data: Transmits data from the GPP to the DSP and it supports both complex and real data formats. • Receive data: Receives data in the GPP from the DSP and it supports both complex and real data formats. • Clear DSP: Deallocates DSP and processor interface resources. 3) Communication Overhead: Using the DSP as a coprocessor incurs an overhead. It is important to be able to measure and quantify the associated overhead to know if the system performance would actually improve, degrade, or remain the same. The flowgraph shown in Figure 8 is used to measure the overhead associated with using the DSP. The flowgraph inputs IQ data from a file, filters the data using a DSP based Finite Impulse Response (FIR) filter, and saves the final data into a file. There are additional functions to accommodate DSP processing, namely scaling which converts data between floating-point (on the GPP)

and fixed-point (on the DSP), and copy buffer to transfer data between the GPP and DSP. The input IQ data is varied from 1.6 × 103 to 16 × 106 samples as discussed in [7]. The following scenarios are used to measure the overhead associated with using the DSP.

Figure 8: GPP/DSP Message Format. Input/output time measurements: The time associated with reading IQ data from an input file and writing final data into an output file. The scenario measures overhead shared between a GPP implementation and a mixed GPP/DSP implementation. • DSP loopback using fixed-point IQ data: Fixedpoint data is sent to the DSP and read back without performing any computation. By transferring fixedpoint data, the flowgraph measures the time associated with tranferring data between the GPP/DSP only. • DSP loopback using floating-point IQ data: Floatingpoint data is sent to the DSP and read back without performing any computation. By transferring floatingpoint data, the flowgraph measures the time associated with converting data to and from fixed-point format and the time with transferring data between the GPP/DSP. • FIR implementation: Measures execution time for GPP-based and DSP-based FIR filter implementation. The timing measurements for the various scenarios is summarized in Table I and Table II. •

Table I: Summary of IQ data overhead in the flow graphs and DSP overhead for buffer copies and data format conversion. Input (16-bit IQ pairs) 1.6 × 103 16 × 103 160 × 103 1.6 × 106 16 × 106

Flowgraph setup IQ data I/O (seconds) 1.718 1.713 1.782 3.159 22.987

Loopback without conversion (seconds) 2.11 2.153 2.361 5.085 37.919

Loopback with Conversion (seconds) 2.128 2.151 2.487 5.735 42.11

4) GNU Radio Blocks: GNU Radio blocks are written as discussed in [5] to interface with the DSP for OMAP processor. The blocks essentially generate messages and send them to the DSP requesting specific DSP blocks, e.g., FIR filters, modulation, demodulation, ... etc, according to the message format discussed earlier. After these blocks are set up in the DSP, the GNU Radio blocks would

Table II: Summary of GPP/DSP based FIR filter execution times and DSP implementation speedup factor. Input (16-bit IQ pairs) 1.6 × 103 16 × 103 160 × 103 1.6 × 106 16 × 106

GPP (seconds) 1.916 3.63 20.519 189.308 1876.937

DSP (seconds) 2.109 2.19 2.494 6.237 46.837

Speedup 0.91 1.66 8.23 30.35 40.07

(a) Bare Platform

(b) Developer Platform Interaction

Figure 10: Platform View subsequently transmit data to the DSP for processing and the DSP is able to associate the data with their respective DSP blocks via the unique block ID and forward the processed data back to the GPP. The blocks are implemented in C++ and are ultimately invoked in Python via the Simplified Wrapper and Interface Generator (SWIG) interface between the C++ implementation and the Python function calls. Figure 9 illustrates how C64x+ DSP blocks look from a GNU Radio user perspective.

Figure 9: DSP Block Abstraction from GNU Radio Perspective. [7] The work discussed in this section: the DSP program, easyCom library, and GNU Radio C64x+ blocks are available for download at [17]. B. Application Development For the purpose of application development, the platform as presented above appears as a black box to the application developer (see 10a). An API defines standardized interfaces to the platform, while hiding the lower level details. Using the methods that the API makes available in the application space, the developer can leverage the full capabilities of the platform without worrying about the lower level interactions and interdependencies. This means that a developer can take a pre-built application, drop it onto the embedded platform, and only have to make minor adjustments (see 10b). This smoothes the transition from GPP based systems to embedded platforms. The approach to embedded platform development presented here provides an excellent foundation for a number of applications. The strength of the approach is the flexibility

to utilize embedded platforms while maintaining abstraction between the lower level details and the application developer’s concerns. While the SDR system is an embedded platform, application developers still benefit from GPP development on the OMAP. With a standard Linux operating system installed, developers can leverage all the tools available in standard GPP-based development. This includes high level programming languages, debuggers, and customizable integrated development environments. Additionally, developers can leverage file I/O and extensive data logging and analysis. All these combine to enable rapid application development and deployment. The application development environment laid out as described was a benefit in our SDR application development efforts. Our application was originally written using Python on a high performance laptop. Python was chosen as flexible language ideally suited to rapid application development, easy to learn and use yet extremely powerful. Because of the design of the development environment with the platform black box and API, we were able to successfully transition our application from high performance laptop to embedded platform with minimal changes. V. C ONCLUSION The paper presents a case study of implementing SDR and CR applications using embedded heterogeneous GPP/DSP based processors. It includes an overview of a development workflow used by the authors and discusses some of the tools and software which aid in realizing such applications on actual GPP/DSP platforms. ACKNOWLEDGMENT This project is supported by Award No. 2009-SQ-B9K011 awarded by the National Institute of Justice, Office of Justice Programs, US Department of Justice. The opinions, findings, and conclusions or recommendations expressed are those of the author and do not necessarily reflect the views of the Department of Justice.

This material is based on research sponsored by the Air Force Research Laboratory under Agreement #FA8750-092-0128. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of Air Force Research Laboratory or the U.S. Government. Almohanad Fayez was supported by the Saudi Arabian Cultural Mission (SACM). Nicholas Kaminski was supported by a Bradley Fellowship through the Bradley Department of Electrical and Computer Engineering. R EFERENCES [1] R. Rangnekar, F. Ge, A. Young, M. Silvius, A. Fayez, and C. Bostian, “A remote control and service access scheme for a vehicular public safety cognitive radio,” in Vehicular Technology Conference Fall (VTC 2009-Fall), 2009 IEEE 70th, 2009, pp. 1 –5. [2] “Universal software radio peripheral.” [Online]. Available: http://www.ettus.com [3] Lyrtech. [Online]. Available: www.lyrtech.com [4] R. University, “Rice warp board.” [Online]. Available: http://warp.rice.edu/trac/ [5] A. Fayez, “Designing a Software Defined Radio to Run on a Heterogeneous Processor,” M.S. Thesis, Virginia Polytechnic Institute and State University, 2011. [6] C. Anderson, G. Schaertl, and P. Balister, “A low-cost embedded sdr solution for prototyping and experimentation,” SDR Technical Conference and Product Exposition, December 2009. [7] A. Fayez, Q. Chen, J. Nounagnon, and C. Bostian, “Leveraging embedded heterogeneous processors for software defined radio applications,” Wireless Innovation Forum Conference and Product Exposition, December 2010. [8] Beagleboard, Beagleboard System Reference Manual Rev C4, Beagleboard, December 2009. [9] “C6000 code generation tools,” Texas Instrument. [10] “Tms320c6000 dsp/bios user’s guide,” Texas Instrument. [11] GNU Radio. [Online]. http://gnuradio.org/redmine/wiki/gnuradio

Available:

[12] MPRG, “Ossie.” http://ossie.wireless.vt.edu/

Available:

[Online].

[13] M. Carrick, S. Sayed, C. Dietrick, and J. Reed, “Integration of fpgas into sdr via memory-mapped i/o,” SDR Technical Conference and Product Exposition, December 2009. [14] “Codec engine application developer user’s guide,” Texas Instrument.

[15] “Dsp/bios link omap3530 evm,” Texas Instrument. [16] “Syslink user guide,” Texas Instrument. [Online]. Available: http://processors.wiki.ti.com/index.php/SysLink UserGuide#Introduction [17] A. Fayez. [Online]. Available: www.github.com/alfayez