Software Framework for Software Radio - FlexNets

Projecte Fi de Carrera

A Software Framework for Software Radio

Radio Communications Research Group

Autor: Ismael Gómez Miguelez Director: Antoni Gelonch Bosch January, 2008

Table of Contents Table of Contents.............................................................................................................5 Index of Figures............................................................................................................... 6 Index of Tables.................................................................................................................9 Introduction................................................................................................................... 11 State of the Art. Enabling Technologies...................................................................... 15 The P-HAL Framework................................................................................................19 P-HAL Implementation Details....................................................................................39 User Guides.................................................................................................................. 112 P-HAL Real Implementations Results.......................................................................152 Current P-HAL Challenges........................................................................................ 164 Conclusions...................................................................................................................167 Future Work................................................................................................................ 169 References.....................................................................................................................170

Index of Figures

Figure 2-1 – Heterogeneous NoC.................................................................................17 Figure 3-2 – P-HAL Layers Schematic........................................................................20 Figure 3-3 – Relevant Parts of a P-HAL Environment..............................................21 Figure 3-4 – The HAL in a Software Stack.................................................................24 Figure 3-5 - P-HAL Components Diagram.................................................................26 Figure 3-6 – Platform Slot Time References...............................................................34 Figure 3-7 – Different Processors use different Slot durations.................................35 Figure 3-8 – Platform Synchronization Hierarchy.....................................................35 Figure 4-9 – Concatenation of Headers.......................................................................59 Figure 4-10 – External Header.....................................................................................60 Figure 4-11 – Control Header.......................................................................................60 Figure 4-12 – Data Header............................................................................................63 Figure 4-13 – Daemon A sending to Daemon B (upwards and downwards)...........65 Figure 4-14 – Hierarchical Resource Identification...................................................68 Figure 4-15 – Hierarchical Software Mapping...........................................................69 Figure 5-16 – Control and Data Plane in the same P-HAL Environment..............115 Figure 5-17 – Daemon Placement in a P-HAL Environment (Example 1)............117 Figure 5-18 – Daemon Placement in a P-HAL Environment (Example 2)............118 Figure 5-19 – Hierarchical Resource Identification.................................................119 Figure 5-20 – Hierarchical Software Mapping.........................................................120 Figure 5-21 – Example Synchronization Network....................................................121 Figure 5-22 – SW daemon example code...................................................................125 Figure 5-23 – UMTS Receiver Object Oriented P-HAL Application.....................140

Figure 5-24 – P-HAL Object Flow Diagram.............................................................143 Figure 5-25 – Schematic source code for a P-HAL Object......................................145 Figure 6-26 – Overhead vs Time Slot duration in PC Linux...................................156 Figure 6-27 – Overhead vs Time Slot duration in Ti DSP.......................................159 Figure 6-28 – GMSK Application Chain...................................................................161 Figure 6-29 – Output of the Shaping Filter (time)....................................................161 Figure 6-30 – Output of the Shaping Filter (frequency)..........................................161 Figure 6-31 – Output of the NCO (frequency)..........................................................162 Figure 6-32 – QPSK Application Chain....................................................................163 Figure 6-33 – Output of the Shaping Filter (time)....................................................164 Figure 6-34 – Output of the Shaping Filter (frequency)..........................................164 Figure 6-35 – Output of the Quadrature Modulator (frequency)...........................164

Index of Tables

Table 3-1: List of specific functions that are provided by P-HAL Hardware Library...............................................................................................................................27 Table 3-2: List of functions that are provided by P-HAL Software Library API...33 Table 4-3: P-HAL External Header.............................................................................60 Table 4-4: P-HAL Control Header..............................................................................61 Table 4-5: P-HAL Control Header TYPE and CMD Field Values .........................61 Table 4-6: Executable Loading Modes........................................................................85 Table 4-7: P-HAL Global Constants for Commands...............................................109 Table 4-8: P-HAL Constants Values for Packets ....................................................110 Table 5-9: P-HAL Software Requestor-Server inter-Daemons Relationship........116 Table 5-10: P-HAL Software Commander-Executer inter-Daemons Relationship .......................................................................................................................................116 Table 5-11: External Interface Ids Specifications....................................................138 Table 6-12: Linux Platform Performance Results....................................................155 Table 6-13: Ti DSP Platform Results.........................................................................158

Introduction During last big inflation in IT world, in the period of 1998-2003, the market experienced the emergency of new technologies in all its fields. We say in all fields because we see, not only how the introduction of new standards or new advanced algorithms (high spectrum efficiency modulations, near Shannon performance codes, large MIMO architectures, etc.) are flooding the market. Also new personalized processing devices (application specific processors, very large FPGAs with embedded microprocessors arrays and analogue functions, etc) are appearing, capable of processing high amounts of data but embedding different and heterogeneous architectures. If we put together these two premises, the extracted consequence is that designers are finding a big confusion when they face the problem of implementing a multi-network gadget in a multi-platform environment. This difficultness might turn, in the near future, in a lack of profit for designers or manufacturers. Today, users want to communicate at any time using any kind of RAT (Radio Access Technology) with the same device, maintaining always the contracted QoS (Quality of Service); users do not want one device to access Internet, another to make outdoor calls, another for indoor’s, another to receive GPS signals, etc. To solve this problem, today devices embed several communication hardware elements (one for every RAT) in the same gadget and they are switched depending on user’s requirements. If a new technology appears, a user has to buy a new gadget because the old one is unable to access such RAT: it has not been defined at design time for this purpose neither any mechanism enabling it to reconfigure itself has been designed. We will try to solve these and other problems in this project.

1.1. The Software Radio Concept The Software Radio concept was first introduced by J. Mitola 1.27.2 as “a radio whose channel modulation waveforms are defined by software”. Traditionally in communications devices, most of signal processing operations were done using hardware; Software Radio tries to move as many processes as possible to software world. Software is understood as a set of instructions loaded in a programmable processor (program) which are read and executed one after another. The concatenation of all of those instructions produces the expected processing function (algorithm). This concept has been well proven during last century thanks to the great evolution in computing machines: any kind of function can be implemented as a concatenation of small functions (instructions). Now the idea is to introduce the same concept in telecommunications devices. However, and due to current limitation in processing power, Software Radio also includes hardware non-programmable but reconfigurable devices (digital or analogue). Implementing a Software radio concept following these guidelines would provide the following example features: • Reconfiguration “on-the-fly”: the same device can act as a cordless phone, a mobile phone, a WiFi adaptor, a GPS receiver, etc. The terminal can switch between one service to another (or even use more than one at once) during its normal execution.

• Quick and easy upgrade to new technologies or enhanced features. This upgrade could be even done over-the-air. • Smart or cognitive radio that adapt the radio environment (background operator network and user terminal) to instantaneous population distribution and QoS requirements. From manufacturer point of view, more advantages appear when a new device development time including a new functionality can be drastically reduced if the company efficiently designs its processes to allow that software or hardware pieces can be reused. This would reduce not only the time-to-market time but consequently the overall cost of the device. At the end, several devices could be sold with different processing power ranges and different features but using the same base design for all of them, and, more interesting, reusing some of their parts (hardware and software). Finally, we should also be concerned about the benefits that such technology should bring in terms of environmental issues. Reusing hardware elements or the possibility to upgrade some device capabilities by terms of attaching another device (but keeping the old one) should contribute to reduce the overall refuses produced by the developed world as, nowadays, huge amounts of old devices are currently being wasted when they are not useful anymore.

1.2. Software Radio Challenges Once Software Radio model is accepted as the first step on providing the previously stated features, we easily realize how it will not be an easy challenge. Software Radio is a technology; is a guideline on how communications systems should be designed if we want to obtain a set of benefits. Its implementation, however, is not a direct step. First of all, the designer will try to find a platform where the communications system (implemented via Software Radio) is going to be executed. The designer will find a great diversity of platform families available in today’s market: several companies manufacture different DSP, different FPGA, different microprocessors (GPP), etc. The designer will have to choose one of these platforms. When a new technology appear or a new feature is desired to be included, it might occur that the selected device is not powerful enough to process a determinate algorithm or a required functionality is not available. The designer will find the problem of selecting another device and as a consequence rewrite some pieces of code, or in most of the cases, the entire code. The Software Radio concept is not efficiently implemented. One of its main objectives, the ability of reusing code and the flexibility is lost as soon as the platform changes. The reason why moving code from one platform to another is so difficult is because of the strong dependence existing between platform architecture and radio applications. These applications require a low-level access to platform resources as timers (real time control is very important in this applications), Codecs, memory controllers, interfaces configuration and interaction, etc. and these interaction is very architecture dependant.

different processors families including several FPGAs, DSPs, PowerPCs, Codecs, analogue devices, etc. can be easily found in today market devices. Such devices are presented to the designer as an accumulation of the same problem: lack of flexibility. Code is not portable; this is, it is difficult to move from one platform to another. Also when we think in what is the common structure, these radio applications share, we easily find out how most of them share some algorithms or pieces; at most, they change a few parameters or variables but they maintain the main structure of the algorithm (for example, UMTS and WLAN standards use both CRCs and Convolutional Codes although their parameters are not the same). We could take advantage of this issue and develop applications reusing some parts of other ones; that means, to implement a concept similar to Object Oriented Programming used in common computing platforms but in signal processing applications. This would drastically reduce the time-to-market of new devices as developer could just link some of the pre-written objects and configure them to match the current application. In the near future, this could not be a feature anymore and become a must. We will begin by drawing down some challenges Software Defined Radios are currently facing up: • Most of the RATs share some algorithms although their parameters are different, a tool enabling the possibility to reuse software pieces is necessary. • Developing or maintaining several of these applications would require of organizations in order to maintain common algorithms unified. • Large matrixes of embedded and heterogeneous chips (NoC) require a tool to homogeneously manage them. • Signal processing applications require a close contact with hardware devices. As a consequence, it is difficult to move code from one platform to another. These and other problems lead to a conclusion: we’ve got the technology, now we need an efficient implementation of it. We need to establish a common design policy surrounded by a background framework where homogeneous solutions for these problems are established or at least a future solution is enabled.

State of the Art. Enabling Technologies. In the last chapter, the advantages provided by Software Radio have been mentioned. This include the possibility to offer personalized and improved radio services, quick standard evolution by means of open standards that could be updated when new market needs or new solutions emerge, adaptation of the user access to the network depending on instantaneous distribution of users (Smart Radio), global roaming, etc. But to implement these functionalities in a real environment, the existence of a framework and a design methodology which allows the complete deployment of Software based Radios, turns necessary. This framework would include, at least, a mechanism to specify which software should be executed on the radio terminal as well as a way to make them available for it, and another one to ensure a secure behaviour of the software, that is, that the radio is working maintaining the standard constraints and respecting the limits agreed between the user and provider contract. A third mechanism would include a network intelligent entity, either centralized or distributed among several nodes, which will determine the when, why, what and how questions (W3HSR) in a reconfigurable Software Radio environment. For example, when will the terminal be reconfigured, how it will be reconfigured and what parts of it have to be configured.

1.3. Platform Homogenizing Mechanisms The problem of homogenizing several different Platforms responds to the following answer: “What happens if an algorithm must be described totally independently of the hardware platform?” That includes, number and type of processors, resources, digital or analogue interfaces, etc. Nowadays, however, portability is limited among a set of platforms with certain common characteristics (usually devices from the same manufacturer and product family). Considering a generic program written using any kind of programming language able to be used over multiple platforms, turns evident that the code should not include any particular instruction not understood by all platforms. This code would be, thus, written using a high level language and then converted into something executable by any platform. This process would require the use of some tool adapted to the platform (a compiler). However, not only the use of a high level language is a requirement. As programs need to access some physic resources (keyboard, hard disk, data interface, analogue devices, timers, etc.) that are very platform dependant, some kind of abstraction mechanism must exist. Several architectures have been proposed over last years to solve this kind of problem. In general purpose computers, famous standards emerged like UNIX/POSIX systems, object oriented architectures like CORBA, higher level abstractions like Virtual Machines, Grid computing frameworks, etc. In signal processing applications, the most advanced project, military, is the JTRS-SCA (Joint Tactical Radio Systems, Software 15

Communications Architecture) based in CORBA 1.27.21.27.2. It’s easy to find out, however, how this inherence from general purpose processing machines is not a good policy as signal processing applications requires an strict time control (Real Time) as well as a minimum overhead introduction. Thus, a new framework must be built specially designed for this kind of applications.

1.4. The Problem of Radio Reconfiguration The first two questions (‘when’ and ‘why’) receive an answer from the status of jointly managed, different wireless access networks. Independently of the complexity of algorithms used to decide whether a terminal should be reconfigured or not, the only issue to consider in the context of this project is the possibility that management elements may have access to contextual information provided by each terminal. The remaining two questions (‘what’ and ‘how’) may be more difficult to answer. Every terminal has its particularities that make it difficult to homogenise the reconfiguration procedure. As a general approach, software may come from different sources, not being the same for different terminals. A possible reason is that software optimisation makes optimised products that are potentially better than competitor’s products. Then it is likely that optimised algorithms are not public to avoid competitors reaching the same performance level. This single reason already advocates for different software solutions on terminals, although others are possible. There are other issues related to software that are not discussed in this project, like software security guarantee, download mechanisms, user-operator contractual bindings’ agreement, etc. 1.27.2. At hardware level, there exist a multitude of different mobile terminals. Such a terminals variety would be, by itself, a source of heterogeneity for the execution context of Software Radio. In base stations, an “unlimited” number of hardware combinations are already possible employing embedded processing cards from different manufacturers. Going beyond, why not consider the possibility to have portable terminals with “variable” plug-and-play hardware? Let’s imagine a user selecting the RF mini-card from manufacturer A because of its high sensibility, dual channel operation, and maximum transmitted power, among others. The IF mini-card may be from manufacturer B because of number of simultaneous channels that it may capture, computational performance, power consumption and so on. Finally, base band (BB) mini-card selected from manufacturer C because of power consumption versus bit rate, weight and price. All these cards can be inserted within a chassis including a general purpose processor running an operating system and interfacing the keyboard, screen, camera or any other terminal peripheral. Over such a heterogeneous platform the software radio application must be executed, not only involving the general-purpose processor but above all the processors within RF, IF and BB sections. Hence, in any radio interface endpoint there is a large amount of heterogeneity. The previous paragraphs introduce the problem of providing a safe, unified, regulated, environment for the reconfiguration of software radio terminals with heterogeneous hardware and probably non-public heterogeneous software solutions. Such environment would allow taking the full advantage of software radio concept to the network management entity, users, hardware manufacturers and software programmers. 16

1.5. Managing Complex Processor Networks (NoC) Moreover of doing the exercise of homogenize the platform architectures, one of the current challenges in complex Network on Chip devices (NoC) is how to deal with such networks. Again, a particular solution for one NoC is not an optimum solution as it won’t enable to be extended to other manufacturers’ chips. Today market chips may include an array of several different devices (Figure 2-1): FPGA, DSP, microcontrollers, PowerPC, analogue functions, etc. So, next step after solving the problem of homogenizing platform architectures is to define a framework where such devices can be threaten as a single device, where applications are loaded and routing and delay management is transparently managed by the device itself. If we go even further, one could build a platform made not only of one complex NoC device: several of these chips could be attached one after another turning the routing of data (or control) problem into something very difficult to solve.

Figure 2-1 – Heterogeneous NoC

1.6. The Execution Environment The Execution Environment concept of SDR systems presents several particularities that must be clearly identified before proceed to design the whole system. It must be 17

seen as the environment where all the objects of an SDR application find the proper functionalities and resources to carry out the tasks that were designed for. It includes all the requirements with respect to the processing data flow but also the related with the control of the execution which in SDR systems is highly linked with the computing resources management. Flexibility, as a property inherent to SDR systems, must be strongly enforced by the Execution Environment. Wireless in SDR systems reveal specific aspects, essentially regarding flexibility and efficiency which have not been jointly considered so far in heterogeneous computing. Such particularities includes • Time slot based division of the transmission medium (radio time slot). • Continuous data transmission and reception with real-time computing requirements on limited computing resources. • RAT-specific quality of service (QoS) targets that implies specific computing requirements. • Different constraints and computing loads for different RATs and radio conditions. • Dynamic reconfiguration of the protocol stack, either partial or total. • Heterogeneous multiprocessor execution platforms. The possibility to dynamically reconfigure an SDR platform provides the full flexibility in adapting an SDR application’s computing requirements to the instantaneous radio environment and Quality of Service (QoS) demands.

18

The P-HAL Framework The current problems and available technologies in the Software Radio world have been presented in the previous chapter. Now, a mechanism must be defined to enable the possibility of solving these problems easily and homogenously. Such mechanism will be a common developing and execution framework which, by definition, does not need to solve all that problems, but define a common strategy which enables to solve all of them (in the future). In this chapter, the Framework will be introduced. We will define first of all the concept; the next step will be to establish the names conventions used among this document. Finally, a deeper description will be done.

1.7. The P-HAL Concept First and basic step to advance in the process of defining a common framework to develop and deploy software radio applications is to eliminate platform (hardware and support software) dependencies. Next step is making simpler to specify radio applications, worrying only about algorithm description. In this context, the possibility to add as much processing resources as required for a given application, making not necessary to change it at all, is of must importance. The possibility of (re)configuring the radio terminal with different hardware from different providers requires the capability of adding (and removing) plug-and-play hardware to (from) the system. Different hardware topologies, configurations and, above all, assigned tasks impose restrictions on the integration of different hardware to construct software radio platforms. Due to these and the previously stated objectives, a multi-platform software abstraction layer and execution environment, P-HAL (Platform-Hardware Abstraction Layer), has been defined and designed to provide such features. The context within the applications will run is a matter to take into account because this strongly determines the framework. Signal processing applications are most of processing input signals and deliver output signals. Thus, the first step would be as simple as provide the mechanism to the application to deal with these input/output signals. These entities can be software or hardware entities, but in any case, abstracted from real implementation. The particularity of such interfaces between several entities is that is unknown at develop time, and only during execution time all the entities will be unified and the entire application will be build. These and more ideas should be implemented in the framework. In the following lines we define in terms of functionalities where the design must support on: • First and most important, flexibility. The framework must efficiently implement the flexibility concept required by SDR. • Execution control management. • Hide the platform heterogeneity to the radio application. Abstraction layers are required. • Computing resource management. 19

• Computing objects data packet oriented messaging. Not processor (device) oriented. • Parameter or variable (signals) evolution capture during the execution of the application. • Processing resource parameters evolution capture for its autonomous management or control. • Auto-learning/cognitive capabilities for the internal resource management.

Figure 3-2 – P-HAL Layers Schematic

To illustrate the concept of P-HAL as a simple mechanism of information exchange lets consider, for example, the following simple piece of code of an execution thread of a general radio application: for ever { sample=read(MY_INPUT,BYTES_PER_SAMPLE); result=process_sample(sample); write(MY_OUTPUT,BYTES_PER_SAMPLE,result); }

The process_sample function of the previous example can be written using any kind of portable programming language (C/C++ or VHDL for example) that can be compiled and translated to processor specific binary files. On the other hand, the read and write operators closely depend on the platform and must be specifically designed for it. Thus, when porting this code to another platform, the read and write specific P-HAL library implementation will be used.

20

Now the question is: would this simple P-HAL be enough to build Software Radio signal processing applications? The answer is yes. Only a short specification of an origin and a destination of a data flow is enough to build any kind of application. Similar to the Turing Machine computing theory, any kind of processing machine based on a single operation of moving data from one origin to a destination is able to execute any kind of processing task. Thus, porting this concept to signal processing tasks is trivial. In Figure 3-2, a representation of the abstraction layers involved in P-HAL is drawn. On the top we see the connection graph of the object pieces building the radio application, this is the Abstract Application Layer. On the next level, objects are mapped into real Processing Elements (PE), they physically need a place to be executed or located on (the mapping pattern is not important by now). The P-HAL Layer is the abstraction layer found between the real application processes and the Hardware Layer. An overall PHAL Platform is represented although three PE (each of them enabling the P-HAL functionalities) are under it.

1.8. P-HAL Name Conventions In this section we will enumerate and define the entities used further in the entire document. This “dictionary” is organized from outside to inside. We start presenting the farthest view of P-HAL (the global environment) and then we step in looking into the elements that form it, and so on. This means that when one element is presented, some names will be used although they are not yet defined, it will be done further. As an introduction to this chapter, Figure 3-3 represents a schematic view of P-HAL most important parts.

Figure 3-3 – Relevant Parts of a P-HAL Environment

21

P-HAL PLATFORM: Also referenced as P-HAL Environment, a P-HAL Platform is defined as a set of Processing Elements (PE) each of them enabled with the P-HAL functionalities. The whole set appear to the user as a unique Virtual Platform, called simply P-HAL Platform. The physical delimitation of the mounted processors is indifferent. This means that several boards can appear with one or more processors. It is not important as the whole set of interconnected and P-HAL enabled PE is understood as only one P-HAL Platform. One of the PE will always act as a MASTER PE in a P-HAL Platform. PLATFORM The term “platform” (in lower-case) will be sometimes referenced in this document as generally speaking architecture or device family, for example: Linux platform, Texas Instruments DSP platforms, FPGA platform, etc. P-HAL PE (PE=Processing Element): A Processing Element is a hardware device able to execute any kind of task. The device can be either digital or analogue; the only requirement is that it must be able to interact with the rest of the P-HAL environment (P-HAL Platform). A PE can be for example: a GPP (PC platform), a DSP, a µC, an FPGA, an analogue FPGA, an ASIP, an ASIC, a configurable CODEC, a tuneable analogue Filter, etc. At least one interface with another PE of the environment must be available. This physical interface must be able to be used under the P-HAL considerations and constraints. This means that the following P-HAL logic interfaces must be able to be mapped into this physical interface: • One master control PE-ITF • Any, one or more slave control PE-ITF • Any or one master sync PE-ITF • Any, one or more slave sync PE-ITF • One or more input/output data PE-ITF It also must provide the following processing (software or hardware) capabilities in terms of P-HAL functionalities: • An underlying P-HAL Hardware Library API • The following P-HAL Software Daemons: o FRONT-END • If processing objects are going to be loaded, also the following daemons: o EXEC o SWLOAD o BRIDGE o P-HAL Services API A PE without a master control interface will be known as MASTER PE. Check section 1.11 for more details on Hardware Library, Software Daemons and PHAL Services API. PHYSICAL ITF: 22

A physical interface is a hardware data interface between one or more PE. Any kind of interface can be understood as physical interface: analogue interface, digital bus data interface, serial interface, shared memory, network IP interface, etc. The physical interfaces must be able to be used by P-HAL to map logic interfaces (PEITF) on it. PE-ITF: A PE-ITF is a logic (abstracted) peer-to-peer packet oriented (real or soft simulated) interface, connecting one P-HAL PE with another P-HAL PE. These interfaces are mapped by P-HAL (or underlying OS or firmware) into physical interfaces. The following types of interfaces are defined: • Master control interface: Bidirectional control interface from where commands are supposed to come. • Slave control interface: Bidirectional control interface towards where commands are propagated to the rest of PE(s). • Master sync interface: Bidirectional interface connected to a PE running SYNC MASTER Daemon. • Slave sync interface: Bidirectional interface connected to a PE running SYNC Daemon. • Data interface: Unidirectional data interface connecting two PE. These interfaces are used only by objects to exchange data among separate PE. Check chapter 1.21.2 for more details on P-HAL networks and interfaces. P-HAL APPLICATION: An application is defined as a set of P-HAL OBJECTS and a set of P-HAL OBJ-ITF. The objects are interconnected using a conventional connection graph. The concatenation of all the objects under the defined graph, executes a complex processing task. P-HAL OBJECT: An object is a set of instructions or hardware elements executing a processing function. It must be described following P-HAL specifications and can not use any platform or hardware dependent functionality. P-HAL OBJ-ITF: An object interface is a unidirectional logic peer-to-peer packet oriented interface interconnecting two objects. Object interfaces can be mapped on any kind of physical interface, even could not represent any real interface, for example in conventional computing processors where internal memory interfaces could be used. On the other hand, a remote IP interface could be enabled between two processors while objects would interact the same way.

23

1.9. Hardware Abstraction If we follow the previously seen arguments about the features such framework should provide, the most important conclusion is that hardware can’t be seen at all. The first level to accomplish this is a Hardware Abstraction Layer (HAL). The HAL can be understood as a software layer typically included inside the operating system (or something similar) which from one side is in close contact with the hardware while in the other side deals with other software components, thus, from the point of view of the software running over the HAL, there is any detail about the underlying hardware. All the hardware procedures, like communication between several processors, in/out interfaces, memory ranges, context switching, etc. are totally hidden to the software piece running over the HAL. The HAL does not introduce any new concept at all. Some tasks are encapsulated and offered as software services to higher layers. But the most important feature is the ability to make portable between heterogeneous platforms these encapsulated tasks. An example of this is a common Operating System. In such architectures, most of the functions and services can be described regardless of the hardware. In Figure 3-4 we can see in the stack as OS services run over HAL services. At a higher level, the application accesses these services with an API provided by the OS.

Application OS API Operating System Services HAL Hardware Figure 3-4 – The HAL in a Software Stack

Then, if we incorporate HAL in a system, building applications requiring some kind of close contact with the hardware becomes easier because it’s not necessary to make different code for different hardware configurations. For example, accessing a resource located outside the common bus of the processor would require a different code depending on the bridge device architecture. HAL routines are ready to deal with these differences. The real access to the resource will be transparent for the application. From the point of view of an application over a Software Radio heterogeneous platform, the HAL concept becomes very useful and must be taken into account. In hard signal processing routines, with several time restrictions, turns very interesting to include the 24

minimum layers possible in order to reach the maximum efficiency. Thus, only one level, the HAL, must be included in this kind of systems.

1.10. P-HAL Functions and Services Moreover of providing a simple abstraction layer, some extra services have been defined to help the management and execution control of radio applications. Such services provided by P-HAL are listed next: 1. Real Time information exchange among two platforms following P-HAL rules. This exchange is, from the application point of view, exactly the same as an exchange of data inside the same platform. 2. Isochronisms of data and processes running in different platforms. 3. Control and temporal scheduling of processes among all platforms. 4. Monitoring and real time statistics generation. 5. Distributed management of real time statistics 6. Initialization parameters and run-time adaptation of them. 7. Homogenous executable loading procedure and management. 8. Homogenous and organized executables repository 9. Auto-Mapping of executable resources into processing resources. 10. Auto-discovering of network architecture and plug-and-play support. This list of functionalities will be always available in the whole environment. However, P-HAL builds a P-HAL Platform joining different devices (PE) and interconnecting them in some defined way. Thus, the services have to be provided by the P-HAL Platform, which means that some PE may not be able to serve all the functionalities, but others will. For example, an FPGA may not include the possibility to access to the executable repository directly but a network neighbour (for example, a Linux computer running P-HAL) should provide them. In other words, this list of services is the list of services provided by P-HAL, not a P-HAL PE.

1.11. P-HAL Architecture In order to join multiple platforms providing support to the application with the previously seen set of functions, P-HAL has defined an underlying structure for both, interoperability of independently developed P-HAL software and portability of P-HAL routines from one platform to another. In this section, the different layers inside P-HAL and their functions are focused. First of all, it is necessary to identify the roles and players within a P-HAL context. Application modules are well known and benefit from the described API to exchange information. But under the modules there are a set of tasks and functions that must implement the requests from the first. In order to achieve the purpose of P-HAL, it is possible to identify a large number of tasks that do not change from platform to platform, while there are other tasks that are platform dependant. Larger is the number of functions that can be included in the first 25

set, easier is the portability of P-HAL from one platform to the other. Conversely, larger is the platform dependant part, lower is the flexibility to adapt the services to a new platform. This is because the offered services become more complex when with increasing software sizes. Then, the P-HAL platform dependant part will define elemental services that can potentially be implemented with a low cost and software depth.

P-HAL Software Daemons OBJECT

FRONT END

STATS

CMD MAN

SYNC MAST

SW LOAD

EXEC CTRL

SW MAN

STATS MAN

SYNC

BRIDGE

SW MAP

API

P-HAL Software Library (Services to Objects )

API

P-HAL Hardware Library (Services to upper P-HAL layers) API

Operating System (optional) Platform Hardware

Figure 3-5 - P-HAL Components Diagram

In Figure 3-5 there is a schematic view of the different P-HAL components and libraries. At the top-left level of the stack there is application software represented here by a single P-HAL application object, which only uses P-HAL functions to interact with its environments. These P-HAL functions are called inside the different objects in the application and their implementation is found within the P-HAL Software Library, which is a platform independent library. These interactions with the environment and the rest of the operations done by the framework are executed by the P-HAL Software Daemons (on the top-right of the stack). These are a set of stand-alone entities each of them in charge of a few operations (see section 1.11.2 for a short summary). Until here, all the implementations of such functionalities are platform dependant. To interact with the hardware, these software pieces use the platform-dependant P-HAL Hardware Library (accessed through the Hardware Library API). This enables the previously stated portability of P-HAL functions. The implementation of the Hardware Library may require, in some platforms, the use of an underlying Operating System (OS) or may access directly to the platform Hardware if, although an OS is present, no support for certain device is provided. 26

1.11.1. Hardware Library API This Library should provide an abstraction of the platform underlying hardware. This abstraction is accessed through an API so that from the Software daemon point of view there is no platform dependence. This behaviour will be enabled using a set of functions. Three are the main features these set of functions would have to provide: • Process/Device execution management • Easy data manipulating • Easy data transfers • Computing resource management Thus, the final set of functions offered by the API should solve these four problems. It is worthy to note that among these functions there are two clearly different sets. The first one is oriented towards P-HAL needs. The second one is a set of utilities that are very popular in some general-purpose environments which make much easier the application development. For instance, all the “libc” components that can be found on a Unix-like platform are necessary or essential for many applications, even if they are simple. Then, a set of these functions can be included in the implementation of the PHAL Hardware Library when the underlying platform does not provide them, or can be directly exported from the underlying platform in case that it supplies them. On the other hand, in hardware devices such as FPGA or analogue devices, some operations may be optimally implemented using manufacturer provided implementations in forms of IP entities. P-HAL can take advantage of these already implemented solutions shortening the design time as well as enabling future version upgrades. Once these basic functions to speed-up the software development at the focused level of complexity are ready, it just remains extending the set with other functions, which will be called the P-HAL Hardware Library Core. In the following table (Table 3-1) there is a list of all the functions included in the P-HAL Hardware Library Core.

Table 3-1: List of specific functions that are provided by P-HAL Hardware Library

27

P-HAL Hardware Library Core Control int PHAL_API_Init(char *daemon _name, callback_t *f); int PHAL_API_Exit(); void PHAL_Check_cmd(); void PHAL_relinquish(unsigned int slots); void PHAL_exit(int state); int PHAL_IsDaemon_Present(char *daemon_name); Internal Messaging int PHAL_snd_cmd(char *daemon_name, int cmd, int *buffer, int len); int PHAL_snd_msg(char *daemon_name, int cmd, int *buffer, int len); Interface Management int int int int int int int int

PHAL_Create_itf(int size); PHAL_Delete_itf(int id); PHAL_Attach_itf(int id, int mode); PHAL_UnAttach_itf(int itf,int mode); PHAL_SetOpt_itf(int itf, int opt, int args); PHAL_set_xitf(int itf_id, int xitf_id, int mode); PHAL_snd_packet(int itf, int *buffer, int sizew); PHAL_rcv_packet(int itf, int *buffer, int sizew); Platform Information

int PHAL_Get_xitf_info(struct xitf_info *xitf, int max_itf); int PHAL_Get_cpu_info(struct cpu_info *info); Process Management int int int int int

PHAL_NewProcessInfo(struct new_proc_i *info); PHAL_CreateProcess(char *buffer, struct new_proc_i *info); PHAL_RegisterProcess(int pid, char *daemon_name); PHAL_RunProcess(int pid); PHAL_RemoveProcess(int pid); Resource Management

int int int int

PHAL_OpenResource(int resource_id, int mode); PHAL_CloseResource(int resource_id); PHAL_ReadResource(int resource_id, int *buffer, int len); PHAL_WriteResource(int resource_id, int *buffer, int len); Timing Control

void PHAL_get_time(time_t *t); void PHAL_set_time(time_t *t); int PHAL_get_tstamp(void); int PHAL_get_tslot_usec(void);

A more detailed view of Hardware Library and its functions will be faced up in chapter 1.21.5.

1.11.2. Software Daemons Moreover of the different library layers represented in Figure 3-5, at the right of the figure there is a representation of software components that belong to P-HAL. They per28

form several tasks that are required to successfully run the user application on top of PHAL. The set of daemons included in P-HAL and a summary of their respective tasks are described following: •

SYSTEM MANAGERS This group of daemons has the objective to provide requested information to a set of local manager daemons. o SW MANAGER: This module manages the application resources. This means that it maintains a database of all possible applications to load, the description of these applications and how to get the executables needed to run the application. The description of the application includes a list of objects and a list of interfaces linking these objects as well as the characteristics of both. During the loading of an application, this module generates a unique Id (which will be released when unloading) for further quickly addressing. The access to the executable resources is also maintained in a database. The executables may appear in different ways (platform independent source code or platform dependant relocatable or non-relocatable binary) and this daemon is also in charge of accessing the correct file depending on the characteristics of the platform. o STATS MANAGER: This module manages all information regarding the statistics and initialization parameters of the application objects. The statistics configuration and the parameters values are kept in a separate resource (file) for each object. The association of such resources with the object is done with a database, maintained also by this daemon, and is addressed using an object name and an application. This enables the user to assign the same parameter values or statistics configurations to several objects or conversely, assign for an object different configurations depending on the application. o COMMAND MANAGER: 29

This daemon has a very small task: it reads user entered commands (in text mode, from a resource like a file), parses it and builds a command packet, understandable by the rest of the System Managers. o SYNC MASTER: The SYNC MASTER daemon answers with its local time under a request from the Sync daemon enabling the synchronization of both processors. •

LOCAL MANAGERS These set of daemons serve the main P-HAL functionalities available to the application objects. o FRONT-END: All the control commands and packets are centralized and managed by this daemon. This daemon must, then, be available in every processor using P-HAL because acts as a gate to the rest of the network. It must ascertain which Daemons are running in the local processor and act as a bridge for control packets going to them. In some cases (daemon not available or not able to serve the request), the FRONT-END daemon will bridge packets to/from other processors able to serve it. Another task associated with this daemon is to automatically discover the network and the list of processors around it (plug-andplay feature). This includes not only the auto-identification of the self processor but the ones connected to it. Finally, and during this registration process, local platform characteristics (time slot duration, processor family, processing power, internal bandwidth, etc.) are registered to the SW MAP daemon. o STATS: This object is an optional object. Only if a processor is going to run an object using statistics variables and initialization parameters (perhaps most of the cases) must have this daemon running on it. During the initialization phase of the objects, this daemon uses STATS MANAGER daemon to gather the information or values requested. Later, during the execution phase, statistics variables are read or modified by this daemon without the intervention of the object. 30

Statistics variables values are reported or modified under a request from STATS MANAGER and packet by STATS daemon. All these operations do not affect the object execution. o SWMAP: This object is in charge of mapping the objects compounding the application into the available resources within the processor network. It acts transparent from the user in every step. In the first one, this module builds a graph of the processor network architecture it is on charge of (including its processing and interface characteristics). In the next step, after a flat list of objects is requested to load, it calculates which is the optimum distribution (mapping) of the objects into the available resources. This operation is transparent to the user, and is obtained using the TW-MAPPING algorithm 1.27.2. This mapping, however, is done in a hierarchical way. The first SWMAP found will do the first iteration, mapping the objects into processors it is in charge, however, one of these processors may be running another SWMAP daemon. In that case, the mapping will iterate by a second round, re-mapping the previously mapped objects into the new processors under the second SWMAP. This behaviour is deeply described in section 1.14. o SWLOAD: This daemon does the mapping of objects into the local processor. In software architecture platforms (GPP, DSP, etc.), that means to load the executables and launch them as well as assign internal interfaces connecting them. Otherwise, in non-programmable devices (but reconfigurable) as FPGA devices, that would mean to reconfigure the appropriate section or area of the device so that the objects can send and receive data from the adequate destinations or origins.

o EXEC CTRL: This daemon controls the execution state of the objects within an application context. The possible object execution status can be (see chapter 1.23): 31

• • • •

INIT RUN PAUSE STOP

Requests are received to change the status of a single object (this daemon is not aware of what does an application mean) by the SW MANAGER daemon (this is the one who knows the list of objects of every application). This daemon also ensures that the object executes following the real time restrictions. This means to ensure that the object not only is in the correct status but it finishes its execution in the correct time stamp. There are two kinds of real time execution control: • Real Time Execution Control: Checks that last relinquish of the object has been done at least a number of timeslots before (depending on state). • Strict Real Time Execution Control: (only available in some platforms) Checks that any object is still running after the end of last timeslot. o BRIDGE: This daemon has the main task of serving as a bridge of data packets to the rest of the platforms in its neighbourhood. Several objects willing to exchange data with another processor may share a unique physical interface and the BRIDGE daemon is the one who will manage this behaviour. How objects interfaces are mapped into external interfaces is done by the SWMAP daemon, the only one who has a graph of the interconnection network. A second task of this daemon is to send local processor identification to all the processor it is connected to. This operation is done at boot and enables the plug-and-play network discovering feature. The local identification is send through all the input interfaces. Conversely, any identification packet received by a neighbour through an input interfaces is sent to SWMAP daemon so that it can build the connectivity graph.

o SYNC: This daemon, only needed if time slot synchronization is required, requests time to the SYNC MASTER daemon adjusting 32

the local processor time, maintaining the synchronization between both.

1.11.3. Software Library API From the user written object point of view, the interaction with P-HAL will be done through the Software Library API. This API will be linked to the object (statically) during the compiling phase and, as well as it uses the Hardware Library to access the hardware resources, the user will have to choose the specific implementation of the API for the platform he is going to use (this operation may be automatically executed by P-HAL in some implementations). However, the implementation of the API is portable between platforms as it uses the Hardware Library to access the hardware. In the next table (Table 3-2) the full list of functions of the API is showed. How to use them will be discussed in chapter 1.23.4. Table 3-2: List of functions that are provided by P-HAL Software Library API

P-HAL SW Library API Control int InitPHAL(); void ClosePHAL(); int Status(void); int GetTempo(float freq); int GetTstamp(); Flow Management int CreateFlow(itfid_t logic_itf_id, int mode); int CloseFlow(itfid_t logic_itf_id); Flow Using int ReadFlow(int fd, int *buffer, int size); int WriteFlow(int fd, int *buffer, int size); Initialization Parameters int InitParamFile(void); int GetParameter(char *name, int *value, int maxlen); Statistics Management int int int int

InitStatsFile(void); InitStat(char *name); SetStatsValue(int statid, int *value, int size); GetStatsValue(int statid, int *value,int maxsize);

33

1.12. Synchronization and Time Control The proposed execution environment includes a temporal control mechanism of the radio application. This requires a reference to be propagated among the whole system. The objectives of this temporal referencing are the following: • Maintain a synchronisation between all the platforms. • Achieve an isochronisms of data • Limit the delay between the origin and the destination of data • Allow a temporal scheduling of the tasks composing the application. To achieve these objectives, the time is divided into successive time slots of regular length like in TDMA systems (Figure 3-6). Thus, the time in the P-HAL Platform is measured using multiples of these time slots. Objects are executed once every time slot and the data they generate are not available to the next object until the next time stamp. This solves the problem of maintaining the isochronisms among different processors as no matter which is the delay existing between them as the delay will be always the same (and deterministic): one time slot. Note also how there is no dependence between the object interchanging of data and the location of those objects.

Figure 3-6 – Platform Slot Time References

If we want a unique virtual time in the whole abstract platform (P-HAL Platform) becomes necessary a synchronization between the internal clocks of each platform. This synchronization should have the enough precision. Many reasons make difficult for this precision to be “perfect”: software access to the clock introduces latencies, interfaces delays and bandwidth limitations introduce also latencies, etc. However, the only requirement will be that this synchronization is “just enough”. For example, if the interval between operations in a platform is of 1ms, a difference of tens of microseconds between clocks may be negligible. A possible solution for this problem may be to use different time slot longitudes for each platform, depending on its time resolution and synchronization error among the rest. So, the relative synchronization error is maintained among the whole environment. This may be a common politic as certain signal processing applications require a very 34

high time resolution while others do not have tasks to execute so frequently, so, the granularity can be increased and the required resources decreased. So, several time slots duration may exist among the whole environment. DSP or FPGA systems are supposed to use short time slots, of tens or hundreds of microseconds. It’s obvious that in high level systems like Linux or other OS, becomes unrealistic try to use such time division, as tasks are executed in a second plane, and timers resolutions are in the order of milliseconds. If several time slot durations are used, the only aspect to be concerned about is that the durations must be multiple between each other. This is because a whole time can only be measured in the system. Thus, the minimum unit will be considered the one of the shortest duration. The rest of the platforms will count their time as multiples of this time slot (Figure 3-7).

Figure 3-7 – Different Processors use different Slot durations

Thus, every P-HAL slave clock synchronizes its time with a P-HAL master clock. How this internal time is managed is not relevant by now. After the synchronization procedure finishes, a whole virtual time can be obtained for the whole virtual platform.

Figure 3-8 – Platform Synchronization Hierarchy

The proposed synchronization procedure is based in a simple CLIENT-SERVER architecture, that is, a question-answer mechanism. The client (here SYNC SLAVE) makes a 35

time reference request to the server (SYNC MASTER) who, after obtaining its local time reference, returns it to the client. The client, then, adjust its local time reference according to the received from the server. The uncertainness in the time adjust comes from several time intervals: the time interval the client and server use to check or modify their time, the time interval the packet is in the trip from the client to the server, and vice versa, etc. All times may be negligible or at least similar in server and client, becoming then the Round Trip Time the only time affecting the uncertainness. The error committed in the alignment of two clocks located in two separate PE with a symmetric Round Trip Time (TR) is: ∆TE < TR/2 Once the platforms are synchronized, however, the synchronization procedure does not end. It’s obvious that all clocks have certain limited precision, or misalignment among time. How often the reference must be re-synchronized or, in other words, the period of synchronization shall be calculated depending on the maximum relative error specified for the whole P-HAL Platform and, obviously, how fast this misalignment grows. For example, in a system where the time slot is 10ms of duration, and a synchronization error of 1.5% is supposed and the maximum relative error among PEs is defined to be of 5%, if our local clock has a 100ppm relative precision, the misalignment will be 100µs every 1s; thus, the period of synchronization will be of 3.5s.

1.13. Interface Definition The interface definition is a key aspect when defining an abstract method to define applications over heterogeneous platforms. Here, interface stands for a mechanism used by several elements in the application for any kind of information exchange. In our context (signal processing applications) information will be limited (in most of the cases) to time variance signals and we will see as, if the interfaces are well defined, the transference of information from one side to another will be independent of the method used to physically transfer this information. The control mechanism used by P-HAL for the interfaces are, first of all, the above seen time slotting, and second one a packet slicing of the information. Several advantages provide the packet oriented interfaces: the manipulation of data and the checking of its integrity are easier for higher layers, delays are easier upper bounded, packets are easier to route in large and complex network architectures (Internet), etc.

The following attributes are defined by P-HAL to characterize interfaces: 36

• Bandwidth (Mbytes/s): The mean data transfer speed allowed through the interface. • Maximum packet size (in words of 32bit): Upper bound for the amount of data allowed in the same packet. Here interface and platform characteristics must be taken into account. For example, in a device as an FPGA, a 10kbytes packet size may be unrealistic as memory is often internally allocated. Conversely, in architectures like PC-Ethernet interfaces, a packet size of 8 bytes is also a bad politic as the overhead introduced by low layer’s headers is intolerable. • Maximum delay (in slots): The packet is never available as soon as the writer puts it into the interface. Until the next time slot, the receiver will not be able to read the packet. Higher delays can be used to obtain different priority interfaces. In such case, P-HAL will have more time to move the packet, using the processor power for other application or P-HAL tasks. • Directionality (Uni/Bi-directional): Some interfaces may allow data to flow in both directions, while others just in one direction.

1.14. Processor Network Management One of the objectives of the P-HAL framework is to provide the user a unique Virtual Platform (P-HAL Platform) with certain characteristics where objects are loaded and their execution environment is jointly managed. Inside this P-HAL Platform, obviously, there is an array of unknown connected PE with an interconnection network architecture which is also unknown. A PE is characterized in P-HAL with the following parameters: • Processor family: Indicates the family of the processor or platform, for example TIC6000, PC Pentium, PowerPC, FPGA, CODEC, tuneable Filter, etc. • Computing Capacity: Measured in MIPS (Millions of Instructions per Second) indicates the available processor computing capacity for the execution of P-HAL context objects. This magnitude should not indicate the absolute processor capacity as the P-HAL used resources should be subtracted. For example, let’s suppose a 8.000 MIPS DSP processor running a P-HAL implementation using a total of 10% of every time slot for control processes letting for the objects a total of 90% of the duration of the slot. In such case, the Computing Capacity parameter should take a value of 7.200 MIPS. • Volatile Memory: The amount of available volatile memory (RAM) must also be defined. It is measured in Mbytes. • External Interfaces: All the external interfaces must be identified and characterized following the parameters seen in section 1.13. • Other functionalities: Such as internal digital-to-analogue converters, internal tuneable filter banks or interfaces to non-volatile memories (ROM, hard disk, etc.) should also be characterized. Not only is the PE network to behave like a single P-HAL Platform is a requirement. Also the possibility of some kind of hardware auto-discovering or plug-and-play feature would be interesting. The reason is to provide a mechanism to increase or decrease the 37

processing capacity “on the fly”, allowing the user to reuse and balance the amount of power required within the amount of available power. The second objective of the P-HAL Platform is the capacity of automatically map the application objects into the processing elements without the intervention of the user at any step of the process. This means that once the network has been discovered (processing elements characteristics and interconnection matrix) by the plug-and-play feature, a mechanism must exist that centralizes this information and executes the mapping algorithm.

38

P-HAL Implementation Details In this chapter we will define the details about how the previously seen set of functions is implemented in a real P-HAL framework. These rules must be strictly followed in order to maintain the coherence and homogeneity between different P-HAL implementations from different platforms while this is one of the framework objectives. Reader will note as documentation is repeated in some occasions, in the Software Daemons (1.15) description chapter and in the Procedure Guide chapter (1.19), this is intended to serve as a two-input guide: if reader needs to know how a Software daemon must be implemented, the first one gives this point of view. On the other hand, a reader willing to know which daemons are implied in a determinate procedure and what information do they have to exchange the second chapter will be the best.

1.15. P-HAL Software Daemons The following specifications are based on the supposition that the reader has a global vision of which are the tasks assigned to each daemon (chapter 1.11.2). In the Daemons description, the contents of what information must be interchanged between daemons will not be defined; it will be found in the Procedures section (1.19). This section, however, gives a detailed view of which should be the behaviour of every daemon. This section is structured in tables, each of them describing one daemon. Every daemon has a similar behaviour (algorithm) where some initialization stuff is done and next it falls to the main loop where data is read from an interface and an action is done depending in the received packet. This is one mechanism to receive action requests; another one is the internal daemons messaging (see Hardware Library API Reference in chapter 1.21.5). Both mechanisms, in other words, all possible input commands are also described in those tables. Commands are often referred by a name in capital letters; it indicates a constant whose value should be checked in section 1.20 (P-HAL Global Constants).

39

FRONT-END Briefing

This daemon must be running on every processor. It bridges incoming control packets to the daemon they are addressed. Refer to chapter 1.17 for more information on how control packets are routed in P-HAL networks. It also carries out the registering to the network at boot as well as the discovering of the surrounding processors.

Implicated Procedures

Processor Hierarchical Identification Procedure Processor Characterization Procedure (the rest of the Procedures indirectly implicate this daemon as it acts as a bridge)

Maintained Variables

Processor Id Parent Id Database of Processors Id under me for every slave interface.

Input Interfaces

One input interface from every available daemon One Master control input interface One or more Slave control input interface.

Output Interfaces

One output interface to every available daemon One Master control output interface One or more Slave control output interface • GET_FIRST_SON: Returns the first son Id found towards a slave interface (used by SWMAP). • SND_CTRL_PKT: Uses master control interface to send a packet (used by BRIDGE) unless destination is present (for example, if destination is SWMAP and it is present, deliver locally).

Internal Commands

External Commands

None

Algorithm

I. II. III. IV.

V.

Discover the available local Daemons using the Hardware Library API. If they are available, create a data interface to every one of them (except SYNC or SYNC MASTER). Discover external control interfaces list. Check if slave interfaces available. Setup the read side of each interface. Check if master interface available. a. If available, setup it and jump to registering step (V) b. If not available set local Processor Id to 1, Parent Id to 0 and jump to step IX. Generate a Processor Id and use Connection Packet to register (see PE Identification Procedure). 40

VI.

Main loop: a. Read packets from internal daemon’s interfaces, from external master interface and ONLY read from slave interfaces when we have been registered (we got a Processor Id). b. If new packet, decide where to go (check Routing Control Packets) and bridge it. c. If packet is from master interface and is a Connection Packet jump to step VII. d. If packet is from slave interface and is a Connection Packet jump to step VIII. VII. Connection Packet received from master interface: a. If we have not been identified yet (we do not have Processor Id) connection packet is for us. Save new Processor Id and Parent Id and jump to step IX. b. If we have been already registered, the packet must follow it way back to the originator, check in our local database (and using Requested Id in the Packet contents) which interface did the packet came from where it first came (step VIII.b.ii). VIII. Connection Packet received from slave interface: a. Setup the write side of the interface. b. Check if proposed Id is already in local database. i. If exists, propose another and replace packet information. ii. If not, save in database as well as which interface did it came from. c. If master interface available, write packet to it. d. If not available return the packet to the interface where it came from. IX. Once we got a Processor Id, send our Id and our Parent Id to the following daemons (if available): SWMAP, SWLOAD, BRIDGE and SWMANAGER. Jump to next step. X. Following the Processor Characterization Packet, gather local Processor characteristics and send to local SWMAP if present or towards external master control interface otherwise.

41

COMMAND MANAGER Briefing

This daemon has a simple task of reading from a resource user entered text commands and builds a P-HAL compatible command packet.


Application Loading Procedure Application Unload Procedure Object Status Change Procedure Stats Report Procedure Stats Set Procedure


None

Input Interfaces

One input interface from FRONT-END

Output Interfaces

One output interface to FRONT-END

Internal Commands

None

External Commands

None

Algorithm

I. II.

Initialize the Command Input Resource. Main loop: a. Read command. b. Parse it and build P-HAL Control Packet depending on command following the Procedure’s guide.

42

SWLOAD Briefing

This daemon maps the objects dealing to load into local resources. It includes not only the creation of the process (or the re-configuration of the device to execute the processing) but also the creation or allocation of the interfaces linking those objects. It also is implied in the downloading of the executable from the SW MANAGER daemon.


Executable Request Procedure Executable Loading Procedure


Internal interfaces database.

Input Interfaces


Output Interfaces


Internal Commands

• •

External Commands

•

• •

Algorithm

I.

GET_OBJ_DATA_ITF_ID Received by Services Library during the initialization phase of the Object and used to configure its interfaces. DEL_OBJ_DATA_ITF_ID: Conversely to the previous, it is executed during the stopping phase and clears all the interface resources. Object Load Packet: o Following the Executable Request Procedure, request the executable or executable information to parent SW MANAGER. o Create the interfaces linking the list of objects. Executable Information Answer Packet: o Request the final executable. Executable Loading Start Packet o Start the Executable Loading Procedure, loading the process and registering to Hardware Library and to EXEC daemon. Main loop: a. Read from FRONT-END interface. i. Process External Command.

43

SW MANAGER Briefing

This daemon maintains a database of all the applications the user can load and how the executables of the applications can be reached.


Application Loading Procedure Application Unloading Procedure Object Status Change Procedure Executable Request Procedure Executable Loading Procedure


Applications resource database. Executable resource database. Object Id database

Input Interfaces


Output Interfaces


Internal Commands

•

SET_PE_ID: Command received from FRONT-END to set the local and parent Id.

External Commands

•

Object Status Change Request Packet: o Search application name in local database. o Build an Object Status Change Packet for every object in the application (gather Id from database) and send packets through FRONT-END interface o If status change is to stop, delete generated Ids from local database. Application Load Request Packet: o Search application name in local database. o Generate a unique Id for every object (see Procedure Guide) and save in local database. o Build Object Load Packet and send through FRONTEND interface Object Name Request Packet: o Search object and application name in generated objects Ids database. o Build Object Name Answer Packet and send through FRONT-END interface Executable Information Request: o Gather COFF information and build Executable Information Answer Packet, send through FRONT-

•

•

•

44

•

Algorithm

I. II.

END interface. Executable Request Packet: o Begin the Executable Loading Procedure. o Relocate or compile the executable if the case. o Download the executable to the processor following the procedure. Read and parse applications and executable resources to fill databases. Main loop: a. Read from FRONT-END interface. i. Process External Command.

45

SWMAP Briefing

This daemon is the one who executes the mapping of the objects into the available computing resources. It has also the task to gather the information required by the algorithm. Refer to Hierarchical Software Mapping Procedure for more information.


Application Load Procedure Processor Characterization Procedure Data Connectivity Matrix Discovering Procedure Hierarchical Software Mapping Procedure


Data Connectivity Matrix Processing Power Resource Vector Son Processors Database

Input Interfaces


Output Interfaces


Input Commands

• •

External Commands

•

• •

Algorithm

I.

NEW_LOCAL_NEIGHBOUR: Command received by BRIDGE when new neighbour has been identified. SET_PE_ID: Command received from FRONT-END to set the local and parent Id. Processor Characterization Packet: o Save Processor characteristics in database, add computing power to Processing Power Resource Vector. o If Parent Id different of zero (it exists a parent), send total amount of computing power to parent. Neighbour Discovered: o Update local databases. o Rebuild Data Connectivity Matrix Object Load Packet: o Execute the algorithm to obtain the mapping over the resources. o Build another Object Load Packet for every destination Processor and send all back through FRONTEND interface. Main loop: a. Read from FRONT-END interface. i. Process External Command. 46

EXEC Briefing

This daemon must be included in every Processor willing to execute objects as it is the daemon in charge of controlling and watch the object execution state.


Object Status Change Procedure


Loaded Objects Database

Input Interfaces


Output Interfaces


Internal Commands

• • • • • •

External Command

Algorithm

•

I.

REGISTER_OBJ: Received by SWLOAD during the Executable Loading Procedure, save new object Id into local database and initialize variables. REMOVE_OBJ: Delete allocations of previous step. Received by SWLOAD if failed loading or by Services Library when object exists. GET_OBJ_PID: Search an object Id into local objects database, returns pid. GET_OBJ_ID: Similar to previous one but now object is searched by pid and Id is returned. SET_OBJ_STATUS: Used by Services Library to set current object status. GET_OBJ_STATUS: With a given Id, it returns the current object status.

Object Status Change Packet: o Obtain time stamp when object must do the status change. o Send to Object’s Services Library the new status change information (new status and when) Main loop: a. Read from FRONT-END interface. i. Process External Command b. In every timestamp, check that all objects are in the status they are supposed to be and that the number of 47

slots elapsed from the last status notification (last execution) is lower than the maximum allowed slots on this state. i. Print Real Time violation error. ii. Check if process still running, otherwise delete from local database. c. If supported by platform (optional behaviour), check that all objects finished their execution before the beginning of the time slot. i. Print Strict Real Time violation error.

48

BRIDGE Briefing

This daemon is only mandatory if local processor has a data interface towards another processor and any object running on it desires to use it. Another task assigned to this daemon is to discover the local neighbours. This is done by sending local identification to all output data interfaces. In the other side, all input interfaces are listened and if this packet is received a notification to parent SWMAP is realized (using FRONT-END).


Data Connectivity Matrix Discovering Procedure


External input/output interfaces database Bridge destination tables

Input Interfaces

Input data interfaces

Output Interfaces

Output data interfaces

Internal Commands

• •

SET_PE_ID: Received by FRONT-END to set local and parent Id. UPDATE_BRIDGE_TABLE: Received by SWLOAD during the assignation of object interfaces. One object willing to be bridged to an input or output external interface will have to link its interface to the external interface. This is done through this command. Local bridging tables are updated to enable the data to come/go from/to the object.

External Commands

None

Algorithm

I. II. III.

Gather external interface information using the Hardware Library API. For all external output interface, build and send the Neighbour Identification Packet Main loop: a. Read all external input interfaces i. If packet received and packet is Neighbour Identification Packet, use FRONT-END to send to parent SWMAP (use a command for 49

this purpose). ii. If packet is data packet, find in local tables the object interface and bridge it. b. Read all configured object output interfaces (found in the local table). i. If packet received bridge to the adequate external output interface.

50

STATS Briefing

This is the daemon used to manage local loaded object statistics and initialization parameter variables. It is not required if no objects are going to be loaded or any of them is using this feature. Statistics variables are implemented so that no implication of the object is required when a variable value is requested to view or modify.


Stats and Parameter Initialization Procedure Parameter Getting Procedure Stats Report Procedure Stats Set Procedure


Local stats configurations database. Local parameter configuration database. Local stats values table. Stats report tables.

Input Interfaces


Output Interfaces


Internal Commands

•

•

•

INIT_PARAMS_FILE: Command received by object’s Services Library during the initialization phase. An Object Name Request Packet will be generated. Also object is registered in local parameter configuration database. Command arguments are: o 1st word: object Id. o String indicating the registered daemon name. Used for later command communication. GET_PARAM: Command received by object’s Services Library during the initialization phase. A Get Parameter Packet must be built and sent through FRONT-END interface. Command arguments are: o 1st word: object Id. o Get Parameter Packet INIT_STATS_FILE: Command received by object’s Services Library during the initialization phase. An Object Name Request Packet will be generated. Also object is registered in local statistics configuration database. Command arguments are: 51

•

•

•

•

External Commands

•

•

•

o 1st word: object Id. o String indicating the registered daemon name. Used for later command communication. INIT_STAT: Command received by object’s Services Library during the initialization phase. Its purpose is to enable a further quicker access to the variable. Also, table resources for the values are allocated. The command returns the position of the variable in the local database: o 1st word: object Id. o String indicating the stat name. CLOSE_STAT: Command received by object’s Services Library during the stopping phase. It frees the resources allocated in local databases by the initialization of the variable phase. Parameters are: o 1st word: object Id. o String indicating the stat name. CLOSE_OBJ_STATS: Command received by object’s Services Library during the stopping phase. It frees the resources allocated by the object (all variables) in local databases. Parameters are: o 1st word: object Id. Set Stat: Command received by object’s Services Library during the execution phase. It updates the local table with new stat value. o 1st word: object Id. o 2nd word: stats pos (returned in the Init Stat command) o Following words: variable value.

Object Name Answer Packet: o Search the object in local database and update it with application and object name. o Depending on the initiator request (stats or parameter initialization) send a Stats Initialization Packet according to it and send through FRONT-END interface. Parameter Answer Packet: o Search the object in the local database and using the name registered in the initialization phase forward command to object’s Services Library daemon. Stats Definitions Packet: o Search the object in the local database and using the name registered in the initialization phase send a confirmation packet. o Also fill local variable configuration database with information received in the packet. 52

•

•

Algorithm

I.

Stats Report Request Packet: o Search the object and stat in local database. o Init local report database resources. Set end of report time stamp according to the duration and period of the request packet. Stats Set Packet: o Search the object and stat in local database. o Update local value in the table and use command to send the new value to the object’s Services Library. Main loop: a. Read from FRONT-END interface. i. Process External Command

53

STATS MANAGER Briefing

This daemon centralizes the configuration of the statistics variables and the initialization parameters values. The data is kept in a common file resource accessed through the Hardware Library API.


Stats and Parameter Initialization Procedure Parameter Getting Procedure Stats Report Procedure Stats Set Procedure


Statistics Resources database Parameters Resources database Statistics Configurations database Parameter Values database

Input Interfaces


Output Interfaces


Internal Commands

None

External Commands

•

•

•

•

Parameter Initialization Packet: o Search the object Id in Parameter Resources database. o Read and parse resource saving the parameter values in the Parameter Values database Parameter Get Packet: o Search the object Id in Parameter Values database and fill and send a Parameter Answer Packet through the FRONT-END interface. Statistics Initialization Packet: o Search the object Id in Statistics Resources database. o Read and parse resource saving the variables configurations in the Statistics Configurations database o Answer with the Statistics Definition Packet Stats Report Data Packet: o Search the object Id and variable name in the local Statistics Configurations database. o If a report resource has been configured for such variable, write to the resource the variable value. 54

Algorithm

I. II.

Read stats and parameters file resource and fill Statistics and Parameters Resources databases. Main loop: a. Read from FRONT-END interface. i. Process External Command

55

SYNC Briefing

The SYNC daemon maintains the synchronization of local time slots with SYNC MASTER’s time slot.


Synchronization Procedure


None

Input Interfaces

One input master synchronization interface

Output Interfaces

One output master synchronization interface

Internal Commands

None

External Commands

None

Algorithm

I. II. III. IV. V. VI.

Create and setup input and output interfaces. Gather time slot duration using the Hardware Library API and calculate synchronization period depending on desired accuracy. Send through output synchronization interface a Synchronization Register Packet. Wait until Synchronization Register Answer Packet is received. Use Hardware Library API to set output synchronization interface option to URGENT using the received interface Id as argument. Main loop: a. Every synchronization period: i. Send a Synchronization Request Packet ii. Wait until Synchronization Time Packet is received. iii. Receive packet and calculate Round Trip Time (time between step i and step ii). iv. Add the half of the RTT to the received time and adjust local time using the Hardware Library API.

56

SYNC MASTER Briefing

This daemon answers the Synchronization Request Packet with local time enabling the remote-local synchronization.


Synchronization Procedure


None

Input Interfaces

One or more input slave synchronization interface

Output Interfaces

One or more output slave synchronization interface • SYNC_REQUEST_PKT: Packets are received through the command function. Local time must be obtained using the Hardware Library API and answered using the Synchronization Time Packet.

Internal Commands

External Commands

None

Algorithm

I. II.

Create and setup input slave interfaces. Main loop: a. Read on all input slave interfaces: i. Answer with the Synchronization Register Answer Packet pasting the associated input interface Id to allow the user to configure URGENT option. ii. Configure associated output slave interface.

57

1.16. P-HAL Packets Any form of information exchange between one P-HAL entity and another (an object or a daemon) is done through FIFO-like packet oriented interfaces. Therefore, all the transmissions of information are done using packets. As a consequence of the layer separation P-HAL internally does, three layers co-exist also when concerning about the PHAL Packets. This layer representation is turned into a concatenation of headers introduced before the packet an object or a daemon desires to send. Depending on the layer they are travelling through, some headers will be introduced or not (Figure 4-9). The layer is the responsible to introduce and understand its headers properly. These headers are defined as: • Software Library Layer: P-HAL Data Header or P-HAL Control Header •

Hardware Library Layer: P-HAL External Header

•

Platform Layer: Other headers introduced by low-level O.S. or the interface driver. For example IP and Ethernet headers, SSL headers, etc.

P-HAL is not responsible of the last layer, so it will not be described in this document. Anyway, only the Hardware Library developer must be concerned about them. Now we describe the types of packets that co-exist in P-HAL as well as which headers must be included in every case: • P-HAL Control Packets: They carry control request, commands or other information. Software Daemons creating such Packets must include, before the payload, a P-HAL Control Header. And before this header, another one will be included or not depending on which interface they are going through: o Internal daemon-to-daemon interfaces: No header must be included. o Inter-PE master/slave control interfaces: A P-HAL External Header must be included by HW Library. •

P-HAL Data Packets: They carry data generated by an object to another. Depending on which interfaces they are going through, the following headers must be included or not: o Internal object-to-object interface: No header must be included. o Inter-PE data interfaces: First a P-HAL Data Header included by BRIDGE daemon and after a P-HAL External Header included by HW Library.

58

CTRL_BODY...

DATA_BODYn

CTRL_BODY1

DATA_BODY...

CTRL_BODY0

DATA_BODY1

CTRL_HEAD2

DATA_BODY0

CTRL_HEAD1

DATA_HEAD1

CTRL_HEAD0

DATA_HEAD0

CREATED BY HW API

DATA PACKET

CREATED BY BRIDGE

CTRL_BODYn

CREATED BY OBJECT

CREATED BY SW DAEMON

CTRL PACKET

EXT_HEAD2 EXT_HEAD1 EXT_HEAD0

Figure 4-9 – Concatenation of Headers

Better to understand this concept with some examples. In the first one we see a daemon sending a control packet to another daemon: 1. A Software daemon wants to send a request to another daemon. It must construct the packet following the Procedure Guides (see chapter 1.19). This packet consists on a P-HAL Control Header and the payload. 2. If the interface where the packet has been sent is locally attached to another daemon, no more headers are included (except by the ones required at Hardware Library Layer, transparent for the Software Library Level). 3. If the interface is going to an external interface, Hardware Library API must introduce a P-HAL External Header. This header is somehow, the way how Hardware Library APIs of both PEs understand each other. 4. The packet will be served to destination daemon without the last header. The destination will obtain the packet exactly as the first daemon sent, exactly as it was originated in the same PE. And now an object sends a packet to another object: 1. An Object generates a data packet and writes to the interface. 2. If the opposite side of the interface is another object in the same PE, no header is introduced at all (except for the internally used by the Hardware Library). 3. On the other hand, if the destinations object is outside the platform it will have to pass through the BRIDGE daemon. This daemon will introduce before the packet a P-HAL Data Header. 4. As the packet will go through an external interface, the Hardware Library will introduce the P-HAL External Header in order the destination to understand it. 5. Finally, the opposite BRIDGE will take out the Data Header and serve the object the packet exactly as it was sent. 59

1.16.1. P-HAL External Packet This packet is inserted before any packet travelling outside the PE. Hardware Library is the one in charge of this kind of communications, so it is its task to paste this header. The header consists in 3 words as seen in Figure 4-10.

MAGIC

SIZE

OPTS

PAYLOAD

1 word

1 word

1 word

variable size (word aligned )

Figure 4-10 – External Header

The purpose of this header is to allow different Hardware Library implementations from different PE to understand each other. Every packet oriented interface needs a set of internal variables or parameters in order to work: FIFO pointers, packet size, interface status signals, etc. When using different interface architectures among several PE some variables must be defined in order to understand each other. In a packet interface, the packet size is the only variable required. The other variables are used for other purposes: Table 4-3: P-HAL External Header

Field

Description Constant number used to check packet validity Size of the packet in words Other packet options (timestamp, urgent packet, etc.)

MAGIC SIZE OPTS

1.16.2. P-HAL Control Packet The control packet is divided into a header and a payload section. The first one is 3 words long (each word is defined as having 32 bits) and the second one may have a variable length depending on the associated procedure (Figure 4-11). TYPE

DST

CMD

PAYLOAD

1 word

1 word

1 word


Figure 4-11 – Control Header

This header indicates some parameters regarding the destination of the message or its required action. In the following table we can see an explanation of these fields:

60

Table 4-4: P-HAL Control Header

Field

Description

TYPE

Affected daemon in the procedure

DST

Destination Processor Id

CMD

Procedure identification

In the following table, the set of possible values these Control Header fields can have depending on their procedure (Table 4-5: P-HAL Control Header TYPE and CMD Field Values) are presented. Reader will notice as constant values are not numerical represented but left as a text description. All P-HAL Constants are a 32-bit signed integer value and, obviously, should agree between different implementations. Reader would have to refer to P-HAL Constants definitions to check their value (section 1.20). Table 4-5: P-HAL Control Header TYPE and CMD Field Values

TYPE

CMD

PROCEDURE

TYPE_IS_CONNECT

blank

Processor Hierarchical Identification Procedure

MAP_OBJ TYPE_IS_SWMAP

TYPE_IS_SWLOAD

TYPE_IS_ SWMANAGER

NEW_NEIGHBOUR

Hierarchical Software Mapping Procedure Data Connectivity Matrix Identification Procedure

ADD_SON_PE

Hierarchical Processor Identification Procedure

LOAD_OBJ

Application Loading Procedure

LOAD_START

Executable Loading Procedure

LOAD_PACKET


LOAD_OK


EXEC_INFO_ANS


GET_EXEC_INFO

Executable Request Procedure

EXEC_REQUEST

Executable Request Procedure

LOAD_APP

Application Loading Procedure

61

OBJ_NAME_REQ

Stats and Parameters Initialization Procedure

STATUS_CHANGE


PARAM_INIT

Stats and Parameters Initialization Procedures

PARAM_GET

Getting Parameters Procedure

PARAM_VALUE

Getting Parameters Procedure

STATS_INIT

Stats and Parameters Initialization Procedures

TYPE_IS_ STATSMANAGER

CMD_IS_STATGET

Stats Report Procedure

STATS_REPORT

Stats Report Procedure

CMD_IS_STATSET STATS_REPORT_START TYPE_IS_STATS

OBJ_NAME_ANS STAT_SET_VALUE

TYPE_IS_EXEC

PHAL_STATUS_RUN PHAL_STATUS_PAUSE PHAL_STATUS_STOP PHAL_STATUS_INIT

Stats Set Procedure Stats Report Procedure Stats and Parameters Initialization Procedure Stats Set Procedure


1.16.3. P-HAL Data Packet This kind of packet is only generated by the BRIDGE daemon as it is the only one who manages this kind of packets. As stated before, for an internal object-to-object communication, no header is introduced. The data packet is also divided into two sections, header and payload. In this case, the header section consists of two words indicating the destination information (Figure 412). After that, the full packet read from the object interface must be copied after. 62

dstObj

dstItf

PAYLOAD

1 word

1 word


Figure 4-12 – Data Header

Where the header fields obviously mean the following: • dstObj: Indicates the destination object Id of the data packet •

dstItf: Indicates the destination object interface Id of the packet

1.17. Routing of P-HAL Packets How the packets arrive to their destination will be discussed in this section. Data Network routing is done at BRIDGE daemon level, or better said, when mapping of objects is done, that is, at SWMAP daemon level. Data interfaces are PE-to-PE links providing a Virtual Interface between objects in both processors. The BRIDGE is the daemon in charge of creating this Virtual link. If multi-hop routing is desired (that means, a Virtual Interface between two PEs not directly linked) the daemon in charge of the object mapping will be the one in charge, also, of configuring every BRIDGE involved in the path. The routing difficulties come out when concerning the Control Network. In this case, multi-hop routing is mandatory. Some premises will be reviewed previously to explain how packets are routed in the P-HAL Control Network: • According to the hierarchy, packets can only travel upwards or downwards. •

The FRONT-END Daemons knows the Id of all the Processors under it, and through which interface can be reached. In the upper direction, only the first Id is known (parent).

•

The FRONT-END is the only daemon able to use the external control interfaces (slave or master). So any packet must go through it.

The addressing of the network is done using the Processor Id (PEID). Thus, a daemon must fill the DST field of the Control Header with the Id of the Processor it wants to send the message to. There are some cases, however, where this Id will be unknown. As defined by the hierarchy, nothing is known of what exist in the higher levels. Some Daemons may require services offered by SYSTEM MANAGER Daemons who are located upwards in the tree (see P-HAL Platform Interconnection Constraints, on User Guides chapter, section 1.21.2) and their Id is not known. In P-HAL, everything in a higher level than me is supposed to be my parent. So, if a service is requested but can’t be served in the local processor, the DST will be replaced by the parent Id. From my point of view, always my parent will be my service provider. 63

On the other hand, there is a special case when COMMAND MANAGER reads a user command and propagates this command down in the tree. This command will travel until it founds a SW or STATS MANAGER able to serve it. COMMAND MANAGER does not have any information about where these Daemons are running. In this case, the DST field of the packet will be set to 0, indicating that must be propagated to all sons. The algorithm for the routing of P-HAL Control Packets is presented next: 1. A daemon needs a service. The contents of the DST field will depend on where is the destination daemon (this must be known): •

DST is 0 if destination is local or under us

•

DST is -1 if destination is under us (but don’t want local to answer)

•

DST is local_id if destination is local or over us

•

DST is parent_id if destination is over us (but don’t want local to answer)

•

DST is any Id if destination processor Id is known (it will be a son of us).

2. FRONT-END receive a packet and first check DST field. If it matches local_id or 0, try to forward packet to local daemon (next step). If not my Id send through the proper interface and jump to step 4. The interface to be sent depends on DST Field: •

If Parent Id goes through master interface

•

If Son Id goes through slave interface

•

If DST is -1, replace by 0 and goes through slave interface

3. If the packet has been served but daemon is not able to process it because does not have access to requested resource the daemon must do the following before returning it to FRONT-END (then jump back to step 2): •

Replace by -1 if DST is set to 0 (it travels downwards)

•

Else replace DST with Parent Id (travels upwards)

This indicates FRONT-END that we don’t want local Daemons to answer the message and we want to be propagate it immediately. 4. If daemon can serve the Packet, packet arrived to correct destination.

64

If DST=my_id but NOT present or DST=parent_id

is Present ?

FRONT-END

? DAEMON B

DAEMON A

DST=0

Can’t serve DST=parent_id

Is B present?

If DST=my_id but NOT present or DST=parent_id

FRONT-END is Present ?

FRONT-END

Can’t Serve DST=-1

? DAEMON B

? DAEMON B

If DST=0 and B is NOT present or DST=-1

Can’t serve DST=parent_id

Is B present?

DST=my_id

DAEMON A

FRONT-END Can’t Serve DST=-1

? DAEMON B

If DST=0 and B is NOT present or DST=-1

Figure 4-13 – Daemon A sending to Daemon B (upwards and downwards)

In fact, there are two kinds of addressing in P-HAL: One uses processor Id, the other uses the “availability of serving” a request. This means that in most of the cases we do not know who is going to process our request because we do not know who will be able to serve it.

1.18. Automatic Resource Mapping in Complex Processor Networks The problem of mapping several application software or hardware elements (objects) into an array of unknown processing elements (PE) has been solved by defining two concepts: • A hierarchical network of processing elements is defined, with a set of arbitrary interconnected elements. • An object mapping algorithm is iterated while descending through the hierarchy The selected mapping algorithm is the TW-MAPPING algorithm 1.27.2. Future changes in this algorithm, or even the possibility of letting the user to choose it, can be easily incorporated into the framework thanks to the separation done in the process of defining the daemons and the close set of tasks they realize. Thus, only the SWMAP daemon should be modified to enable such new features.

65

The next step would be thus to characterize this (or similar) algorithm in terms of inputs and outputs. We need to know which kind of information about the processor network it will need and what will do obtain after its execution. We define, thus, the following algorithm inputs: 1. Processing Power Resource Vector: This is a vector of length N processing elements (PE) indicating their Processing Power P for resource i :

P = ( P1

P2



PN )

2. Object Processing Requirement Vector: A vector of length M objects indicating Processing Power Requirement m for object i:

m = ( m1 m2  mM ) 3. Processors Bandwidth Matrix: Matrix of size NxN (N is number of processors) where position (i,j) indicates the bandwidth of the data interface between processor i and processor j.  B11  B B =  21   B  N1

B1N    B2 N       BNN 

B12



B22  BN 2

4. Object Bandwidth Requirement Matrix: Matrix of size MxM (M is number of objects) where position (i,j) indicates the required bandwidth between object i and object j.  b11  b b =  21   b  M1

b12 b22 

 b1M  b2 M  

bM 2

 bMM

      

On the other hand, the outputs of this algorithm are: 1. Object Mapping Vector: A vector of length M objects containing the result of the mapping. The result is the index of the Processor where it has to run following the index of the vector P.

T (m) = ( T (m1 ) T (m2 )  T (mM ) ) T(m) would be then the mapping function. 2. P vector and B matrix update: The algorithm updates the P and B vectors subtracting the Processor Power and the Bandwidth used by the objects after the mapping. Thus, another mapping can be done taking into account the previously loaded objects. 66

The inputs 2 and 4 are established during the process of defining the radio application requirements in term of processing power required by the algorithms and the interconnection of such pieces. The inputs 1 and 3 would require the definition of two procedures so that the environment (better said, the SWMAP daemon) ends gathering the required information. These procedures are: 1. Hierarchical Processor Identification: The plug-and-play hierarchical hardware attaching is solved by defining this procedure which enables a PE to be identified and characterized during its boot process. It is realized by the FRONT-END daemon when it is started. The information is centralized and processed by the SWMAP daemon which internally generates a vector (input 1) with the full set of elements it has down in the hierarchy. 2. Data Connectivity Matrix Discovering: After the full set of processors is characterized by the SWMAP daemon, the interconnection matrix (input 2) is calculated by letting the BRIDGE daemon use the output external data interfaces to send an identification packet. The opposite side of the interface (BRIDGE again) will receive such information and propagate towards the first upper SWMAP daemon. Again, SWMAP centralizes all the information and thus calculates the required matrix. Reader will note how the characterization of the processors is done in the digital world only, not enabling the limitation or definition of other (analogue) features as access to analogue converters or tuneable filters. This limitation is due to the use of a limited algorithm designed to be used in common processing (digital) computers. Again, the system is ready to enable future changes either in this algorithm or even change it all. A future algorithm, should include, similarly to this one, several input matrices (one for each restriction) characterizing other PE features. Once all the inputs are available, the algorithm is initiated and an output is obtained. The resulting mapping function (output 1) will be used to finally map the processing pieces in the processing elements. However, this is only the first iteration in the hierarchy. The SWMAP daemon is configured to not propagate upwards in the tree the gathered information about the underlying elements architecture; it only propagates the total amount of processing capacity. This allows defining a scalable hierarchical architecture where the mapping procedure is iterated while we descend in the tree. Thanks to this feature, we could for example attach a new platform with a total of 4 PE with a capacity of 2000 MIPS of processing power each of them while this would be seen to the rest of the network (indeed only to the upper SWMAP entity) as a single 8000 MIPS processor.

67

Figure 4-14 – Hierarchical Resource Identification

In Figure 4-14 we see an example of this concept. The highest PE in the tree sees under it 3 PE, 2 with 1000 MIPS and another with 2100 MIPS. Altogether, concerning another (not shown) SWMAP over it, he has a total of 4200 MIPS. The same happens with one of its sons, which had 2100 MIPS. Actually, it runs a SWMAP (although the previous does not sees) and it is hiding the architecture under it. Under this second processor of 100 MIPS running a SWMAP, there are 3 more PEs with 1000 MIPS everyone.

68

Figure 4-15 – Hierarchical Software Mapping

Then, the first SWMAP will do the mapping taking into account only a total of 4 PE, without knowing that one if these PE have another 3 sons more. This can be seen in Figure 4-15 where a command for loading 29 objects is received. The first iteration of the algorithm (PE1) will map this set of 29 objects into its 3 PE under it. One of these, will receive an order to load 12 objects (more than the rest because it has more total power) and as a SWMAP daemon is present, another iteration of the algorithm will take place, mapping these 12 into the three PE available. The mapping algorithm ends here as no more PE with a SWMAP daemon exists under this level.

1.19. P-HAL Inter-daemon Procedures This section describes in detail all the procedures involving one or more P-HAL Daemons. Each procedure is based on the exchange of P-HAL Control Packets. The guide only defines the messages that must be interchanged between Daemons in several PE in order to accomplish some kind of procedure. The communication of messages within Daemons running in the same PE is not defined. This local communication is done, again, using the Hardware Library API functions. Thus, in order to maintain the objective of enabling an easy portability of P-HAL Software Daemons between platforms, any one desiring to share a daemon among other designers should define the set of internal commands it accepts, and how the data must be written. Commands are accepted using a local function defined in the initialization of 69

the Hardware. In this function, the designer will implement the processing of the local message received from other local Daemons. The format of the guide will be common for every procedure. The routing of the Control Packets throughout the network is not detailed as it has been explained previously. During the entire guide, the constants values in the packets representing the commands or its command contents are not numerical represented, they are represented by a name in capital letters. This is just for clearness, user should refer to P-HAL Constants specifications (section 1.20) before using this packets. The packet is represented entirely, including the Control Header. So, the shown fields represent the whole packet a daemon must write to the interface.

Processor Hierarchical Identification Procedure Originator daemon

FRONT-END

Destination daemon

FRONT-END – MASTER PLATFORM

Description

At boot, FRONT-END daemon is in charge of registering to the PHAL network. The objective of this registering is nothing but make all the parents in the tree aware of my presence. Thus, it is not important to my sons to know about me. During this Identification procedure, the PE will do nothing but this procedure, neither to read from any slave control interface. This registering or identification procedure is done then in the upwards (through the master control interface) direction. FRONT-END daemon generates a packet (specified below) carrying a pre-generated Id. This packet keeps going up in the hierarchy, within its way all the parents save the Id information. But, where does this packet ends the trip? There is only one PE in the whole Environment who does not need to register; the one which is in the top of the tree. This PE does not have a master control interface because is not connected to any parent. Thus, this PE does not need to register to anyone else (the registering is only done upwards). This PE is called MASTER PE, and the only difference with the rest of the PEs is that it does not register to anyone. Oppositely, it will be the one who will answer to all identification packets. The FRONT-END must determine before starting this procedure the 70

interfaces available in the processor. This is done using the Hardware Library Function listed below. See HW Library reference for details about external interface information gathering.

Packets Used

Hierarchical Identification Procedure Packet

Structures Used by Packet

None

Hardware Library API used Functions

PHAL_Get_xitf_info();

Important Notes

At this point, we have accomplished three objectives: • Every PE in the Environment has a unique Id.

Hierarchical Identification Procedure Packet

•

Every PE knows all the Ids of everyone under it and the first over it

•

Every PE knows towards which slave interface can be reached its sons.

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_CONNECT

2

DST

0

3

CMD

0

4

PEID

Requested Id

5

RPID

Previously requested Id

6

PARID

71

Id of previous one in the tree

Processor Characterization Procedure Originator daemon

FRONT-END

Destination daemon

SWMAP

Description

Once the control plane network is fully identified the data network architecture and PE’s characteristics must be identified. The daemon affected by this procedure is, thus, the SWMAP daemon. As this daemon is not mandatory, the launcher of this procedure must be FRONT-END. The procedure is initiated just after the PE Hierarchical Identification Procedure. The steps of the procedure and the description of the fields of the packet involved are described next. The P-HAL Software Daemons are platform independent, so, how can any daemon know the PE characteristics if it does not even know what kind of platform are running on? The answer is using the PHAL Hardware Library API.

Packets Used

Processor Characterization Packet


P-HAL Processor Identification Structure


int PHAL_Get_cpu_info(struct cpu_info *info);

Procedure Guide

I. FRONT-END gets processor information using the P-HAL Hardware API functions and generates the Processor Characterization Packet. II. If local SWMAP is present send to it, if not, paste Parent Id in DST field and send through master control interface. III. After routing the control packet, a parent SWMAP receives the packet and updates its local database with the new processor information. IV. SWMAP updates its Processing Power Resource Vector (see 72

Hierarchical Software Mapping Procedure) by adding new processor capabilities (*) into the position of the processor who started the procedure. V. SWMAP sums all the values of last vector and builds another Identification packet which is bridged back to FRONT-END. The destination of this packet is, obviously, its parent Id. (*) Processing power is indicated in a field in the cpu_info structure.

Important Notes

Processor Characterization Packet

Now four goals have been accomplished: • Every PE “thinks” that a SWMAP is present on its parent level. •

Every SWMAP has a complete description of the processor power it has under him, and whom is in charge of mapping resources on.

•

Every SWMAP hides the under laying processor information to the parent SWMAP.

•

New processors can be plugged-in during normal execution while the Processing Power Resource Vector is kept updated.

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

2

DST

3

CMD

4

PEID

5..9

CPU_INFO

TYPE_IS_SWMAP Local_Id (if SWMAP present) Parent_Id (else) ADD_SON_PE Id of the Processor who launched the procedure. Structure filled by the Hardware Library API function

73

Data Connectivity Matrix Discovering Procedure Originator daemon

BRIDGE

Destination daemon

SWMAP

Description

In order SWMAP to map a list of objects into a list of available resources, it needs to know the Data Connectivity Matrix (see next procedure). The procedure of building it will be described in this section. The only daemon that is in close contact with external data interfaces is the BRIDGE Software daemon. This is the other affected daemon by this procedure. This daemon will obtain the information of its data interfaces using the proper Hardware Library API Functions, and then will send an identification packet through all the output interfaces. Remote BRIDGEs will receive these packets, sending an acknowledgment to the parent SWMAP which will generate a matrix of the interconnected PE.

Packets Used

Neighbour Identification Packet Neighbour Discovered Packet


P-HAL External Interface Structure


int PHAL_Get_xitf_info(struct xitf_info *xitf, int max_itf);

Procedure Guide

I. BRIDGE builds a Neighbour Identification Packet for every output data interface and sends through it. II. BRIDGE listens in all input data interfaces and if first word matches the Neighbour Identification Packet contents, it continues on next step. III. Builds a Neighbour Discovered Packet and uses the FRONTEND command SEND_CTRL_PKT to deliver to local or send to parent SWMAP. IV. Packet arrives to a SWMAP (either local or parent) and calculates the Data Connectivity Matrix as shown in next procedure.

74

Important Notes

Neighbour Identification Packet

Neighbour Discovered Packet

As the Neighbour Identification Packet travels through the data interfaces, there must be a method for differentiating them from the normal data packets. This is done by using the destination information field (see chapter 1.16.3 for data packets description) by a blank destination. As any object is able to exist with a null Id, this type packet is uniquely identified using only the first word.

WORD

FIELD

1

MAGIC

2

PEID

3..7

XITF_INFO

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_SWMAP

2

DST

Local_Id

3

CMD

NEW_NEIGHBOUR

4

PEID

Id of the PE who launched the procedure.

5..9

XITF_INFO

Structure filled by the Hardware Library API function

75

VALUE/DESCRIPTION 0. Uniquely identified from the rest of data packets. Id of the PE who launched the procedure. Structure filled by the Hardware Library API function

Application Loading Procedure Originator daemon

COMMAND MANAGER

Destination daemon

SW MANAGER, SWLOAD, SWMAP

Description

This procedure can be launched anytime during the normal execution of the Environment. The procedure is initiated by a user command, read by COMMAND MANAGER. The command will indicate an Application Name to load. The application description will be specified in a configuration resource accessible by the SW MANAGER. As soon as the SW MANAGER reads the application to load, it generates a unique object Id (for the entire environment) for every object. It must keep a database which relations the Ids with the pair name-application of the object. It will be used later by the object during the initialization of the statistics. In order to guarantee the uniqueness of the Id, every SW MANAGER will generate the id incorporating its PE Id, following the example under: Object_Id = local_pe_id*1000+random[0,1000]; The list of objects to load will be mapped into the available resources as explained in the Hierarchical Software Mapping Procedure, however, some processors will not require the execution of this procedure as no SWMAP daemon will be available. In that case, the FRONT-END daemon will deliver the message to the SWLOAD daemon which will load the full set of objects, without doing any mapping at all. To achieve this behaviour, the FRONT-END daemon has to be configured to deliver the Objects Load Packet to the SWLOAD daemon when no SWMAP is available, although the destination is the second one.

Packets Used

Application Load Packet Objects Load Packet

Structures Used by Packet Hardware Library API used Functions 76

Procedure Guide

I. User enters string command: >> phload app_name

II. COMMAND MANAGER reads command and builds Application Load Packet sends to local FRONT-END III. FRONT-END checks if SW MANAGER daemon is available, if so bridge to it and jump to step 5, if not send to all slave interfaces IV. All FRONT-END Daemons in the lower level repeat step 3. V. SW MANAGER receive packet and search in its Applications Names Resources if it is in charge of this application. If so, jump to next step, if not use FRONT-END to keep propagating the packet downwards in the tree. VI. SW MANAGER accesses the Application Configuration Resource, reads and parses it. Generates an Id for every Object and fills local databases. VII. Sends the entire list of objects back to FRONT-END, using the Objects Load Packet, first SWMAP found (local or downwards) will receive it (see Hierarchical Application Mapping Procedure). VIII. The packet propagates downwards in the tree. All sons receive it, if the processor owns a SWMAP, the procedure Hierarchical Software Mapping Procedure is initiated, otherwise the SWLOAD daemon receive it and initiates the Executable Loading Process. Important Notes

Application Load Packet

Objects Load

Note that the packet propagates down in the hierarchy to all slave interfaces, in every level. This means that packets are duplicated. This is not a problem since one and only one SW MANAGER daemon is in charge of loading this application. This means that there are no duplicates between the Applications Names Resource of all SW MANAGER Daemons. WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_SWMANAGER

2

DST

0 (all sons)

3

CMD

LOAD_APP

4

APP_NAME

WORD

FIELD 77

String of 16 characters indicating the application name.

VALUE/DESCRIPTION

Packet

1

TYPE

2

DST

Id of the Processor where objects must be mapped into.

3

CMD

LOAD_OBJ

4

NOF_OBJ

Number of object to load (M)

5

NOF_ITF

Total number of interfaces of all objects (L)

6..(10M+6)

OBJ_STRUCT0 OBJ_STRUCT 1 … OBJ_STRUCT M

List of Object Structures one after another. Object Structure is 10 words long, so, this field is 10M words long

(10M+6)... (10M+6+7L )

ITF_STRUCT0 ITF_STRUCT 1 … ITF_STRUCT L

Just after the 10M words, the Interface Structures are pasted one after another. Now the size of this field will be 7*L words.

78

TYPE_IS_SWMAP

Hierarchical Software Mapping Procedure Originator daemon

SWMAP

Destination daemon

SWMAP or SWLOAD

Description

This procedure is initiated by SWMAP after receiving a Software Mapping Packet, generated by SW MANAGER daemon during the last procedure (Application Loading Procedure). The design of the application involves only the description of the objects which, concatenated one after another, forms the function we want to do. What does not include this description, is where this objects are going to be loaded. This is not included because the application designer is never aware of the PEs, its characteristics, but more important its computing resource power and interface bandwidth between them. Conversely, the Environment designer is neither aware of which applications will be loaded into the array of processors he do know its characteristics. Moreover of the applications (he would don’t need to know what are they going to process) it does not know the processing requirements this applications will have neither the bandwidth requirements between the objects composing the application. P-HAL appears then as the third entity that do know all variables: the available resource power, the required computing power, the available bandwidth between processors and the required bandwidth between objects. This mapping of objects into resources is done using the TW-MAPPING The details about this algorithm as well as its performance analysis are out of the scope of this document, the only thing we need to be aware of is the inputs this algorithm requires: •

Processing Power Resource Vector: This is a vector of length N processors indicating Processing Power P for resource i :

P = ( P1

79

P2



PN )

•

Object Processing Requirement Vector: A vector of length M objects indicating Processing Power Requirement m for object i:

m = ( m1 m2  mM ) •

Processors Bandwidth Matrix: Matrix of size NxN (N is number of processors) where position (i,j) indicates the bandwidth of the data interface between processor i and processor j.  B11  B B =  21   B  N1

•

B12 B22  BN 2

B1N    B2 N       BNN  

Object Bandwidth Requirement Matrix: Matrix of size MxM (M is number of objects) where position (i,j) indicates the required bandwidth between object i and object j.  b11  b b =  21   b  M1

b12 b22  bM 2

 b1M    b2 M       bMM 

On the other hand, the outputs of this algorithm are: • Object Mapping Vector: A vector of length M objects containing the result of the mapping. The result is the index of the Processor where it has to run following the index of the vector P.

T (m) = ( T (m1 ) T (m2 )  T (mM ) ) T(m) would be then the mapping function. •

P vector and B matrix update: The algorithm updates the P and B vectors subtracting the Processor Power and the Bandwidth used by the objects after the mapping. Thus, another mapping can be done taking into account the previously loaded objects.

Thus, in order to use the TW-Mapping Algorithm we will need to fill this matrixes and vectors previously with the proper data. Both the P vector and B matrix are calculated on-the-fly following the previously seen procedures, so, they are already supposed to be updated.

80

Packets Used

Objects Load Packet


P-HAL Object Structure P-HAL Interface Structure

Hardware Library API used Functions Procedure Guide

I. FRONT-END Receive an Objects Load Packet from either a SW MANAGER daemon or a SWMAP daemon (don’t known a priori). If destination Id is my processor continue on next step, if not bridge to proper slave interface. II. If SWMAP is present and packet do not come from myself, bridge to it and continue. Else, bridge to SWLOAD and jump to step VIII III. SWMAP builds the m vector by reading the Processor Power Requirement for every received object in the packet. IV. SWMAP builds the b matrix by matching linked interfaces in the object list and saving its Bandwidth Requirements. V. SWMAP launches the TW-Mapping Algorithm. VI. SWMAP reads from the Object Mapping Vector the destination Processor where every object must be loaded. VII. SWMAP generates an Objects Load Packet, fills destination Id with Processor Id kept in local database and sends to FRONTEND. Return to step I. VIII. SWLOAD receive the list and starts the Software Loading Procedure. IX. Packets will keep going down until everyone finds its destination. Now continues with procedure Software Loading Procedure.

Important Notes

Note here that the same packet will be used either for loading or mapping of the objects. There is no distinction between loading a list of objects and mapping a list of objects. If the SW MANAGER orders to load a list of objects and no SWMAP is present in the same PE, all the objects will be loaded on it. This is because the presence of SWMAP is what indicates that we want to map objects into some resources. 81

After any SWMAP entity has done the mapping, the things do not end here. As explained in section 1.18, P-HAL does a hierarchical mapping of the objects. This means that although the mapping has already been done, another mapping can be done in the next level of the hierarchy. Only the presence of SWMAP will make this mapping to be executed, or conversely, the lack of this daemon will make the processor to load all the objects with no mapping at all.

Objects Load Packet

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_SWMAP

2

DST

Id of the Processor where objects must be mapped into.

3

CMD

LOAD_OBJ

4

NOF_OBJ

Number of object to load (M)

5

NOF_ITF

Total number of interfaces of all objects (L)

6..(10M+6)

(10M+6)... (10M+6+7L)

List of Object Structures one OBJ_STRUCT0 after another. OBJ_STRUCT 1 Object Structure is 10 words … long, so, this field is 10M words OBJ_STRUCT M long ITF_STRUCT0 ITF_STRUCT 1 … ITF_STRUCT L

82

Just after the 10M words, the Interface Structures are pasted one after another. Now the size of this field will be 7*L words.

Executable Request Procedure Originator daemon

SWLOAD

Destination daemon

SW MANAGER

Description

This procedure is initiated by SWLOAD Daemons as soon as an Object Loading Packet arrives to it. This is the flat list of objects it has to locally load, as well as the interfaces which link them locally. But the received list does not contain the executable itself. It will be requested to a SWMANAGER entity able to provide it. Once the request is processed, SWMANAGER will execute a command to the SWLOAD daemon who requested that indicating the start of the loading process. This will be part of the next Procedure, the Executable Load Procedure. The loading of an object may vary a lot between platforms. For example, in a programmable device as a GPP or a DSP the object must be a binary stream containing the instructions to be executed by the processor. This stream will be copied to the memory and then an entry point will be defined. The process will begin its execution as soon as the Program Counter of the processor is set to the entry point. The programmable devices may allow, indeed, two modes of loading an executable: • Absolute addressed: All memory or program accesses are absolute addressed, thus, it must be previously relocated in order to fit in the destination memory address (it won’t be known when first build by the user). During this process, the memory resources needed by the executable must be reserved before relocating. •

Relative addressed: All addresses are relative, thus, the underlying operating system in the platform performs the relocating process when executing it. Thus, no relocation process must be done before.

For illustrating purposes, a common example of Relative Addressed Loading may be high level Operating Systems platforms such UNIX, Linux or Windows. On the other hand, common DSP need absolute addressing executables and a re-locating process must be performed before loading. Although any of these loading methods shall be used, the format of 83

the compiled file must be a standard COFF executable file. On the other hand, in non-programmable hardware devices (but reconfigurable such as an FPGA for example), the description of the object may be a binary file containing platform specific instructions. The device will read these instructions and re-configure itself in order to do the required processing task by the object. Anyway, from now over we will refer to both types as executable files. Also, we will use Build for the process of obtaining the executable from the source file. For programmable devices executables, there is another process called relocate, meaning an operation done to relocatable files where some symbols tagged as relocatables are set to a proper address. There are multiple forms in which objects may be presented or available in a system. However, it’s easy to find out how certain agreement must exist between the implicated entities in the loading process. That means, if certain platform requests an executable, some SW MANAGER daemon should have this executable for this platform, otherwise, it could be presented in source code, and be able to build the source for this platform. This gathering of this information about the platform and the available loading modes is done through Hardware Library API listed below. The process of building or relocating a file must be done, most of the cases, using local compilers or linkers. In most of the cases, user will not be able to access its source code, so, it would be very difficult for SW MANAGER daemon to execute these programs as its access is platform dependant. Thus, this process is done by the Hardware Library API. Some constraints apply to SW MANAGER daemon as not all the Hardware Library running over the platforms will be able to execute these tasks. In the next table (Table 4-6) we show the possible modes an executable may be available and what kind of operations shall be done depending on the mode it is stored (in the SW MANAGER repository).

84

Table 4-6: Executable Loading Modes

Loading Modes

Storing Modes

Operation(s) 1. Build relocatable 2. Get Memory Length

Compiled

3. Get Memory Dest 4. Relocate 5. Load

Absolute Addressed

1. Get Memory Length Relocatable

2. Get Memory Dest 3. Relocate 4. Load

Non-Relocatable Compiled Relative Addressed

Relocatable

Impossible to load 1. Build non-relocatable 2. Load 1. Re-Link if possible to be non-relocatable 2. Load

Non-Relocatable

1. Load

Packets Used

Executable Information Request Packet Executable Information Answer Packet Executable Request Packet


P-HAL Process Information Structure


int int int int

Procedure Guide

PHAL_Get_Platform_Family(); PHAL_Get_NewProcess_mode(); PHAL_Cmpl_src(int res_id, struct new_proc_i *info); PHAL_Lnk_exc(int res_id, struct new_proc_i *info);

I. SWLOAD gets information about possible loading modes to Hardware Library API. For absolute addressing continue (RELOCATED) with step II, for relative addressing (NON-RELO85

CATED) jump to step IV. II. If absolute addressing, request to SWMANAGER file memory requirements using the Executable Information Request Packet. III. SWMANAGER replies with an Executable Information Answer Packet. Then reserve data memory for process getting the start memory address. IV. Requests the executable (with destination memory if relative addressed, calculated on previous step) with the Executable Request Packet. V. In order to serve a packet, SW MANAGER must belong the executable compiled for the requestor platform, or belong a source code version and a suitable compiler for the requestor platform. Any other cases, packet will keep going up. VI. SWMANAGER initiates at this point the Executable Loading Procedure.

Important Notes

Executable Information Request Packet

Executable In-

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_SWMANAGER

2

DST

Local_Id

3

CMD

GET_EXEC_INFO

4

PEID

Id of the originator Processor

5..11

NAME

Name of the executable (24 characters)

12

PLT_FAM

13

MODE

Platform Family Id Possible executable loading modes: RELOCATED or NON-RELOCATED.

WORD

FIELD 86

VALUE/DESCRIPTION

formation Answer Packet

1

TYPE_IS_SWLOAD

2

DST

Id of the processor who did the request (in Executable Information Request Packet)

3

CMD

EXEC_INFO_ANS

4

PEID


5..11

NAME

12

PLT_FAM

13…20 Executable Request Packet

TYPE

Name of the executable (24 characters) Platform Family Id

PROC_INFO Process Information Structure

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_SWMANAGER

2

DST

Local_Id

3

CMD

EXECUTABLE_REQUEST

4

PEID


5..11

NAME

12

PLT_FAM

13…20

Name of the executable (24 characters) Platform Family Id


87

Executable Loading Procedure Originator daemon

SW MANAGER

Destination daemon

SWLOAD

Description

Once the processor has all the information needed to load the executable, it begins the Executable Loading Procedure. This procedure will conclude when the object is ready to begin its processing tasks. It won’t start to process any data at all, since it’s P-HAL Status will be set to STATUS_STARTED, in which state the object is kept in the Status() function. The final loading of the binary stream will be done through Hardware Library API as it is a platform dependant process. However, there are some common operations that will be performed by SWLOAD daemon. These operations include the downloading of the executable, the registering of the object to EXEC daemon and to Hardware Library API (see involved functions).

Packets Used

Executable Loading Start Packet Executable Loading Packet Executable Loading Finish Packet




int PHAL_CreateProcess(char *buffer, struct new_proc_i *info); int PHAL_RegisterProcess(int pid, char *daemon_name); int PHAL_RunProcess(int pid); int PHAL_RemoveProcess(int pid);

Procedure Guide

I. SW MANAGER generates and sends the Executable Loading Start Packet with information about the process. II. SWLOAD prepares receiving buffer according to file length. III. SW MANAGER sends the whole file divided in variable size packets following the Executable Loading Packet structure. Size can be chosen arbitrary. IV. SWLOAD receives packets concatenating the payload into the receiving buffer. Transmission ends when all words have been 88

received. V. SW MANAGER calculates the file checksum and sends the Executable Loading Finish Packet. VI. SWLOAD checks with checksum if data is correct. VII. If downloaded successfully, creates the process using the CreateNewProcess Hardware Library API function. VIII. Registers to Hardware Library using the RegisterProcess function. IX. Registers pid and Object Id to EXEC daemon. X. Runs the process using the RunProcess Hardware Library function. XI. If any step fails the resources may be cleared. If process has been created use the RemoveProcess function to clear its allocated resources.

Important Notes

The executable will be downloaded to a local buffer before launching it through the Hardware Library API. After that, the executable will be allocated to some other Processor resources (local Program Memory). Then, the local buffer will be useless any more. Dynamic memory allocation is recommended for this purpose as the executable file may be large. Checksum is calculated by summing all words as signed integer.

Executable Loading Start packet

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_SWLOAD

2

DST

Id of the Processor where object must be loaded (PEID of the Executable Request Packet)

3

CMD

LOAD_START

4…11 Executable Loading Packet


WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_SWLOAD

2

DST


3

CMD

LOAD_PACKET

89

Executable Loading Finish Packet

4..N

FILE DATA

N words of file data

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_SWLOAD

2

DST


3

CMD

LOAD_OK

4

CHECKSUM

90

Checksum of the transmitted file (see notes)

Executable Unloading Procedure Originator daemon

SERVICES

Destination daemon

EXEC, STATS (local)

Description

This procedure is started by the object calling the ClosePHAL() SW API function when a STATUS STOP is received. The calling to this function will end with the unloading of the executable and the de-allocation of all its resources used in P-HAL. EXEC and STATS Daemons are also involved in this procedure while their databases containing object information must also be freed.

Packets Used Structures Used by Packet Hardware Library API used Functions

Procedure Guide

int PHAL_UnRegisterProcess(int pid, char *daemon_name); int PHAL_RemoveProcess(int pid); int rtc_exit(int state);

I. Object reads an STATUS STOP and calls ClosePHAL() function. II. SERVICES sends a command to EXEC and STATS local Daemons with the id of the originator object. III. EXEC and STATS clears their local resources involving the object. IV. SERVICES calls the PHAL_UnRegisterProcess() HW Library function to un-register from the API. V. Calls the PHAL_RemoveProcess() function to de-allocate the resources used by the executable. VI. Calls rtc_exit() function to indicate the end of the execution.

91

Object Status Change Procedure Originator daemon

COMMAND MANAGER

Destination daemon

SWMANAGER, EXEC

Description

With this procedure we will be able to change the P-HAL STATUS of the objects running an application. Status values can be: • STATUS_INIT •

STATUS_RUN

•

STATUS_PAUSE

•

STATUS_STOP

The procedure is started under user command, entered in COMMAND MANAGER. This daemon, however, is not aware of which applications are available, so, the command is sent to SW MANAGER. Again, SW MANAGER does not know where these objects are running (in which processor). A query to SWMAP could be done to obtain the current objects mapping. However, the second should maintain an updated table symbolizing the current loaded applications. This is not the function of SWMAP, furthermore, this procedure shall be launched while other applications are running, and so, minimizing the overhead it introduces to the system is a priority. These are the reasons why a special routing has been designed for this procedure. A unique Object Id will be generated by SW MANAGER when boots and parses the applications resources. The object will be thus addressed by its Id. This means that the second command generated by SW MANAGER will be downwards propagated to all sons. As defined in the routing methods for control packets, only the EXEC daemon able to serve the packet (this is, the one that owns the object) will process it, keeping the propagation in other case. Another aspect is concerning the temporal slicing. The STATUS modification can not be done as soon as packet is received. The sequencing of the real time application requires all the objects in different platforms to start at same time slot, or at least, minimizing the difference. As the propagation times between SW MANAGER and the 92

set of platforms shall be different and uncontrollable, a guard time is imposed by SW MANAGER (this time must be at least bigger than the maximum delay) so all the objects in all the platforms realize their STATUS change at the same time. Packets Used

Object Status Change Request Packet Object Status Change Packet

Structures Used by Packet Hardware Library API used Functions Procedure Guide

I. COMMAND MANAGER reads command: >> phrun/phinit/phpause/phstop app_name

II. Builds the Object Status Change Request Packet and sends to sons (to the first SW MANAGER able to serve it) III. SW MANAGER receives command. Gets all Objects Id the application contains (generated during the Loading Procedure) and sends an Object Status Change Packet for every Object and send it. IV. The EXEC daemon receives the command and forwards it to the object (indeed, to the P-HAL Services API attached on it) indicating the next P-HAL STATUS and from which time slot the new status is valid. V. When the current time slot is equal or bigger than the previously calculated time slot, the P-HAL Services API updates the PHAL STATUS got from Status() API function. Important Notes Object Status Change Request Packet

Object Status

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_SWMANAGER

2

DST

0

3

CMD

STATUS_CHANGE

4

STATUS

5..11

APPNAME

WORD

FIELD 93

New STATUS desired. Application Name (24 chars)

VALUE/DESCRIPTION

Change Packet

1

TYPE

TYPE_IS_EXEC

2

DST

0

CMD

PHAL_STATUS_RUN or PHAL_STATUS_PAUSE or PHAL_STATUS_INIT or PHAL_STATUS_STOP

3

4 5

OBJECT_I D SLOT

94

Id of the Object to change its status. Slot where the change is valid.

Stats and Parameters Initialization Procedures Originator dae- STATS (by SERVICES command) mon Destination daemon

STATS MANAGER

Description

During the object initialization process (P-HAL Status INIT), the object may require the allocation of statistics or parameters resources. The information concerning object statistics and parameters is hold by the STATS MANAGER daemon (who has access to some file resources where configurations are defined). Those files are addressed to the object by associating its name and application with the configuration file. As the STATS daemon or the object does not know its name neither its application (only the Id), a request to the SW MANAGER daemon will take place. First of all, the name and application will be obtained and after the initialization request will occur. The resolved name for the Id will be cached by STATS to avoid future requests. The object will use the P-HAL SERVICES API to tell the STATS daemon to start the initialization procedure. This will, thus, request to STATS MANAGER to access this resources and provide the useful data. The file where the configuration is defined is not known by the object neither the STATS daemon. This is only known by the STATS MANAGER. The request will be done to an object name. The files where this statistics or parameters are configured are hold by STATS MANAGER and designed by the application designer. In this file, a resource Id is associated for every object name within an application context. The association of an object name and application with an object Id is done during the Application Loading Procedure.

Packets Used

Object Name Request Packet Object Name Answer Packet Stats Initialization Packet Stats Definitions Packet

95


P-HAL Stats Structure


Procedure Guide

I. Object executes InitStatsFile() or InitParamsFile() SOFTWARE API functions. II. P-HAL Services sends a message to STATS daemon indicating the Object id that did the request. III. STATS daemon builds the Object Name Request Packet and sends to parent SW MANAGER IV. STATS receives the Object Name Answer Packet and keeps the object and application name associated with the object id. V. STATS daemon builds the Stats Initialization Packet and sends to parent STATS MANAGER. VI. STATS MANAGER reads and parses requested configuration file, saving the data. In case of statistics initialization, answer with a Stats Definitions Packet (the requestor processor Id is got from previous packet). In case of parameters don’t do anything. VII. STATS receive the statistics definitions (if it’s the case) saving the data into a local database (also saves Id of the object for further faster access).

Important Notes Object Name Request Packet

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_SWMANAGER

2

DST

Local Id

3

CMD

OBJ_NAME_REQ

4

PEID

Local Id

5

OBJECTID

96

Id of the object

Object Name Answer Packet

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

2

DST

3

CMD

TYPE_IS_STATS PEID in last Object Name Request Packet OBJ_NAME_ANS

4

OBJECTID OBJ_NAM E APP_NAM E

5..10 11..17

Stats Initialization Packet

Name of the object (24 chars) Name of the application (24 chars)

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_STATSMANAGER

2

DST

Local Id

3

CMD

STATS_INIT or PARAMS_INIT

4

PEID

Id of the processor where the object is running

5..10 11..17

Stats Definitions Packet

Id of the object

OBJ_NAM E APP_NAM E

Name of the object (24 chars) Name of the application (24 chars)

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_STATS

2

DST

PEID in last Stats Initialization Packet

3

CMD OBJ_NAM E APP_NAM E STATS_ST RUCT

4..9 10..16 16... (16+7N)

97

STATS_DEFINITIONS Name of the object (24 chars) Name of the application (24 chars) N concatenated Stats Structures (of size 7 words)

Getting Parameters Procedure Originator daemon

STATS (by SERVICES command)

Destination daemon

STATS MANAGER

Description

Once the resources has been read or allocated, the object may get the parameter value. Analogously to the previous procedure, it will be launched after the object calls a P-HAL Software API function. The command will be forwarded to the STATS daemon who will send the request to STATS MANAGER, the entity in charge of maintaining these parameters values. From now over, the SERVICES daemon will request to the STATS daemon in terms of object Id (not object name) in order to speed up the access to these statistics (they may be used during execution time so minimum overhead is desired). Note that during the previous procedure, the STATS daemon saved the Object Id and name who did the request, then, it can translate from one to another (STATS MANAGER is never aware of what Id has every object, only the names are known).

Packets Used

Parameter Get Packet Parameter Answer Packet

Structures Used by Packet Hardware Library API used Functions

Procedure Guide

I. Object executes GetParameter() function II. P-HAL Services sends a message to STATS daemon indicating the Object Id that did the request. III. STATS daemon builds the Parameter Get Packet and sends to parent STATS MANAGER. IV. STATS MANAGER gets the parameter value from the Paramet98

ers Configuration Resource and sends the value using the Parameter Answer Packet V. STATS receive the value forwarding it to proper SERVICES daemon that did the call (the object, indeed). Important Notes Parameter Get Packet

Parameter Answer Packet

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE


2

DST

Local Id or Parent Id

3

CMD

PARAM_GET

4

PEID

Id of the processor where the object is running

5..10

OBJ_NAME

Name of the object (24 chars)

11..16

APP_NAME

Name of the application (24 chars)

17..23

PARAMNAME

Name of the parameter (24 chars)

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_STATS

2

DST

PEID in last Parameter Get Packet

3

CMD

PARAM_VALUE

4..9

OBJ_NAME


10..15

APP_NAME


15..20

PARAMNAME

Name of the parameter (24 chars)

20..(20+N)

PARAMVALUE

N words containing the parameter value

99

Stats Report Procedure Originator daemon

COMMAND MANAGER

Destination daemon

STATS, STATS MANAGER

Description

During the normal execution (P-HAL Status RUN), the user may require the report of a stats value. This is done by issuing a command through COMMAND MANAGER who will tell STATS to initiate the report process. As we want to minimize the overhead introduced to object processing, the stats values are duplicated in both the object memory and the stats database memory. The STATS daemon maintains a local database of the last stats values introduced by the objects so they can be provided to STATS MANAGER under request without disturbing the object. This means that every time an object uses a P-HAL Software API function to modify a variable value, P-HAL Services daemon must send this new value to STATS daemon in order to update its replicated table. There are two actions the user can do: read a statistic variable or modify it. The viewing of the variable is done through simple reports using the following parameters: • Periodicity: Period in time slots of value capture •

Window size: Length of the window to capture

•

Duration: Number of reports to get

This means that once the STATS Daemon receives the command it will capture window size values every periodicity slots during duration time. Every periodicity time slots, the STATS daemon will send the report using the Stats Report Data Packet to STATS MANAGER. Packets Used

Stats Report Request Packet Stats Report Data Packet

Structures Used by Packet 100


I. User enters command to COMMAND MANAGER: >> statget obj_name=”obj” stats_name=”name” window_sz=W periodicity=P duration=D

II. Builds the Stats Report Request Packet and sends to first son STATS MANAGER daemon. III. STATS MANAGER knows in which processor the object is running because the statistics have been previously initialized. Builds the Stats Report Start Packet and sends to proper processor. IV. Every periodicity time slots, STATS daemon sends a Stats Report Data Packet to STATS MANAGER containing the variables values. Important Notes

Stats Report Request Packet

No post-processing has been defined for this stats values reporting, just the methods to get the information. In this implementation, STATS MANAGER prints to standard output the values got from the report, doing nothing with them. Future versions could implement hierarchical stats reporting throughout parents STATS MANAGER or just a normal file dump for post processing with other common statistics tools. WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_STATS

2

DST

0

3

CMD

CMD_IS_STATGET

4..9

OBJ_NAME


10..15

APP_NAME


15..20

Stats Report Data Packet

STATSNAME Name of the variable (24 chars)

21

WIN_SZ

Size of the window

22

PERIOD

Periodicity

23

DURATION

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE


101

Duration

2

DST

Local Id or Parent Id

3

CMD

STATS_REPORT

4..9

OBJ_NAME


10..15

APP_NAME


15..20 20.. (20+W*N)

STATSNAME Name of the variable (24 chars) Concatenated W (window size) valDATA ues of the variable of size N.

102

Stats Set Procedure Originator daemon

COMMAND MANAGER

Destination daemon

STATS, STATS MANAGER

Description

Reading a statistics variable from the user side is also done involving first the COMMAND MANAGER, then the STATS MANAGER and latest the STATS Daemons. Conversely to the last procedure, when the STATS daemon receive the modification order it will have to forward first modify its local variables table and second forward the order to the P-HAL Services daemon so it also maintains updated its table. Hence, as soon as the object uses the P-HAL Software API to access this variable, the new value will be obtained.

Packets Used

Stats Set Request Packet Stats Set Packet



I. User enters command to COMMAND MANAGER: >> statset obj_name=”obj” stats_name=”name” value=”1,2,4,8”

II. Builds the Stats Set Request Packet and sends to first son STATS MANAGER daemon. III. STATS MANAGER knows in which processor the object is running because the statistics have been previously initialized. Builds the Stats Set Packet and sends to proper processor. IV. STATS receives packet, updates local table and forwards request to SERVICES daemon. 103

Important Notes Stats Set Packet

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

TYPE_IS_STATS

2

DST

PEID got from the Initialization Procedure

3

CMD

CMD_IS_STATSET

4..9

OBJ_NAME


10…15

APP_NAME


15…20 20… (20+N)

STATSNAME Name of the variable (24 chars) DATA

104

N words containing the variable data

Synchronization Procedure Originator daemon

SYNC

Destination daemon

SYNC MASTER

Description

Synchronization between a set of platforms using totally different clock architectures is not an easy challenge. The platforms can use any internal time slot generation, at the required (or possible) precision. At P-HAL level, time is expressed using the P-HAL Time Structure which mainly counts the seconds and microseconds since P-HAL was started (the beginning time is not important at all). So, the time precision is 1 µs. Although the platform time slot duration may be much bigger than this precision, another clock could be internally used to generate the desired time slot rate. In such case, less error will be committed when propagating time among several platforms. Time is propagated to the rest of the Environment using a simple CLIENT to SERVER architecture: the CLIENT (SYNC) requests the time to the SERVER (SYNC MASTER) who answers with its local time. At the end (adding a propagation time uncertainly error) all the platforms will have the same P-HAL Time. The CLIENT supposes a symmetric round trip time, so adding half of it to the received SERVER time leads to a reduction in the uncertainness. In order to achieve this symmetry, the request must be processed as quickly as possible by the SERVER daemon (SYNC MASTER daemon). Therefore, the SYNC MASTER daemon will use a Hardware Library API to define the input interface as URGENT. When this option is activated, the Process is interrupted by a call to the daemon Callback Function which should process the request packet. Thus, the processing time is minimized. Finally, the entire packet interchanging between SYNC and SYNC MASTER is done through dedicated synchronization interfaces so we minimize both the propagation time for synchronization packets and the overhead introduced to the rest of the interfaces.

105

Packets Used

Synchronization Register Packet Synchronization Register Answer Packet Synchronization Request Packet Synchronization Time Packet


P-HAL Time Structure


void get_time(time_t *t);

Procedure Guide

void set_time(time_t *t); int PHAL_SetOpt_itf(int itf, int opt, int args);

I. On boot, SYNC sends a Synchronization Register Packet to request URGENT information to the MASTER. II. Information is received (Synchronization Register Answer Packet) and Hardware Library API is used to set URGENT options for the interface. III. SYNC daemon generates a Synchronization Request Packet and sends through the synchronization master interface. IV. SYNC MASTER receive packet through a synchronization slave interface, fills Synchronization Time Packet with local time (got from Hardware Library API) and sends back through the same interface. V. SYNC receives packet and sets local time using Hardware Library API.

Important Notes Synchronization Register Packet

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

SYNC_CONN_MAGIC

Synchronization Register Answer Packet

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

SYNC_CONN_OK

2

ITF_ID

Argument to use in when setting URGENT option

106

Synchronization Request Packet

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

SYNCHRONIZATION_REQUEST

Synchronization Time Packet

WORD

FIELD

VALUE/DESCRIPTION

1

TYPE

SYNC_TIME_MAGIC

2..4

TIME_STR UCT

P-HAL Time Structure containing the SERVER local time.

1.19.1. P-HAL Structures Definition In this section all the previously seen structures are defined: P-HAL Processor Identification Structure

P-HAL External Interface Structure

P-HAL Object

WORD

FIELD

DESCRIPTION

1

PLT_FAM

Platform Family Identification

2

PROC_FAM

Processor Family Identification CPU Processing Resources (MIPS) represented in Standard IEEE floating point.

3

CPU_RES

4

OPTS

WORD

FIELD

1

ITF_TYPE

2

ITF_ID

3

BW

4

DELAY

Propagation time in microseconds

WORD

FIELD

DESCRIPTION

107

Other optional data

DESCRIPTION Interface Type Identification Interface Identification Interface BW in MBPS represented by a Standard IEEE floating point.

Structure

P-HAL Interface Structure


P-HAL Time Structure

1...4

OBJ_NAME

5

OBJ_ID

6

PROC_REQ

MIPS requirement expressed in Standard IEEE floating point

7

ITF_IDX

Starting interface index in interface database

8

NOF_ITF

Number of Interfaces

WORD

FIELD

DESCRIPTION

1

ITF_ID

Interface Id

2

REMOTE_ID

3

EXT_ID

4

MODE

5

BW_REQ

6

DELAY

WORD

FIELD

1

PLT_FAM

2

MODE

3

PROG_BLEN

4

PROG_WLEN

5

MEM_ST

6

MEM_SZ

WORD

FIELD

DESCRIPTION

1

TV_SEC

Unsigned Integer indicating the number of seconds

108

String of 24 characters Unique Id for the Object

Interface Id in remote object Id of the external data interface if needed 1 for input, 2 for output Standard IEEE Floating point number representing the bandwidth requirements in MBPS Delay in timeslots of the interface, if applicable

DESCRIPTION Platform Family Identification Loading modes, RELOCATED or NON-RELOCATED Length of executable file in bytes Length of executable file in rounded up word Start address to be relocated to (if needed) Length of memory used by the program

2

Unsigned Integer indicating the number of microseconds

TV_USEC

1.20. P-HAL Global Constants In this section the P-HAL Global Constants will be enumerated and defined. It is important in order to keep inter-platform compatibility that every P-HAL implementation uses the same constants when implementing inter-daemon procedures (daemons exchanging packets between them) or any kind of communication between P-HAL entities either locally or remote. The section is organized in two tables: the first one (Table 4-7) describes the constants involved in internal daemon-to-daemon command communications; the second one (Table 4-8) describes the constants involved in the packet oriented commands used for external (sometimes internal) communication between daemons. Table 4-7: P-HAL Global Constants for Commands

Constant

Value

Description

GLOBAL SET_PE_ID

0x0001

Tells the local PEID and the Parent ID to the rest of the daemons

FRONT-END GET_FIRST_SON

0x1001

Gets the Id of the first son for an slave control interface

SND_CTRL_PKT

0x1002

Delivers a control packet to a local daemon or sends through external interface

SWLOAD GET_OBJ_DATA_ITF_ID

0x2001

Obtains the interface Id assigned to an object

DEL_OBJ_DATA_ITF_ID

0x2002

Deletes an object data interface

SWMAP NEW_LOCAL_NEIGH

0x3001

Identifies a new discovered neighbor

STATS INIT_PARAMS_FILE GET_PARAM INIT_STATS_FILE

0x6001 0x6002 0x6003 109

Initializes parameters file Gets a parameter value Initializes statistics file

INIT_STAT CLOSE_STAT CLOSE_OBJECT SET_STAT

0x6004 0x6005 0x6006 0x6007

Initializes an statistic variable Removes a variable from database Remove all variables used by an object Modifies a variable value

EXEC REGISTER_OBJ REMOVE_OBJ GET_OBJ_PID GET_OBJ_ID SET_OBJ_STATUS GET_OBJ_STATUS

0x7001 0x7002 0x7003 0x7004 0x7005 0x7006

Registers an object to local database Remove object from database Returns object PID Returns object Id Set object status Get object status

BRIDGE UPDATE_BRIDGE_TAB

0x8001

Creates a new entry in the bridging table

SYNC_MASTER SYNC_REQUEST_PKT

Receives a synchronization request packet

0x9001

Table 4-8: P-HAL Constants Values for Packets

Constant

Value

Description

GLOBAL TYPE_IS_CONNECT TYPE_IS_SWLOAD

0xA000

Connection packet

0xA002

Packet addressed to SWLOAD

TYPE_IS_SWMAP

0xA003

Packet addressed to SWMAP

TYPE_IS_SWMANAGER

0xA004

Packet addressed to SWMANAGER


0xA005

Packet addressed to STATSMANAGER

TYPE_IS_STATS

0xA006

Packet addressed to STATS

TYPE_IS_EXEC

0xA007

Packet addressed to EXEC

SWLOAD LOAD_OBJ LOAD_START LOAD_PACKET LOAD_OK EXEC_INFO_ANS GET_EXEC_INFO

0x2101 0x2102 0x2103 0x2104 0x2105 0x2106 110

Application Loading Procedure Executable Loading Procedure Executable Loading Procedure Executable Loading Procedure Executable Loading Procedure Executable Request Procedure

EXEC_REQUEST LOAD_APP STATUS_CHANGE

0x2107 0x2108 0x2109

Executable Request Procedure Application Loading Procedure Object Status Change Procedure

SWMAP MAP_OBJ

0x3101

NEW_NEIGHBOUR

0x3102

EXEC_INFO_ANS

0x3103

ADD_SON_PE

0x3104

Hierarchical Software Mapping Procedure Data Connectivity Matrix Identification Procedure Executable Loading Procedure Hierarchical Processor Identification Procedure

SWMANAGER GET_EXEC_INFO EXEC_REQUEST LOAD_APP

0x4101 0x4102 0x4103

OBJ_NAME_REQ

0x4105

STATUS_CHANGE

0x4104

Executable Request Procedure Executable Request Procedure Application Loading Procedure Stats and Parameters Initialization Procedure Object Status Change Procedure

STATSMANAGER PARAM_INIT

0x5101

PARAM_GET PARAM_VALUE

0x5102 0x5103

STATS_INIT

0x5104

CMD_IS_STATGET STATS_REPORT CMD_IS_STATSET

0x5105 0x5106 0x5107

Stats and Parameters Initialization Procedure Getting Parameters Procedure Getting Parameters Procedure Stats and Parameters Initialization Procedure Stats Report Procedure Stats Report Procedure Stats Set Procedure

STATS STATS_REPORT_START

0x6101

OBJ_NAME_ANS

0x6102

STAT_SET_VALUE

0x6103

Stats Report Procedure Stats and Parameters Initialization Procedure Stats Set Procedure

EXEC PHAL_STATUS_RUN PHAL_STATUS_PAUSE PHAL_STATUS_STOP PHAL_STATUS_INIT

0x7001 0x7002 0x7003 0x7004

111

Object Status Change Procedure Object Status Change Procedure Object Status Change Procedure Object Status Change Procedure

User Guides This section will serve as a guide on how to deal with P-HAL. Three different points of view can be defined when we focus on how P-HAL interacts with the user. We will define three entities or three kinds of users that do three kinds of different tasks and (this is one of the objectives) can be totally unaware of the others actions: 112

•

•

•

P-HAL Platform Designer: This user is in charge of the correct behaviour of the whole P-HAL Platform (environment). This includes: identify the set of PE running P-HAL, the kind or family of such PE, how they are connected, etc. The hardware design part as well as the definition of its characteristics would be carried out by the platform manufacturer; however, this task is assigned to this user as he is the one who has to ensure of the whole Platform functionalities. In a similar way, the design of the Software Daemons is also included in this phase. To sum up, this user provides the hardware and software elements required to work with a P-HAL Platform. Radio Application Designer: This one will define which objects are needed to accomplish the application objective, how they are linked and where they can be found, adjust their initialization parameters depending on the desired behaviour, etc. Object Algorithm Designer: Finally, this last user will write the flat algorithm (objects) as well as define its available parameters and how they influence in its behaviour.

Then, this guide will be split into three chapters each of them serving as a guide for each kind of user. The reader will note how these three tasks are separated at the user definition phase but linked by the underlying P-HAL Framework during the execution phase.

1.21. P-HAL Platform Designer User Guide In the following sections we will see what constraints or aspects must be taken into account and what rules one should follow when interconnecting several PE. These constraints affect first of all to the assignment of which set of Software Daemons is going to be available in every PE, then, the interconnectivity of these PE. To sum up, a short step-by-step guide will be presented as a summary of the entire chapter.

1.21.1. Software Daemons Mapping Constraints Taking into account the tasks that P-HAL Software Daemons do, we can easily realize that it is not arbitrary to place them in one PE or another. Some Daemons may need some PE dependant resources, some others may be required by another daemon. These constraints shall be summarized as follows: • Processing architecture: The daemons may need a complex state-control machine to achieve its task. So, if we have programmable and non-programmable devices (DSP and FPGA respectively) or even analogue devices (Codecs or filters) it would be a good decision to run these kinds of tasks (complex Software Daemons) in common programmable devices as they are better designed for such tasks. •

Memory constraints: Similar as before, there are some Software Daemons that shall require different types of memory sizes or access. Again, some processing objects may not require such type of memory, so reserving determinate PE to run processing objects and assign others to complex Software Daemons would be a good politic. 113

•

Non-volatile data access: Other daemons must access some non-volatile data (local or remote hard-disk files, external ROM memory, etc.) to read some fundamental user configurations or even to read the commands to execute. The daemons who need these kind of resource are: o COMMAND MANAGER: Reads user commands. o SW MANAGER: Reads application configuration and object executables. o STATS MANAGER: Reads stats definitions and parameter values.

•

Daemon dependencies: There is only one inter-daemon dependency, concerning the running of objects. If one PE is going to run objects, the following daemons must be present in it: o SWLOAD: Used to load the executable and map local interfaces o EXEC CTRL: Control the execution state of the object o STATS: Only needed if the object is going to use statistics or initialization parameters.

•

Special Hardware Library Features: Finally, the SW MANAGER daemon may require the Hardware Library to support some functions regarding the building or relocating of an executable.

1.21.2. P-HAL Platform Interconnection Constraints P-HAL separates the control traffic of the data traffic and the synchronization mechanisms. This separation is done to accomplish one of the objectives of P-HAL, that is, to minimize the overhead introduced in the execution plane. Thus, three types of interfaces are defined in P-HAL: • Bi-directional control interfaces. •

Bi-directional synchronization interfaces

•

Uni-directional data interfaces

This leads to the co-existence of three different networks in the same P-HAL Platform: The Control network, the Synchronization network, and the Data network. In Figure 516, the control and data planes are drawn to illustrate this idea.

114

Figure 5-16 – Control and Data Plane in the same P-HAL Environment

1.21.2.1. Control Network Constraints The functionalities of the Software Daemons running on each PE are the main constraint of the Control Network. This network is used by daemons to exchange messages between them. We can group these messages (packets) depending on their context: • Packets requesting a service •

Packets issuing a command

Thereby, the daemons requesting a service should be able to easily reach the daemons who will serve them. Conversely, the daemons executing a command to another set of daemons should be also able to reach its destination. As in most of the cases the set of daemons executing commands and the set of daemons requesting the services are known (SYSTEM MANAGER Daemons normally provide services to the rest of daemons), we can easily define this dependence using hierarchical network architectures. On the top of the hierarchy we will place the daemons who will answer the commands requested by their sons. In order to define the direction of the hierarchy, P-HAL defines two types of control interfaces: • Master Control Interface: It connects to the upper level in the hierarchy. It is where are supposed the commands to come, and towards are supposed the requests to go. Only one can be present in the Platform. •

Slave Control Interface(s): These are the interfaces connecting to the lower level. There can be more than one as this is how the tree is built. 115

This few rules give us a lot of flexibility when designing the control network. The only constraint we have to follow are that if some daemon is requesting a service, it must find the daemon able to serve him anywhere upwards in the tree. Conversely, a daemon issuing a command must find the destination daemon downward in the tree. In Table 5-9 we show these relationships between daemons: the Server daemon should be then, always, upwards in the tree. Next, in Table 5-10 we show the opposite: the Executer daemon must be always in a lower level than the Commander daemon. Table 5-9: P-HAL Software Requestor-Server inter-Daemons Relationship

Requestor daemon(s) STATS SWLOAD

Server daemon(s) STATS MANAGER SW MANAGER

Table 5-10: P-HAL Software Commander-Executer inter-Daemons Relationship

Commander daemon(s) STATS MANAGER SW MANAGER COMMAND MANAGER

Executer daemon(s) STATS SWMAP/SWLOAD SW MANAGER STATS MANAGER EXEC

Note that the hierarchy does not impose restrictions in the number of hops to get the Server or the Executer. This means that if we have for example several STATS Daemons, we do not need one STATS MANAGER Daemons able to serve all of them. The only constraint is that any of all the available Server Daemons is able to answer to my request. In a similar way, we do not need all the Executer Daemons to be lower than all the Commander Daemons; but only the daemons under the Commander daemon will answer to its command, not the upper ones.

116

Figure 5-17 – Daemon Placement in a P-HAL Environment (Example 1)

In Figure 5-17 we can see an example where COMMAND MANAGER reads the command “set stat for object A and D”. It sends the packet down to the tree. The first PE does not have STATS MANAGER, so packet keeps going down. PE2 can process the command for object D but not the other one. Command for object A keeps going down. Finally, PE4 will be executing a STATS MANAGER daemon and will be able to serve the command for object A, after it is processed, the command procedure ends.

117

Figure 5-18 – Daemon Placement in a P-HAL Environment (Example 2)

Analogously, in the example 2 (Figure 5-18), a Platform requests the executable OBJECT C. Again, the first SWMANAGER found does not own the C object, so packet keeps going until P2 is reached. There, a SWMANAGER owning the object C is able to process the command. The daemon will start the downloading process of the executable. The routing scheme would be the same as in example 1 (downwards) although it has not been drawn for clearness. The final thing that the architecture designer must be aware of is related to SWMAP. This daemon is used to map the objects to load into the resources it is in charge. The designer can place the SWMAP daemon anywhere in the tree. The only thing to take into account is that this daemon hides the architecture of what it has under it to the other SWMAP who is over it.

118

Figure 5-19 – Hierarchical Resource Identification

In Figure 5-19 we see an example of this concept. The highest PE in the tree sees under it 3 PEs, 2 with 1000 MIPS and another with 2100 MIPS. Altogether, concerning another (not shown) SWMAP over it, he has a total of 4200 MIPS. The same happens with one of its sons, which had 2100 MIPS. Actually, it runs a SWMAP (although the previous does not sees) and it is hiding the architecture under it. Under this second PE of 100 MIPS running a SWMAP, there are 3 more PEs with 1000 MIPS everyone.

119

Figure 5-20 – Hierarchical Software Mapping

Then, the first SWMAP will do the mapping taking into account only a total of 4 PEs, without knowing that one if these PEs have another 3 sons more. This idea is illustrated in Figure 5-20 where a command for loading 29 objects is received. The first iteration of the algorithm (PE1) will map this set of 29 objects into its 3 PEs under it. One of these, will receive an order to load 12 objects (more than the rest because it has more total power) and as a SWMAP daemon is present, another iteration of the algorithm will take place, mapping these 12 into the three PEs available. The mapping algorithm ends here as no more PEs with a SWMAP daemon exists under this level. 1.21.2.2. Synchronization Network Constraints The importance of the presence of a synchronization mechanism in a P-HAL Environment has been deeply presented in previous chapters. In this chapter, we will discuss about how the interconnection of the PEs affect to the synchronization requirements. As explained in previous chapters, synchronization in P-HAL is accomplished following a CLIENT to SERVER architecture. CLIENT sends a Synchronization Request Packet and SERVER replies with a Synchronization Time Packet containing its local time. Then, the CLIENT calculates the server time by adding an estimation of the propagation time as TR/2 (where TR is the Round Trip Time, supposed symmetric) and then reducing the uncertainness to this value, TR/2. Hence, time is propagated from SERVER to CLIENT with an error of T R/2. No multi-hop is supported when concerning this kind of packets, thus, when covering a network with more than 2 PEs, some kind of network must be chosen. Several architec120

tures can be implemented. In every case the error we commit between in clock synchronization must be strictly analyzed. In the following paragraphs we will discuss how to do it. Let’s define the TR matrix:

 0 TR12 TR13  0 TR 23  TR 21 0 TR =  TR 31 TR 31      T  R N 1 TR N 2 TR N 3

 TR1N    TR 2 N   TR 3 N       0 

Where TR ij is the Round Trip Time found between PE i and j in a general network of N Platforms. This time has a resolution of 1µs and is calculated from the moment the CLIENT calls a function to send the synchronization request packet, until the moment it receives the answer through, again, a function call. Now let’s define the uncertainness or the error between a CLIENT and a SERVER as:

TE ( l , k ) =

TR ( l , k ) 2

Thus, the existing error between clocks in PE i and PE j will result in adding all the errors between the CLIENT and SERVERS found in the trip of M stages from i to j: M −1

TE ( i, j ) = ∑TE ( l , k )

;

l=server and k=client in stage m

m =0

In Figure 5-21 we see an example network, and follows and example of how to calculate the error:

Figure 5-21 – Example Synchronization Network

121

Where the TR matrix would be like: 0  46 TR =  ∞  ∞ 80 

∞ 0 120 ∞ ∞

∞ ∞ 0 100 ∞

∞ ∞ ∞ 0 ∞

∞  ∞ ∞ µs  ∞ ∞ 

Then, the error between clocks of element 5 and element 4 will be: TE ( 5,4 ) = TE ( 5,1) + TE ( 2,1) + TE ( 3,2 ) + TE ( 4,3) =

80 46 120 100 + + + = 173 µs 2 2 2 2

Steps are obtained as follows: m=1: CLIENT=5; SERVER=1 m=2: CLIENT=2; SERVER=1 m=3: CLIENT=3; SERVER=2 m=4: CLIENT=4; SERVER=3 As the time slot duration is constrained by the error committed in the synchronization mechanism the network architecture we choose or, broadly, where synchronization interfaces or daemons (SYNC and SYNC MASTER) must be mapped is a matter to be concerned about. 1.21.2.3. Data Network Constraints Data Interfaces do not have any constraint at all about its interconnection. The Designer can connect the PE following any other physic or implementation related constraints. The network will be automatically discovered and objects will be mapped according to it.

1.21.3. P-HAL Environment Step-by-Step Design Guide Now we will summarize the last sections in a short guide to help the P-HAL Platform Designer to define the architecture of the whole environment: 1. Define the total processing power we want to reach, and enumerate the total PE available in order to reach this power. 2. Decide which PE will do management tasks, which will do processing task and which will do both tasks. 3. Apply time constraints and define how the synchronization network will be like. 4. Apply Software daemon Constraints to decide where to run each needed Software daemon. 122

5. Assign to every PE the set of daemons it is going to execute, taking into account the service requests and commands executing rules. 6. Decide how the mapping of the application objects we want to be. That means to decide where to place SW-MAP daemon(s). 7. Build the hierarchy by defining master control interfaces and slave control interfaces. 8. Define the data interfaces between the PE able to execute application objects. 9. At this point, any new PE can be included in the Environment by simply connecting to any slave interface at any point (decided following the previous rules).

1.21.4. Writing a P-HAL Software daemon The set of functionalities provided by the Framework are not a closed definition. Future improvements on their capabilities, or even new functionalities could be introduced by P-HAL manufacturers but basic guidelines defined in the implementation chapter should be always maintained in order to keep back-wards compatibility. A P-HAL SW daemon is defined as a piece of software providing a set of services which execute an action after the reception of a command or a packet. This piece of code will usually be executed as a stand-alone process but in some cases a periodic call to a function could be used. The range of possible commands should be well defined by the manufacturer when writing it so the rest of daemons or users can access its services. As “well defined” we understand to provide the list of words expected to be received for every behaviour (service) and the list of words that will be answered if the case. Services offered by a P-HAL SW daemon will be requested in all cases by another SW daemon. Only a special daemon, the P-HAL Services daemon receives commands from the application objects through an API, even though a request to other daemons will help to process most of the services. This information exchange between SW Daemons is done using two mechanisms provided by the P-HAL HW Library: •

Commands: Uses the HW Library to send a packet to another SW daemon registered within the same HW Library. Note that only local communication can be achieved with this mechanism. A special word (see function reference) is defined to help receiver to quickly identify the command type. The daemon name is used to indicate the destination of the command.

•

Packets: Uses a packet interface to send (or receive) packets to another SW daemon. In this case, the other daemon can be either local (in the same PE) or remote (in such case, an assignation to an external interface should be used).

123

The first method is intended to be used for short command requesting. It has the option to halt the execution thread of the caller SW daemon until the called daemon processes the command and writes an answer. So, quick messages or commands can be easily sent between daemons using these functions. The HW Library has two methods of calling a SW daemon to process the command. Both methods are transparent to the SW daemon writer and its code should have no dependence on this election. The mechanisms are: • Push commands: The HW Library executes a callback function defined at the initialization of the daemon. Command is immediately processed in this case. If an answer is requested, the message will be sent back as soon as the callback function is exited. •

Poll commands: The daemon regularly calls a HW Library function in its main execution thread which will again call the same callback function. Again, the answer will be sent back as soon as the function is exited.

The introducing of these two methods is intended to make easy the implementation of HW Libraries in such different sort of platforms. In some of them, sending short messages may be an easy solution (implemented by lower OS layers) and Push method would be preferred. However, this method can be slow or difficult to implement in other architectures, and an immediate call to a callback function to process the message would be preferred to avoid the use of extra buffers or memory to interchange packets. Note that as SW daemon must be independent of the HW Library implementation, the writer of the daemon must be aware of concurrent access to memory variables, so defining ways to avoid such situations is under the programmer responsibility. On the other hand, a packet interface can be used to exchange information between daemons. This kind of interface, however, is designed to allow higher data rates but indeed, the same behaviour of the previous method could be achieved if desired while in this case, packets are written to an interface and destination daemon must have been previously attached to the same one in order to be able to read the packets. Thus, in the case the designer desires to use this method, a previous interchange of commands should be used to communicate the interface id to the destination daemon. Anyway, a SW daemon manufacturer should provide when designing the daemon the following information: • Full list of services, indicating for each of them the command identification word as well as the contents and the answer content. •

For every service, which mechanism is used to serve it: command interface or packet interface. In case of packet interface, how it can be attached by remote daemons.

•

The set of daemons required to be running in the same Platform (dependencies).

As defined in previous chapters, depending on the Platform architecture the SW daemon code will be launched by a single process or by a periodic routine call which will execute such piece of code. Thus, the HW Library defines a global constant, named HWL124

which defines if the daemon should include a main() routine or not. In any case, the daemon manufacturer must provide the name of the routine. IB_RUN_AS_PROC

This concept is well seen in Figure 5-22, where a schematic view of a SW daemon source code using the HW Library API is presented. Figure 5-22 – SW daemon example code. #include #include #include void DAEMON_Run(); int DAEMON_Process_cmd(int cmd, int *buffer, int len); #ifdef HWLIB_RUN_AS_PROC int main() { DAEMON_Run(); } #endif int initialized=0; void DAEMON_Run() { callback_t f; if (!initialized) { f.callback=DAEMON_Process_cmd; PHAL_API_Init(“daemon_name”,&f); /* do more initialization tasks */ /* create interfaces if need to be used */ initialized=1; } while(1) { /* check other daemons commands */ PHAL_Cmd_chk();

I

/* receive from some interface (optional) */ n=PHAL_rcv_packet(fd_r,packet,BUFF_SZ); if (n>0) { Processs_packet(packet,n); } /* do more tasks */

} } int DAEMON_Process_cmd(int cmd, int *buffer, int len) { switch(cmd) { case CMD_IS_ONE: /* process */ return (result); case CMD_IS_TWO: /* process */ return (result); break; } }

1.21.5. Hardware Library API Reference Prototype

int PHAL_API_Init(char *daemon _name, callback_t *f);

125

Parameters

Returns

daemon_name

Name of the daemon previously registered.

f

Pointer to the callback function for command processing.

>0

If successful If error

0 0

If daemon is present If daemon not present If error

0 0 0

If deleted If error

0 0 0

If successful If error 129

Returns

0 =0 0 =0 0

Number of words received No data is available If error

Returns

=0 0

Number of external interfaces in Platform If error

Returns

0


0


0


0 0 0


0


0 0


0 =0 0 =0 Dispatch the task -> Return”

The structure an object must follow is represented in flow diagram showed in Figure 524. The diagram shows as every module must announce its presence to P-HAL and after that wait the order to execute (STATUS). The first order the object will receive will be to initialize (INIT) itself. Then, the object will read the initialization parameters (using the P-HAL Software API), activate the communications links with other objects and declare the statistics variables. This procedure is just executed once in the life of the object.

142

If an execution order is received (RUN), the object will receive the pending message, dispatch the required task (for example process a block of data and send the result through another interface) and return to the initial point. The last case is the reception of the order to close (STOP) the object. In such case, the object must tell to P-HAL to close all the allocated resources and finish its execution. Note that the object owns the CPU during the time it process the message. After that, the object returns to the STATUS point handing the CPU over again to P-HAL. In anomalous or time rules violation, P-HAL may release the ownership from the object before returning to the STATUS point.

START Register to P-HAL Status INIT Read Configuration Setup Communications Setup Statistics STOP

Status RUN

Close Resources

Unregister P-HAL

analyse message if any dispatch task if any return

Tx

EXIT

Figure 5-24 – P-HAL Object Flow Diagram

1.23.2. Executing Time Radio applications maintain a big dependence with the time. The information changes at a regular frequency, the instant when this information must be transmitted is one and only one, etc. However, among the whole document, a framework where the object or the programmer is not concerned about the time reference or even the time slot duration of the platform where the application will be execute has been presented. The consequence of this ignorance about the time control mechanisms is that the objects can not react to hardware events immediately. The reaction will be then, quick enough. In every time slot, every object will have its opportunity to process all the messages other objects sent to him in previous slots. The environment do not run at Real Time then, it just becomes necessary to run really fast. This only depends on the computing power 143

and the algorithm complexity. P-HAL may even allow that, due to a punctual CPU peak demand, the object ends its tasks after the end of the time slot. In effect, this is a violation of P-HAL time rules but will not become a problem if the excess of CPU in future slots can compensate the previous peak.

1.23.3. Describing an Object The programming language used to describe an object may change between different processing devices, depending on the tools provided by the device manufacturer. However, it would be very interesting to use always the same language so the code can be reused. This is unrealistic, because currently programmable devices as GPP or DSP can use C/C++ while others like logic programmable devices as FPGA use totally different languages (VHDL). Nevertheless, these are the chosen standard programming language for the P-HAL Framework: • C/C++ Language for programmable devices • VHDL Language for FPGA devices Not only the use of a standard language is a requirement to enable the whole set of functionalities provided by P-HAL. Also the use of the P-HAL Software Library API is a requirement as well as other considerations. An Object writer must be concerned about the misinformation of the final platform to be executed on. This means that very important concepts (in software radio) like sampling frequency, time slot duration, memory distribution or architecture, etc. are totally unknown when objects are designed. Thus, it would need to support such sort of frequency ranges and architectures under parameter reconfigurations. A special function provided by the P-HAL Software Library API will serve for this purpose. The number of samples to be generated every time the object is executed (every time slot) will be obtained using the following function: int GetTempo(float freq);

This function receive has a parameter the frequency the object desires to operate. As the environment (P-HAL) is the only who knows the time slot duration of the PE, a simply operation will be done to calculate the number of samples to be generated at that precisely moment. The amount of samples that the object needs at its input will depend on the algorithm, more precisely, in the rate relation between the input and output frequency. Anyway, the object will wait until all the data it needs is available. Note that this could lead to time violations if frequency relations between collateral objects should be defined with caution.

The following concepts should be taken into account when designing an object: 144

•

Sampling frequency and time slot is unknown: the length of input and output packet (packet length divided with the time slot results on frequency) should be allowed to change and should be limited.

•

Processing time over data unit (cycles/sample) should be taken into account when defining the required processing resources depending on the operating frequency.

•

The format of the input and output data has to be well defined.

•

The values acquired from parameters initialization should always be checked, to ensure they follow our local constraints.

Finally, in the next figure (Figure 5-25) a sample code of a P-HAL object can be seen for illustrating purposes: Figure 5-25 – Schematic source code for a P-HAL Object #include #include int data_in[128],data_out[128]; int fir_coef[12]; void main() { int status; int fdi,fdo; int rcv_len; int long_block; int nof_coef; float freq; int stat_outsignal; switch(status) { case PHAL_STATUS_INIT: /* init interfaces */ fdi=CreateFlow(0x1,FLOW_READ_ONLY); if (fdi