Towards a REST-style Architecture for Networked Vehicles and Sensors

Towards a REST-style Architecture for Networked Vehicles and Sensors José Pinto, Ricardo Martins, João Borges Sousa Department of Computing and Electrical Engineering School of Engineering, Porto University Porto, Portugal {zepinto,rasm,jtasso}@fe.up.pt

Abstract—In this paper we describe our efforts towards the creation of an open web-based architecture for controlling the behavior of systems composed by static sensors, mobile sensors and unmanned vehicles. We document the initial implementation of the architecture and its application to our hardware devices. Despite being still in its infancy the architecture and implementation have already been validated by real-world deployments involving drifting sensors, unmanned vehicles and wireless sensor networks, having one or more human operators controlling the overall behavior of the system. Keywords-Network Robot System, Architecture, Robotics, Wireless Sensor Networks

I.

INTRODUCTION

Recent advancements in micro-electronics, computing and communications technology have generated a wave of interest over the creation of systems composed both by robots and environment sensors, acting together according to the needs of human operators. As with any emerging technology, there are currently different definitions and approaches to such systems including Pervasive Systems, Networked Robots, Cyber-Physical Systems and Network Robot Systems. With this paper we don’t try to impose any approach, being the architecture in this paper applicable to any actuator/sensor networks whose nodes are connected, continuously or from time to time, to other nodes in the same network. The network can be local or can also be a global network (Internet). Examples of these systems are car-to-car networks [1], Wireless Sensor Networks (WSN) [2], Adaptive Sensing Networks [3], among others. In these networks, there exist multiple nodes with capabilities that vary (sense different variables or produce different actions) according to node’s physical properties or according to node’s contextual state. Individual nodes cooperate in order to attain global intended behaviors by exposing their capabilities to remaining nodes. For instance, cars making part of car networks broadcast their velocity to peers in order to prevent accidents; automated sliding doors may let themselves be controlled by robots that want to travel to other rooms; wireless sensors broadcast their readings for allowing aggregated data to be calculated at a WSN’s sink node. Borrowing ideas from Pervasive Computing and Software Architecture, we find it practical and advantageous to look at the capabilities of each node (sensor / actuator / robot) as a set of services that are provided to the network.

All nodes in the network are able to request a service in any of its neighbor nodes (network visibility) while simultaneously providing a set of services. Moreover, humans can also be looked as service requesters / providers the same way as remaining nodes. Take for instance the example when a robot faces a difficult situation and requests a human resolution: the human provides a “resolve situation” service that can be requested by the network whenever an unpredicted state is faced. Following these lines, the Underwater Systems and Technology Laboratory is developing several different unmanned vehicles and sensors, aiming the creation of novel applications of networked robotics. Our vehicles are heterogeneous in that they travel through air, sea surface or underwater according to their type; sensed environment variables also vary according to vehicle. Moreover, we are also using environmental sensors that may be static (embedded in the environment) or mobile (carried by humans or drifting). Our current focus is in building an infrastructure that provides human-tangible interfaces for peer-to-peer (time-varying topologies) networks whose mobile nodes adapt in order to answer to human commands and environment interferences. In this paper we briefly describe our system’s existing control and dissemination architecture and introduce a new architecture that provides means for cooperation, service integration and service discovery recurring only to ubiquitous web technologies. II.

NETWORKED VEHICLES AND SENSORS AT USTL

At the Underwater Systems and Technology Laboratory, we aim to build systems composed of heterogeneous unmanned vehicles, environment sensors and human operators. Already we have created and tested several different robots like Autonomous Underwater Vehicles (AUVs), Autonomous Surface Vehicles (ASVs), Remotely Operated Submarines (ROVs) and Unmanned Air Vehicles (UAVs) which can be seen in Figure 1. Vehicle heterogeneity brings many added applications that exploit vehicle complementarities. However, heterogeneity among sensors and robots increases the system’s complexity to a new order of magnitude since we must cope with hardware, software and communications specificities. Our approach to this problem was to build a modular software system whose components can be shared among different vehicles, sensors and operator consoles.

controlling authority as long as the controlling authority uses a compatible message set for communication (see Figure 2). Layered control concepts External controller Team/task

Vehicle

Mission supervisor

Maneuver commands Configuration commands

Execution events State

Link

Vehicle supervisor Abort Configurations

State Messages

Vehicle maneuvers

Maneuver controller Vehicle commands

Platform

Vehicle Mission

State Messages

Vehicle commands

Figure 1. Unmanned vehicles being developed at USTL Figure 2. Different layers in the vehicle’s control architecture

A. Inter-Module Communication Protocol The communication protocol that we use for real-time mission execution is called IMC (Inter-Module Communication) [4]. The IMC protocol is a message-oriented protocol targeting networks of vehicles and consoles. This protocol consists in a unified specification of message structures (using a single XML file). IMC specification supports message composition (a message field may correspond to an inline message). The message specification can be used to generate bindings for different languages like Java, C++ or C#. Currently, all data between vehicles and consoles (and even between different vehicle's internal tasks) is transferred using the IMC protocol. All code involved in message serialization and deserialization is generated from the definition XML by using a XSLT style sheet that matches the intended language. For the architecture presented in this paper we added a new IMC implementation which is decoupled from the current message set definitions and also supports self-describing message structures (see Section IV). B. Vehicles’ Onboard Software Architecture All USTL vehicles run the same onboard software (DUNE) [4] adapted to individual vehicles by including their specific hardware drivers and vehicle-specific motion controllers. DUNE is composed by various asynchronous tasks that communicate with one another through IMC messages. The fact that we use the same communication protocol inside and outside vehicles makes our system very flexible both because it allows a task to be easily interchanged with another compatible task – a task that understands the same IMC message subset – and also because it allows part of the tasks to migrate to external entities as needed. Take, for example, the Mission Supervisor task; this task can run aboard the vehicle when the vehicle is disconnected from the network or otherwise can run remotely when the vehicle is connected to a

C. Wireless Sensor Networks Unmanned vehicles can be used for acquiring spatially distributed data that changes slowly (e.g. bathymetric profiles) but when the data changes very dynamically or needs to be monitored for long periods of time, one can use Wireless Sensor Networks whose nodes are static and able to operate for long periods of time. At USTL, we use MoteIV TmoteSky devices running a TinyOS [5] application that provides sensor acquisition through multi-hop networking (TinyOS Surge) being possible to reprogram the application through the air (TinyOS Deluge). Using this application, all nodes route information to a sink node which is attached to a base gateway. This gateway receives data from the WSN and translates the data into IMC messages that are stored to disk (text serialization) and sent to the local network, allowing the data to be monitored in real-time at operating consoles and vehicles.

Figure 3. Drifting Sensor (left) and WSN node (right)

1) GSM Sensors To monitor water currents, we use buoys that drift with the water current (drifters) and have onboard GPS. The drifters are based on the low cost TELIT GM862 module which provides GSM and GPRS communications along with onboard programmable microcontroller. We use SMS messaging as an interface for the drifters. For instance, we use SMS to control the drifter’s data acquisition mode (sleeping, fast or power-conservative), to retrieve the current position and to request FTP log uploads. These devices have a fully working TCP stack which makes it possible to exchange data through HTTP and FTP. This happens because a working subset of HTTP and FTP can easily be implemented on top of TCP. III.

A GENERIC DATA DISSEMINATION AND CONTROL ARCHITECTURE

In this section we give a general overview of the needs of our system and guidelines in terms of components and abstraction. Our networks are built of heterogeneous nodes (sensors, vehicles, operator consoles) that connect continuously or sporadically to a network. Nodes can be static, mobile and / or actuated. Nodes have a state that evolves according to environment changes (sensors), executed actions (actuators) and internal configuration. For instance, a vehicle’s state is composed by its sensor states, the mission plan being executed, the map of the environment, the operator that is controlling its behavior, etc. Some internal state variables can be changed by authorized external nodes (like the mission plan) while others can only be read (like vehicle’s position). Moreover, some variables may be very dynamic (sensors) while others never or very rarely change (configurations). The state of each node can be looked as a set of resources that evolve as a function of current state (internal resources) and inputs from the environment (interaction with other nodes and sensors). For instance, an unmanned vehicle’s state should change according to operator commands, onboard sensors and active tasks. We propose using a REST-style approach for accessing and representing data from these vehicle and sensor networks. We suggest that each resource is represented as a list of name-value pairs, as a mean to shorten the number of resources and encapsulate similar data inside the same scope. As an illustration, vehicle’s position can be seen as a resource that includes latitude, longitude and altitude fields. Each node in the network can be queried about its owned resources that can be used. The answer should be a list of resource identifiers and the ways they can be used, depending on the resource nature. Following, we list the different resource natures in our system:  Periodic Data: Read-only resources that are periodically generated from one or more sensors in the network. The application may loose any amount of intermediate states since the most accurate representation of the resource is always considered to be its last version.



Static Data: This data is statically stored inside the node that owns this resource. Usually corresponds to configuration data that is used by the node and possibly is changed by other nodes in the network (when it’s not read-only). Examples of static data are environmental maps, network configurations, etc.  Queries: Queries are not necessarily mapped to stored data. Instead, a query is a logical abstraction for a function that takes as its input data from one or more nodes in the network producing a virtual resource. Queries may be used to aggregate data from several nodes in the network, to list previous states of the resources (historical data), to list supported types of a resource, etc. Along with the possibility to query about a node’s capability, there should also be possible to disseminate this information throughout the network either by peer-to-peer broadcasting or by using repositories that can be “queried” about network resources. IV. IMPLEMENTATION In this section we describe a new implementation data dissemination and control based on ubiquitous web technologies and agnostic towards a serialization format. In our approach, the resources are serialized into IMC message structures and use a self-describing serialization format for transmitting these messages through HTTP. HTTP allows the reliable transmission of data since it’s based on TCP but on the other hand, HTTP doesn't allow local message broadcasting, thus preventing peer-to-peer service discovery. However, we propose a service discovery approach that uses a hierarchy of repositories borrowing concepts from DNS. A.

REST-style HTTP protocol We find HTTP to be an effective protocol for data. HTTP is ubiquitous from the point of view of platform support (from embedded hardware devices to cloud computing data centers), and from the point of view of programming languages, since almost any language has a HTTP-client library and/or can be used to at create a CGI script that generates content. HTTP is reliable because is built on top of TCP. HTTP can be easily extendable by the inclusion of additional header fields (for security, application-specific parameters, etc) and, finally, HTTP doesn’t require any serialization format, being this format chosen by the application and stated in the content-type HTTP header [6]. a) Resource identification All resources are mapped into one or more URLs. The hostname of the URL corresponds to the node that contains the resource, while the path specifies the resource identifier, a desired serialization scheme (optional) and any query specific parameters. Next, we provide a list of possible URLs and their mapping to resources.  http://192.168.106.34/dune/imc/Position.xml – corresponds to the last Position message that was generated in the vehicle at address 192.168.106.34 in XML format.





http://lauv-blue/dune/imc/Position.lsf – corresponds to the last Position message that was generated by the LAUV-Blue vehicle in its binary (LSTS Serialization Format) representation. http://lauv/dune/imc/Position.lsf?q=altitude>0 – corresponds to a virtual resource that holds all Position messages stored in the vehicle having a value bigger than 0 in the altitude field of the resource (> is the URL encoding of >).

Figure 4. IMC-XML serialization example

b) Changing static resources Some resources, like the vehicle’s position are read-only but others can be written by external entities by using the HTTP PUT method. As an example, the current sampling frequency in a wireless sensor network is mapped onto a resource field that can be either consulted using the HTTP GET method or overridden using an HTTP PUT request (at the WSN sink node). If the client is not authorized to change the resource, it will receive a HTTP Forbidden (3xx) error response and if it is authorized, a HTTP Success (2xx) is given which includes also the new local timestamp of the resource under the Date HTTP header. Afterwards, clients can monitor resource changes by using HTTP HEAD requests whose answers contain the field Date but no content is actually retrieved. c) Listing network resources Each node in the network provides a read-only resource called Announce. This resource can be queried directly to list resources in the node or may be disseminated in the network. HTTP protocol has many advantages over protocols like UDP and RTPS but, on the other hand, doesn’t support broadcasting. To overcome this problem, we use central repositories that use the Atom standard to list identifiers of existing resources in the network, their URLs and timestamp of the latest changes (all part of the Atom standard). Clock synchronization is not required since all timestamps are considered to be local time – the time at the node that contains the resource. Whenever a new node joins the network, he can connect to this central repository and download information about existing services and respective entry points by using a query that lists Announce resources sent to the central repository. As an alternative, a pub sub service like Google’s

Pubsubhubbub [7] can be used to receive updates notifications in real-time. d) Managing connectivity interruption The central repository that stores service information is identified with a unique URL. For networks that don’t have continuous connectivity to the repository and for efficiency purposes, caching proxies can be used transparently in a way that is reminiscent of the Domain Name System (DNS) scheme. For instance, a submarine may have its own caching proxy while travelling underwater that connects to the actual repository (or another proxy) when the submarine surfaces and connects to the network. Caching proxies may be, nonetheless, overridden by issuing the HTTP header “Pragma: no-cache”. e) Resource data serialization Some resources are mapped to static files which can be stored and read from any file system. If historical data is needed for files, any existing versioning scheme can be used locally like CVS or Subversion in order to save disk space. Other resources that represent structured (hierarchical) data are represented according to the IMC protocol. All IMC serialization formats are supported: text, binary LSF (LSTS Serialization Format) and IMC-XML (a self-describing XML format). IMC-XML is particularly interesting because it doesn’t require prior agreement over the message structures and can be transformed into other formats by linking the serialization with a XSLT style sheet; however it imposes a performance penalty (see Figure 4). IMC-XML is simple enough to be generated and parsed at embedded devices like the drifters USTL is currently using. This format can be converted as needed into LSF; for instance, DUNE software supports IMC-XML by converting stored resources to this format on demand (when an URL corresponding to an IMC-XML format is requested) and converts any received IMC-XML messages into LSF structures that are then broadcast inside the vehicle. Structured data following the IMC protocol can be stored as files or in an embedded database for faster data retrieval. We are currently using the SQLite database engine for storing data since it is very efficient and supported by many languages and platforms. We use a unique database engine and a common SQL schema in order to use these databases also as log files. By using a relational database, becomes very easy to support querying of data. f) Web Server Embedding In order to provide access to its resources, some nodes in the network must include an embedded web server. DUNE onboard vehicle software includes a web server created from scratch that can be extended by adding new entry points that map into additional URLs. The operator consoles are based on the Neptus command and control software infrastructure which uses the Java programming language. A specific Java component was created that uses the Jetty embedded web server and new entry points (Java Servlets) are added as needed. This component can run conjointly with the operator consoles or alternatively can run by itself, for use in gateways, WSN base nodes and in a central repository.

g) Resource polling and streamming Periodic data can be accessed using GET requests or, when supported, can also be accessed as HTTP streams of data. GET method is generically used to poll periodic data but there may be applications like sensor sharing that require real-time state updates. In these applications, we use HTTP streaming for data transmission. Dedicated Web Server

IMC-XML

Neptus CCU

Neptus CCU WSN Base Station

IMC + UDP All states are broadcasted in the network All states are broadcasted in the network. Better performance Unreliable

Sporadic state consulting Real-time data monitoring

IMC-XML

IMC-XML

can be consulted in a browser and additionally, by using XSLT, this configuration can be changed through a HTML form.

IMC-XML

IMC-XML IMC-XML

IMC

IMC Neptus CCU

Unmanned vehicles

Drifting Sensors

Sending Configurations Supported formats Resource announcements

Reliably, only small messages (IMC structures) UDP broadcast and Multicast

REST-style HTTP GET method HTTP data streaming. Intermediate states dropped HTTP PUT method Binary files, IMC structures, IMC-XML Atom standard, directory-based

Figure 6. Comparison table between communications architectures

Figure 5. Communications infrastructure example

HTTP streaming is supported in a similar way to queries, where more than one resource states may be retrieved. In a message stream, the connection isn’t closed by the server and whenever a new state is generated or received, it is sent to all opened streams associated with that resource. In the case of periodic data, any intermediate states can be lost so our approach is to serialize only one message at a time and send it to the HTTP stream. In the case of slow connections, fewer states are transferred this way, optimizing the connection speed. V.

EVALUATION

In this section we evaluate the proposed architecture and implementation by presenting results form field deployments and comparing this implementation with UDP-based communication (typically used for robot networks). Before using a REST-style approach, USTL was using UDP-based IMC message communication. This protocol is useful for real-time applications (like tele-operation or sensor sharing) but, on the other hand, is not supported by some embedded sensors and additionally message delivery is unreliable, despite the nature of the resource being communicated. IMC-XML message sending was easily implemented at the drifting sensors allowing them to send state updates as a resource that is stored at a central repository. The central repository can then be queried for last states of all drifters in the network or also may be queried for historical data. A Web Mashup application was built in PHP that processes drifters’ data, showing them as KML in Google Earth Plugin as seen in Figure 7. The presented architecture is also valuable when configuring nodes through the network. Previously, the only way to configure robotic vehicles was by starting a remote session and editing text configuration files but now all configurations are mapped into an IMC structure and these

Figure 7. Web mashup for monitoring drifting sensors

VI.

CONCLUSIONS AND FUTURE WORK

The presented architecture is still starting to be tested in real-world deployments; however it has already proven to be useful. We found that mixing both UDP messaging for resource change announcements and web technologies for third-party mission following and systems integration is a very practical approach. Web enabling our system and using web standards like Atom and XML eases the creation of new applications by reusing existing web services (web mashups). IMC-XML serialization format is mostly used by third-party partners and applications because there is no need for prior knowing of message structures; and XML can be easily transformed into a tree structure that maps the same structured nature of IMC messages. The query approach for data retrieval is also

very powerful in that it can be used for historical data retrieval, service discovery and subscription-based streams. In the future, we will continue ongoing work on automatic service composition by creating a declarative service composition language inspired by [9]. Using this language, new orchestrated services can be described by specifying their pre-requirements (network resource states) and changes that need to be made to the network resources in order to provide the service. One issue that hasn’t been addressed is security. We could think of security simply as HTTP authorization but we find it important that the access level of any individual resource changes according to the current operating mode. We are looking forward to test new networked vehicles’ applications with this architecture and build new query processors that are able to aggregate data from multiple nodes. We are also interested in the integration of existing data sources and external sensing data into our system. ACKNOWLEDGMENTS The authors are very grateful to all members of the USTL team, without their hard-work none of these ideas would ever come out of paper.

REFERENCES [1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

Abderrahim Benslimane, “Optimized dissemination of alarm messages in vehicular ad-hoc networks (VANET),” Lecture Notes in Computer Science, vol. 3079, pp. 655–666, 2004. I. F. Akyildiz, W. Su, Y. Sankarasubramaniam and E. Cayirci, “Wireless sensor networks: a survey,” Computer Networks, vol. 38, pp. 393–422, 2002. T. Arici and Y. Altunbasak, “Adaptive sensing for environment monitoring using wireless sensor networks,” IEEE Wireless Communications and Networking, vol. 4, pp 2347–2352, 2004. Ricardo Martins, Paulo Dias, Eduardo Marques, José Pinto, João B. Sousa and Fernando L. Pereira, “IMC: a communication protocol for networks vehicles and sensors,” IEEE Oceans Europe, 2009. P. Levis, S. Madden, J. Polastre, R. Szewczyk, K. Whitehouse, A. Woo, D. Gay, J. Hill, M. Welsh, E. Brewer and D. Culler, “TinyOS: An Operating System for Sensor Networks,” Ambient Intelligence, part II, pp 115–148, 2005. Roy T. Fielding, “Architectural Styles and the Design of Networkbased Software Architectures,” Doctoral Thesis, University of California at Irvine , 2000. Brad Fitzpatrick, Brett Slatkin and Martin Atkins, “PubSubHubbub Core 2.0: Working Draft,” Web Document, http://pubsubhubbub.googlecode.com/svn/trunk/pubsubhubbub-core0.2.html, 2009. Paulo Dias, João B. Sousa and Fernando L. Pereira, “Networked Operations with Neptus,” MAST – Maritime Systems and Technologies Conference, 2008. Robert Dickerson, Jiakang Lu, Jian Lu and Kamin Whitehouse, “Stream Feeds - An Abstraction for the World Wide Sensor Web,” The Internet of Things, 2008