Proceedings of the IASTED International Conference Robotics (Robo 2011) November 7 - 9, 2011 Pittsburgh, USA
CLOUD ROBOTICS: TOWARDS CONTEXT AWARE ROBOTIC NETWORKS João M. Quintas Pedro Nunes Institute Automations and Systems Laboratory Coimbra/Portugal
[email protected]
Paulo J. Menezes University of Coimbra Institute of Systems and Robotics Coimbra/Portugal
[email protected]
of technologies, Application Programming Interfaces (APIs), auxiliary infrastructure extensions, and various other products. Service-oriented computing concept becomes a distinct architectural model that has been considered by the community as one that can fully leverage the open interoperability potential of web services. For instance, if applications reusable logic is exposed as web services, the reuse potential is significantly increased. Since service logic is now possible to be accessed through open communications framework, it becomes available to a wider range of service consumers. Given that web services provide a communications framework based on physically decoupled contracts it allows services communication to be fully standardized independently from its implementation. This allows a potentially high level of service abstraction while rising the possibility to decouple the service from proprietary implementation details. Buyya et.al, in [2] considered that “Cloud computing refers to the provision of computational resources on demand via a computer network”. An example of how this concept works is found in everyday services provided by companies like Google when they provide a word processing service to clients without actually possessing the software to do it locally in their machines. Since the cloud is the underlying delivery mechanism, cloud based applications and services may support any type of software application or service in use today. Extending that concept, cloud computing allows a functional separation between the resources used and the user's computer, usually residing outside the local network. Consumers now routinely use data intensive applications driven by cloud technology which were previously unavailable due to cost and deployment complexity. In many companies’ employees and company departments are bringing a flood of consumer technology into the workplace which raises legal compliance and security concerns for the corporation. These issues could be tackled using cloud computing, where the company would contract one service that could be used by each one of its employees. Cloud computing is now so popular that distinct research groups are exploring this idea applied to robotics. Robots are being prepared to connect to a cloud computing infrastructure and access vast amounts of processing
ABSTRACT This paper proposes an approach for an automated system composed by mobile robots and a smart-room following service oriented architecture, aiming to undertake complex and heavily computational tasks to aid the user in the execution of determined tasks. The proposed approach is inspired by the principles of Service Oriented Architecture, relying in cloud computing to provide an increased degree of scalability to the system. The robotic system will complement the group of virtual networks that the user may already be a part of, contributing as a connection bridge between virtual and real “worlds”. The objective of this work targets the implementation of a service robotic system that allows distant groups of robots to share and exchange learned skills and improve cooperation with human agents. The connection to the cloud plays the role of knowledge repository for the robotic system. This will allow for distant groups of robots to share and exchange each other’s learned skills and adapt to new situations of cooperation with human agents. A use case scenario is presented and suggests the application of the system in Assisted Living situations. In this scenario the context aware ability orchestrate the system towards providing health care services. KEY WORDS Service Oriented Architecture, Mobile Robots, Cloud Computing, Technology and architecture evolution for Robotics and Humanoids.
1.
Introduction
The technical foundations of the Service-Oriented Architecture (SOA) vision and web services are wellknown and widely recognized and accepted as a suitable architectural style for developing modern applications [1]. Service-Oriented Architecture (SOA) proposes an architectural model which objectives are: improve efficiency, improve agility, and improve productivity of an enterprise considering services as the principal means over which solution logic is represented for supporting the realization of strategic aims related with service-oriented computing. A SOA implementation can involve a mixture
DOI: 10.2316/P.2011.752-062
Jorge M. Dias (*) University of Coimbra Institute of Systems and Robotics Coimbra/Portugal
[email protected]
420
power and data. This approach, which some are calling “cloud robotics,” would allow robots to off-load more computational intensive tasks and even originate a more flexible and cooperative machine learning mechanism. Currently conventional robots are limited to the built-in hardware and software, and usually dedicated to a specialized task operating a well-known and structured environment. To these robots, every task from moving a foot to recognizing a face requires a significant amount of processing and pre-programmed information. According to James Kuffner, from Carnegie Mellon University, cloud-oriented robots could switch CPU-heavy tasks to remote servers, relying on smaller and less powerhungry on-board computers. The paper’s motivation is to present the benefits of applying SOA principles in the design of an infrastructure to support a robot undertaking more complex tasks. The authors are strongly motivated to apply these principles to the implementation of artificial cognitive systems aiming to address some of the challenges in the domains of developmental robotics and behavior analysis for the next 5 to 10 years [3]. Thereby, the paper presents the conceptual design for a service robot system supported by cloud computed services focused on the exchange and learning of relevant information that might be applied in human-robot cooperative tasks. Robotic systems are advantageous when physical operations are required, but due to some technological and major economic reasons, there is still a gap until we can find service and assistive robots working together with the common consumer people. Nevertheless, more specific systems can be implemented and tested in some specific domains. In this section we presented some initial remarks regarding the principles of SOA and Cloud Computing as well as Cloud Robotics and in which terms the usefulness of the robotic systems can be found. The following sections present preliminary considerations about the work we aim to develop under our research . Section 2 will present some of the related works found in the recent literature. In section 3 we propose a use case scenario that motivates the following sections. In section 4 is presented a conceptual perspective for the application of Context Awareness and Cloud Robotics. Section 5 will address the system’s architecture. In section 6 is presented the implementation of the system. Finally in sections 7 and 8 we discuss some considerations about future work and the conclusions.
2.
complexity increases. This is particularly expected to occur in the field of smart social robots. In the other hand, although these concepts are not new, with some works dated back in the 1990s, we are now in better conditions to give these approaches a renewed try. Several researchers describe their efforts to create an infrastructure to enable web services for robotics [4]. The authors discussed a ubiquitous control platform for an autonomous robot that can access distributed application logic based on recent network technologies like XML, SOAP, WSDL and UDDI. To solve the ad hoc problem of how the distributed application logic can be invoked by the robot “Web Services” are presented as the best solution. Web services can speed development with a more flexible infrastructure where multiple services can work together to provide data and services for the application. Semantic web and web services can be applied on robotics in order to facilitate cooperation between robots for joint tasks execution [5]. By implementing Semantic Web Services on top of isolated robots within a network viewpoint they can be regarded as distributed web services that communicate between them semantically, allowing a real time operation. Yinong Chen, et.al, presented research done on service oriented robotics computing and their design, implementation and evaluation of Robot as a Service (RaaS) unit [6]. Their work was sponsored by Microsoft, showing the interest that big companies are putting in these topics. Similarly Intel also follows and supports innovation groups that propose potentially disruptive ideas. Follow a presentation of some well-known Cloud Robotics projects. RoboEarth is a European project led by the Eindhoven University of Technology, in the Netherlands, to develop a “World Wide Web for robots”, a giant database where robots can share information about objects, environments, and tasks [12]. Considering the large amount of information shared on such a vast scale, and with businesses and academics contributing independently on a common language platform, RoboEarth has the potential to provide a powerful source to leverage any robot’s 3D sensing, acting and learning capabilities. ASORO (A-Star Social Robotics Laboratory) from a Singapore researchers group tries built a cloudcomputing infrastructure that allows robots to generate 3-D maps of their environments much faster than they could with their onboard computers [13]. Google engineers developed Android-powered robot software that allows a smartphone to control robots based on platforms like Lego Mindstorms, iRobot create, and Vex Pro. Researchers at the Laboratory of Analysis and Architecture of Systems, in Toulouse, France, are creating “user manual” repositories of everyday objects to help robots with manipulation tasks. At a children’s hospital in Italy, NAO humanoid robots, created by the French firm Aldebaran Robotics, will rely on a cloud infrastructure to perform speech recognition, face detection, and other
Related Work
Cloud Robotics and similar subjects (e.g. Internet Robots, Robots as Web Services, etc.) are assisting to an increase of interest by the scientific community. In one hand the basic concepts related with these topics are very attractive for the future developments in robotics, which will demand more computational resources as the tasks
421
actuated just in time – he almost hit a pedestrian crossing the street in the front of his own house. This does not happen frequently in the outskirts of the city. The garage door, as well as the front door opened as soon as the security system in his house detected the RFID signal transmitted by his personal wrist device. He appreciated the welcoming whiff of balmy air activated by the remote climate control anticipating his arrival, and the welcome of his companion robot, which helps him inside the house.
tasks that might help improve their interaction with patients [14]. Another example is DAvinCi, a software framework attempting to offload data and computationally intensive workloads from the onboard resources of robots to a backend cluster system shared by multiple clients. This framework performs computationally intensive tasks and enables the exchanging of useful data obtained from local preprocessing [7]. More recently, the renowned magazine “Robotics and Automation Magazine” [8] published by IEEE Robotics and Automation Society, dedicated an issue to the topic of robots and the World Wide Web. In this issue the reader is presented with several articles making the overview of current works where effort is being applied to put robots sharing the information that they create with humans or other robots through search engines such as Google images, video, or three-dimensional warehouse.
3.
Use Case Scenarios
In the previous sections we trace an overview of recent works that not only mark the tendency for communication between robots and the way they might share knowledge between themselves, but also there are strong indicators that we could be facing a turning point regarding architectural aspects of software developmental environments for robots, that will allow more open and agile methods for developing, deploying and maintaining robotic systems. These innovative approaches will lead to improvements in several areas of application of robotic systems that are now receiving great interest by the community. Health care and wellbeing embrace a diversified set of scenarios where robotics solutions could play an important role. Ageing related concerns are being proposed with service robotic solutions that aim to improve elder’s quality of life by promoting independent living and socialization. Consider the scenario, illustrated in Figure 1, where we present a day in Tom’s life. Tom is a 75 year old man widow and father of two that lives alone in the outskirts of the city. One afternoon he is returning home from his Thursday driving tour. Tom turned off the autopilot in his smart electric vehicle, which he called it “Auto” – enjoying the feeling of control. He likes to take over from the autopilot to keep up his driving skills and anyway, his coach encourages him to practice as much as possible without automation. Tom interacts with the autopilot via voice commands, as it understands natural language. As Tom this time was taking control, Auto just monitors his driving and advices about the directions to take, as a regular GPS navigator. Meanwhile, Tom passes by one of his neighbors and he loses his attention in the road – the Auto’s automobile safety system detected an attention lapse by his eye movement pattern. The automatic system intervention
Figure 1 Scenario illustration. Auto just parked itself in the garage and initiated its selfcharging process. Suddenly, the ambient living system inside the house told Auto to ignite the engine and prepare to go, as Tom was not feeling well. Considering timefactor, the house decided that calling in an ambulance could be too late. The companion robot helped Tom going to the car, gently laying him inside. Auto knew already what to do exactly, taking Tom to the nearest hospital with signaled emergence march. Staff in the hospital was already notified about Tom’s arrival, waiting at the door with a reanimation kit if that was necessary. Fortunately, Tom fainted due to low blood-pressure, and there was no immediate danger to his health. Accompanied with his daughter Maria, who was also notified by the house, Tom agreed to sleep in Maria’s home. “This time, I want you to take me to Maria’s house, Auto”. “Already going sir. I’m glad you feel better now”. Replied Auto. “Yes, thank you Auto”. Scenarios like this, address modern society’s important emerging concerns regarding population ageing tendency and the increasing costs in health care services. These matters demand solutions that promote increases in services quality while decreasing associated resource consumption costs. ICT based solutions are growing in number and in the mid-term service robots are expected to join the group of valid choices. In this sense, we will focus
422
different systems corresponding to a robot and the smartroom, which manage their own contextual information and share that information between them using a bridge like adapter called Context Discovery Adapter (CDA). This approach was firstly proposed by P. Pawar, et.al in [15], where a Context Discovery Adapter was presented as a solution to bridge two different CMSs in two distinct and heterogeneous networks (one dedicated to ad-hoc connections and other focused on mobile networks). In the scope of the cooperation between the network of robots and the smart-room, an approach like CDA could prove advantageous, although it would require extending the current implementation to a more diversified group of devices.
our research in the design and construction of more interactive service robots taking advantage of cloud computing infrastructures.
4.
Conceptual design
However the important breakthroughs and progresses assisted in service robotics, there are challenges still need to be tackled in order to obtain sophisticated solutions like the one described above. The proposed scenario would require that all the involved robotic systems could be capable of networking and sharing contextual information about the user. The importance of context awareness in service robots is motivated by the dynamical characteristics of the information related with a person’s activities. In order to obtain a ubiquitous robotized environment the capability of being sensitive to changes in context is determinant to correctly adapt system’s reaction to the moment intentions of the user. Context awareness has being studied in the scope of mobility systems, usually related with personal devices like PDA’s and Smart phones, and how they adapt to changes in network connections. The application of context awareness in robotics will require the integration of much more information sources than the ones available in the first examples. However, the particular case of sharing context information between distinct and heterogeneous networks without having to develop an entire system from scratch is still a challenge.
5.
Architecture aspects
In order to satisfy the needs of similar scenarios, the system could be though as the composition of a set of services provided by each robot, thus making it following service oriented architecture. We propose a service oriented architecture that will address the collaboration between a group of robots and a smart-room taking into consideration context sharing and management. The contextual information available in each robot will be combined to produce a representation of the user’s situation. In Figure 3 is provided a representation of the system’s architecture design. The objective of distribute the robots workload according to different infrastructural nodes, the routing mechanism should consist in a set of brokers, according to the type of data to be processed. The brokers encapsulate a group of messaging queues for distributing the load among different nodes. The architecture proposed in this paper aims to integrate the mobile robot and associated functional capabilities as services. The mobile robot will act as service provider and consumer. Services will be published into a common service repository, thus making them discoverable by other remote services. A service could correspond to a skill learned by the robot, which would be published in the cloud and be usable/learned by other robotic agents (even in a remote location). Some illustrative examples of services that could be provided by the robotic system are listed in the next table. Table 1. Examples of services provided by the robot Robot Services Mobility helper
Figure 2. Context Awareness System general overview
Butler Gaming companion (using physical objects)
In Figure 2, a schematic is presented illustrating the concept behind the context information sharing and management between different Context Management Systems (CMS). In the figure the CMSs represent the
423
Figure 3 System's architecture design In order to keep up with user’s service requests and for the sake of the system’s “evolution”, the robot will rely on the cloud to obtain new services. In case it cannot fulfill its user request the robot will connect to the cloud skill repository and download the requested skill. This mechanism will allow for a dynamic behavior, and will allow for a lower degree of computational requirement in the robots hardware, since the services can be loaded and unloaded according to needs of the user. Figure 4 illustrates the mechanism of reacting to a service request that is unknown to the robot. Besides the possibility of fetching new skills from the cloud, the robot will have mechanisms of learning. The learning mechanism will allow the user to teach him new skills that may be very specific or may not be present in the cloud skills repository. In Figure 5 we illustrate the learning and publish procedure. The cloud skills repository consists in a public area where different robotic systems would load their different learned skills in order to share them with sibling robotic platforms. Human Requesting Service from Robot
Receives Service from Robot
Teach new skill by interaction
Provide Robot’s datasheet
Robot Learn / Train selected skill
Cloud Share / Publish learned skill
Figure 5 Teaching and sharing robot skills
6.
Implementation and work in progress
In this section the authors present the recent implemented work. The scenario presented in Section 3 consisted in the collaboration of robots and the smart environment. Thus it is justified to address both parts in parallel with small increments by iteration. It is also acceptable that more simple scenarios are addressed in order to guide these iterations. Until now, the authors focused in the implementation of the fixed infrastructure. Therefore, in this section is presented some results obtained at the moment for this part of the system. The previous sections presented the authors vision about the service robot conceptual design that will work in collaboration with the fixed infrastructure, which will be developed in future work.
Cloud
Robot Can I provide the service?
Human
Fetch required service
Figure 4 Obtain new services from the cloud
424
These facts prevent that a more detailed assessment is presented in this work regarding the robotic system. Nevertheless, a detailed insight about the results obtained for the infrastructure used to obtain 3D volumetric reconstruction of the scene can be found in [9]. The following sub-section presents a scenario concentrating in surveillance tasks, which provided some requirements for the implementation of the fixed infrastructure.
6.2 Work in progress In this section it is presented the current stage of the implementation. The most recent accomplishments include the fixed vision-based system and the 3D body reconstruction algorithm. In Figure 7 is represented an example of the results obtained so far.
6.1 Implementation Scenario In this scenario the mobile robots are called to approach a detected person and give remote access to images from the area under surveillance, the sensor network infrastructure would be in charge for executing the 3D reconstruction from the person in the scene. These two systems communicate together through web-services. The work flow of the service provided by the security system of “Scout the area” emerge from the interaction between the different services provided by each one of the individual systems. In Figure 6 we represent the idea for the scenario that in under work.
Figure 7 Results of the 3D volumetric reconstruction of the person in the scene. The depicted results in Figure 7 represent the 3D volumetric reconstruction. These results were obtained while the space was observed by the IMU+cameras network. The left pictures show the images of the person before and after background subtraction. The right image depict the results of reconstruction based in 35 inertialbased virtual planes, with an interval distance of 50 mm. 6.3 Infrastructure overview The previous results were obtained using an experimental setup that consists in an infrastructure divided in three main parts. The infrastructure is divided into a fixed vision-based system, mobile robotic platforms and the computational framework. 6.3.1 Fixed vision-based system The fixed vision-based system is composed by a network of Ethernet cameras coupled together with Inertial Measuring Units (IMUs) and will provide vision-based services that demand high computational requirements, for example 3D body on-line reconstruction [9]. 6.3.2 Mobile robotic platforms Additionally to the fixed infrastructure, the system will take into account with the mobile robotic platforms. At the moment the authors are using two different mobile platforms. One is based in a modified Nomadic Scout robot. The onboard computer was replaced by a laptop to provide a reduction of energy consumption and better user interface. The scout unit has a group of sonars that are used for navigation purposes, a camera mounted on a pantilt unit. The other platform is based on a Segway RMP platform in which we mounted a robotic head with stereovision, microphones, inertial sensors and laser [11]. The next two figures illustrate these two robotic platforms.
Figure 6 Scenario of a robot interacting with a person and using the infrastructure’s services to compute the 3D Human Reconstruction, Facial Identification and Human Detection.
425
1 GB RAM, 80GB HDD and a PCI-Express NVIDIA GeForce 9800 GTX+). The computational framework will also take into account with the use of mobile devices such as smart-phones, PDAs, and biometric sensors that may be wearied by the user in special scenarios. In the next section we present the stage of the implementation of this work, which correspond to fully implementation of the fixed part of the infrastructure, corresponding to the vision-based system, and the implementation of a 3D body reconstruction algorithm service.
7. Figure 8 Robotic platform based on Nomadic Scout
Future Work
The mid to long-term goals of the work aim to address some of the challenges proposed to Developmental Robotics for the next decade as referred in literature [3], where a set of challenges are presented toward Action and Language integration in robotics. From the technological point of view, the requirements imposed by these challenges suggest the need for very demanding and heavy computational needs. Service-Oriented Architecture allow for an elegant distributed system, where flexibility and scalability should be assured. The future of this work will aim to contribute for the field of developmental robotics with important breakthroughs with respect to action and language integration in social robots, as well as in the field of behavior analysis. The presented concepts will drive the work towards the implementation of a mobile robot with advanced socialization skills. Furthermore, one of the principal guidelines for the work will be the implementation of biological inspired developmental learning processes applied in artificial systems. The domains of application for this approach are vast and rich. The authors envisage increased value in applications for Ambient Assisted Living (AAL), Security, Social Robotics and rehabilitation support.
Figure 9 Robotic platform based on Segway RPM and robotic multimodal perception head. 6.3.3 Computational framework The computational framework responsible for controlling the fixed vision-system is comprised by seven computers connected to the network infrastructure. This equipment is used to support the software infrastructure. The technical details of the computers are:
8.
• Fujitsu Siemens ESPRIMO P5925 EPA • Processor Q9300 • Processor Cores Quad • Memory 2GB • Disk 500GB SATA 7200rpm • Drive bay DVDRW • S.O. Ubuntu 9.04
Conclusion
The paper presented the design of a context aware cooperative framework involving mobile robots and a distributed sensor network infrastructure. The objective of this research is to provide a contribution to the service robotics field. The concept presented in this paper focus the capacity of a service robot to use cloud computed services to share and acquire knowledge in order to execute more complex and adaptable behavior, which might be advantageous in collaborative tasks with human agents. From the architectural perspective, the paper suggested Service Oriented Architecture to be adequate to implement the system. The conceptual and architectural aspects where presented and the implementation status was explained. The proposed approach is directed towards
This “cluster” is responsible for the synchronized acquisition and streaming of visual data to the algorithm processing unit. The processing unit responsible for all the sensory and vision algorithm (including CUDA processing) is composed by a PC (Intel Dual Core Pentium D 950 3.40GHz processor, Cache L1 (32KB) and L2 (2048KB),
426
[8] IEEE Robotics and Automation Magazine, Volume 18 Issue 2, June 2011. [9] Aliakbarpour, H. and Dias, J., “Multi-resolution Virtual Plane based 3D Reconstruction using InertialVisual Data Fusion.”, In International Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, pp. 5-7, Algarve, Portugal, 2011. [10] Lobo, J., Almeida, L., Alves, J., and Dias, J., “Registration and segmentation for 3d map building- a solution based on stereo vision and inertial sensors.”, In Robotics and Automation, 2003. Proceedings, vol 1, pp. 139 – 144, 2003 [11] Ferreira, J. F., Lobo, J., Dias, J., “Bayesian RealTime Perception Algorithms on GPU - Real-Time Implementation of Bayesian Models for Multimodal Perception Using CUDA.”, Journal of Real-Time Image Processing, Special Issue, Springer Berlin/Heidelberg, published online (ISSN: 1861-8219), February 26, 2010. [12] RoboEarth (http://www.roboearth.org/) [13] ASORO (http://www.asoro.astar.edu.sg/research_main.html) [14] NAO robot (http://www.aldebaran-robotics.com/) [15] Pravin Pawar, Hanga Boros, Fei Liu, Geert Heijenk, and Bert-Jan van Beijnum, “Bridging Context Management Systems in the Ad Hoc and Mobile Environments”, IEEE, 2009.
the cloud robotics concept, where the infrastructure is computationally powerful and allows the execution of more processing demanding tasks by distributing the workload among different processing nodes (e.g. Robots and Infrastructure). Finally the authors present their future intentions regarding how they intend to direct their work.
Acknowledgement(s) The authors gratefully acknowledge support from Institute of Systems and Robotics at University of Coimbra (ISRUC), from Portuguese Foundation for Science and Technology (FCT) and Pedro Nunes Institute. Jorge Manuel Miranda Dias (*) is on sabbatical leave at Khalifa University of Science, Technology and Research (KUSTAR), Abu Dhabi, UAE
References [1]
Erl, T., Service-Oriented Architecture: Technology, and Design, Prentice Hall, 2005.
Concepts,
[2] Buyya, R., Yeo C. S., Venugopal S., “Market-Oriented Cloud Computing: Vision, Hype, and Reality for Delivering IT Services as Computing Utilities.”, Department of Computer Science and Software Engineering, pp. 9, University of Melbourne, Australia, 2009. [3] Cangelosi, A., Metta, G., Sagerer, G., Nolfi, S., Nehaniv, C. Fischer, K. Tani, J. Belpaeme, T. Sandini, G. Nori, F. Fadiga, L. Wrede, B. Rohlfing, K. Tuci, E. Dautenhahn, K. Saunders, J. Zeschel, A., “Integration of action and language knowledge: A roadmap for developmental robotics.”, Autonomous Mental Development, IEEE Transactions on, vol. 2, pp. 167 – 195, 2010. [4] Bong Keun Kim, Miyazaki, M., Ohba, K., Hirai, S., Tanie, K., “Web Services Based Robot Control Platform for Ubiquitous Functions”, In Proceedings of IEEE Robotics and Automation, pp. 691- 696, 2005. [5] Vasiliu, L., Sakpota, B., Hong-Gee Kim, “A Semantic Web Services Driven Application on Humanoid Robots.”, In the Fourth IEEE Workshop on Software Technologies for Future Embedded and Ubiquitous Systems and the Second International Workshop on Collaborative Computing Integration and Assurance, pp. 236-244, 2006. [6] Chen, Y., Du, Z., Garcia-Acosta, M., “Robot as a Service in Cloud Computing.”, In Fifth IEEE International Symposium on Service Oriented System Engineering, pp. 151-158, Nanjing, 2010. [7] Arumugam, R., Enti, V.R., Baskaran, K., Kumar, A.S., “DAvinCi: A cloud computing framework for service robots.”, In Robotics and Automation (ICRA), pp. 30843089, Anchorage, 2010.
427
