Information Distribution in Dynamic and Unreliable Environments

Information Distribution in Dynamic and Unreliable Environments

Carl Barck-Holst

TRITA-NA-E03067

NADA Numerisk analys och datalogi KTH 100 44 Stockholm

Department of Numerical Analysis and Computer Science Royal Institute of Technology SE-100 44 Stockholm, Sweden

Information Distribution in Dynamic and Unreliable Environments

Carl Barck-Holst

TRITA-NA-E03067

Master’s Thesis in Computer Science (20 credits) at the School of Computer Science and Engineering, Royal Institute of Technology year 2003 Supervisor at Nada was Henrik Christensen Examiner was Henrik Christensen

Abstract Sharing data within a group of cooperating robots is an essential problem in mobile robotics. This thesis investigates, designs and implements a communication system for this particular purpose. The problem is non-trivial due to the changing configuration of the robots as well as limited bandwidth in the network. A design choice is to use a UDP broadcast scheme in concert with a mechanism to retrieve lost data. This approach gives a system with a good average performance, although there is no guarantee on an upper bound of propagation time. Another characteristic of the implemented system is that the propagation times grow linearly with the number of steps the sensor reading is required to travel before reaching the destination platform. The applicability of the communications system is demonstrated in a number of experiments. Several different configurations of robots perform a cooperative mapping effort in an indoor environment.

Informationsspridning i dynamiska och otillförlitliga miljöer

Sammanfattning Att ha en gemensam lägesuppfattning inom en grupp mobila robotar är en viktig egenskap för att de ska kunna samarbeta med varandra. Detta arbete analyserar och designar ett kommunikationssystem för detta ändamål. Problemet är icketrivialt då robotarnas mobilitet och nätverkets begränsningar med avseende på räckvidd och bandbredd ständigt förändrar gruppens kommunikationslänkar. Ett broadcastsystem i kombination med en sökmekanism för förlorad information presenteras. Angreppssättet ger ett system med god medelprestanda. Tiden som krävs för att överföra information är linjärt proportionell mot antalet robotar som informationen är tvungen att överföras över för att nå destinationen. Det är inte möjligt att garantera en övre gräns för den tid som krävs för att flytta information inom gruppen. Kommunikationssystemets duglighet analyseras experimentellt med flera kartläggningsuppdrag.

Contents 1 Introduction 1.1 The Problem Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Challenges With Creating a Distributed Database . . . . . . . . . . . 1.3 Formal Problem Description . . . . . . . . . . . . . . . . . . . . . . . 1.4 Required Properties of a Distributed Database System . . . . . . . . 1.5 Potential Uses For a Distributed Database System . . . . . . . . . . 1.5.1 Application Scenario: Mapping of Unknown Terrain . . . . . 1.5.2 Application Scenario: Robots Playing Soccer . . . . . . . . . 1.5.3 Application Scenario: Command and Control for Fire Brigades 1.6 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6.1 Distributed Autonomous Mobile Robotics . . . . . . . . . . . 1.6.2 Wireless Sensor Networks . . . . . . . . . . . . . . . . . . . . 1.6.3 Distributed Database Systems . . . . . . . . . . . . . . . . . . 1.6.4 Summary: Related Work . . . . . . . . . . . . . . . . . . . . . 1.6.5 Report Outline . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1 1 2 3 3 3 3 4 6 6 11 12 12 13

2 Conceptual Design 2.1 Information Representation . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 Raw Range Measurements . . . . . . . . . . . . . . . . . . . . 2.1.2 Part of Map and Complete Map . . . . . . . . . . . . . . . . 2.2 Information Location . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Store All Information on One Central Robot Repository . . . 2.2.2 Store All Information on a Few Replicated Robot Repositories 2.2.3 Replicate All Information on All Robots . . . . . . . . . . . . 2.3 Design Choices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Design Choice: Information Representation . . . . . . . . . . 2.3.2 Design Choice: Information Location . . . . . . . . . . . . . .

14 14 14 15 15 15 16 17 18 18 18

3 Detailed Design 3.1 Key Design Areas . . . . . . . . . . 3.2 Communication . . . . . . . . . . . 3.2.1 Broadcast Communication . 3.2.2 Direct Communication Link 3.2.3 Benchmark . . . . . . . . .

20 20 20 21 21 22

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

3.3

3.4

3.5

3.2.4 Communication Load . . . . . . . . . . . 3.2.5 Design Choice: Communication . . . . . . Measurement Propagation . . . . . . . . . . . . . 3.3.1 Push . . . . . . . . . . . . . . . . . . . . . 3.3.2 Pull . . . . . . . . . . . . . . . . . . . . . 3.3.3 Push and Pull in Combination . . . . . . 3.3.4 Dynamic Network Topology . . . . . . . . 3.3.5 Design Choice: Measurement Propagation Measurement Retrieval . . . . . . . . . . . . . . . 3.4.1 Querying One Robot . . . . . . . . . . . . 3.4.2 Querying All Robots . . . . . . . . . . . . 3.4.3 Design Choice: Measurement Retrieval . Summary . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

4 Implementation 4.1 Summary of Design Choices and Required Properties 4.2 Software Functionality . . . . . . . . . . . . . . . . . 4.3 Software Module Design . . . . . . . . . . . . . . . . 4.3.1 Data Source . . . . . . . . . . . . . . . . . . . 4.3.2 Database Client . . . . . . . . . . . . . . . . . 4.3.3 Map Generator . . . . . . . . . . . . . . . . . 4.4 Messages . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Measurement Identification . . . . . . . . . . 4.4.2 Network Messages . . . . . . . . . . . . . . . 4.5 Implementation Software . . . . . . . . . . . . . . . . 4.5.1 Dave’s Robotic Operating System . . . . . . . 5 Results 5.1 Real Time Mapping . . . . 5.1.1 Objective . . . . . . 5.1.2 Experimental Setup 5.1.3 Configuration . . . . 5.1.4 Results . . . . . . . 5.2 Measurement Propagation . 5.2.1 Objective . . . . . . 5.2.2 Experimental Setup 5.2.3 Configuration . . . . 5.2.4 Results . . . . . . . 5.3 Measurement Retrieval . . . 5.3.1 Objective . . . . . . 5.3.2 Experimental Setup 5.3.3 Configuration . . . . 5.3.4 Results . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . .

23 24 24 24 25 25 28 29 29 31 31 31 31

. . . . . . . . . . .

33 33 33 34 35 35 35 36 36 36 38 38

. . . . . . . . . . . . . . .

39 39 39 39 39 39 41 41 41 41 41 43 43 43 43 43

6 Summary and Conclusion 6.1 Project Overview . . . . . . . . . . . . . . . 6.1.1 Problem Description . . . . . . . . . 6.1.2 Design . . . . . . . . . . . . . . . . . 6.1.3 Results . . . . . . . . . . . . . . . . 6.2 Future Work . . . . . . . . . . . . . . . . . 6.2.1 Routing in Broken ad hoc Networks 6.2.2 Information Fusion . . . . . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

46 46 46 46 47 48 48 48

References

50

A Data From Propagation Experiment

53

B Using PostGres as an Information Repository

57

Chapter 1

Introduction 1.1

The Problem Overview

We wish to analyze, design and implement a distributed database system (DDBS) that facilitates sharing of information gathered by a number of mobile robots. A robot utilizing the DDBS can then use the information shared by others to maximize it performance. The robots will be working in very large environments where the risk for communication failures is high and where the communication channels have limited bandwidth. These problem properties imply interesting routing problems to ensure that all sharing robots receive all the information shared. The usefulness of the DDBS will be investigated by applying the DDBS architecture to a joint mapping task. Multiple robots will be given the task to explore disjointed regions and jointly create a map representation of the regions.

1.2

Challenges With Creating a Distributed Database

There a several challenges involved when creating a distributed database system. Most challenges are related to the communication aspects of the system: • Temporary loss of communication channel: Dropouts will arise when robots are out of radio reach. This can happen for all sorts of reasons, there might be obstacles in the way or the robots may be too far away from each other. Dropouts really mean missed data. How will we know what we missed? How will we get it? • Limited bandwidth: Bandwidth will limit the amount of data that can be transferred at any one time. Careful consideration will have to be given regarding how to shuffle data in the system. Where should data be stored so that it can be accessed and contributed to by all robots? • System architecture: A system with dropouts raises questions regarding how to keep track of which robots are cooperating. Should we use a central server 1

to keep track of robots cooperating, or should we use another model? If a new robot joins the network, how will the other robots find out? How should measurements be transferred so the new robot quickly is updated on the current state of the system? • Communications architecture: If the area that the robots are working in is bigger than their radio reach, then there might be robots that never are in contact with each other. How will the robots find out and access each other’s data? Given the challenges listed above, is it possible to rely on something that is inherently unreliable?

1.3

Formal Problem Description

To understand the task of the DDBS properly we need to define the systems components and their limitations. • We have a team of robots t = {r1 , r2 , . . . , rn }. • The number of robots n will change over time. • Every robot has multiple sensors rn = {s1 , s2 , . . . , sv }. • Every sensor generates information I(ri , sj , t) • The robot pose p (coordinates and heading of robot) will change over time as the robots move in the world. p(rn , x, y, θ) • Every robot r is equipped with a radio communications device with a limited range R. R(rn )