Robot Middleware and its Standardization

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Robot Middleware and its Standardization in OMG October 10, 2006 International convention Center Beijing, China

Tetsuo KOTOKU 䎱䎤䎷䎬䎲䎱䎤䎯䎃䎬䎱䎶䎷䎬䎷䎸䎷䎨䎃䎲䎩

䎤䎧䎹䎤䎱䎦䎨䎧䎃䎬䎱䎧䎸䎶䎷䎵䎬䎤䎯䎃䎶䎦䎬䎨䎱䎦䎨䎃䎤䎱䎧䎃䎷䎨䎦䎫䎱䎲䎯䎲䎪䎼䎃䎋䎤䎬䎶䎷䎌

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Introduction Robot Society in the 21st century increase of the high elderly population

䌐䌒䌏䌂䌌䌅䌍䌓

•how to support the elderly in their daily lives •how to keep enough labour force in industrial and social activities

Expanding Robot Application from industry to non-industry Manufacturing Automation

Maintenance Medical Home Service Security Communication Entertainment etc

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Introduction Robot Society in the 21st century

With the rapid progress in computer and communication technology, the robot systems are fast becoming larger and more complicated. Therefore, there is a real need for the software technologies for efficient developments. Now various software technologies are proposed and implemented respectively. Rapid progress: Computer Technology

Single robot

Robot Systems

Networked robot

• larger

• more complicated

Network Technology

Efficient Efficient Development Development

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Technology Strategy (JARA) How new robotic products will be produced? Needs

Integrating Components

New Robots

Components Market ( RT Components )

Motors

Sensors

Robot arm

Made-to-Order Made-to-Order Business Business

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Technology Strategy (JARA) Solution Business Design Information

Order System Integrator

Customer

Order to make

Component Component Component Companies Companies Companies

Supplying Component

Supply

Technical seeds

Academia manufacturer

from robots to RT 21st Century Business Model

Made-to-Order Made-to-Order Business Business

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

RT Middleware Project Conventional Robot Systems

Component Based Robot Systems Application of Robot A

Application of Robot B

Application of Robot C

Robot C Robot B Robot A 䉸䊐䊃䉡䉣䉝 䊤䉟䊑䊤䊥 䉸䊐䊃 䊤䉟 Servo Control

Network 䊝䊷䉺 䉶

䊝䊷 Force Sensor

䉶 Motor

ƔRobot Maker makes Everything of each robot. ƔInterfaces of modules in each robot are not defined well. So, it is difficult to re-use them in other robot systems. ƔCost of development is high. ƔIt is difficult to create a variety of robots

Servo Control

Force Sensor

Motor

ƔIt will be easy to create new robot by re-using existing modules. ƔCost of development of new robot will be low. ƔModule suppliers, software module suppliers and system integrators can join the new robot business. ƔIt will be easy to develop a variety of robots.

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Important Issues • Preparing for Technological Infrastructure for the System Integration Industry • Robotic components with open architecture controller should be supplied to the market. • Middle-ware, a kind of software which standardizes robotic component connection should be considered. • A specially designed processor for open controller of robotic system should be developed.

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Standardization Activity in OMG Object Management Group • Worldwide software consortium – Distributed Object Middleware (CORBA) – Object Model Language (UML) – Model Driven Architecture (MDA)

• Application Fields Specific Standardization (Business Enterprise Integration, C4I, Finance, Healthcare, Life Science Research, Manufacture, Software-based Communication, Space, Robotics) -> Domain Technology Committee http://www.omg.org/ http://www.omg.org/

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

(Since Dec. 9, 2005) • Yun-Koo Chung (ETRI, Korea) • Tetsuo Kotoku (AIST, Japan) • Hung Pham (RTI, USA)

http:// http://robotics.omg.org/ p robotics.omg.org g g/

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Standardization Activities Unfortunately, most of pioneering initiatives are developed independently of the others, driven by specific applications and objectives. In order to settle this state of chaos, we would like to contribute to the promotion of standardization in the field of robotics based on the mutual understanding between the relevant parties. for application A Design A for application B Design B

for objective D Design D

Interoperability

for application C Design C

Integration of robot systems based on modular components

for objective E Design E for objective F Design F

Robotics Robotics obot cs standards standards sta da ds

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Standardization Process • Agreement based on discussion • Strict process for fairness • Leadership by volunteers Initial Survey

Priority

Volunteer

RFI issue

Roadmap

Scope

Agreement

RFP Issue

WG

Initial and Revised submission

5 meetings /year

Review

Documentation

AB endorsement TC recommendation BoD adoption

䌗䌇䋱 䌗䌇䋲 䌗䌇䋳

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Robotics Domain Task Force Initial Survey:

Initial Survey

Priority

Volunteer

Scope

Agreement

Review

• Recruiting discussion members • Get information for various activities Request For Information (RFI)

• Presentation and discussion • Setting up working groups http://robotics.omg.org/ http://robotics.omg.org/

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Robotic Systems RFI Scope of Robotic Systems : “Systems “Systemsthat thatprovide provideintelligent intelligentservices servicesand and information informationby byinteracting interactingwith withtheir theirenvironment, environment, including includinghuman humanbeings, beings,via viathe theuse useof ofvarious various sensors, sensors,actuators actuatorsand andhuman humaninterfaces.” interfaces.” Large variation of physical characteristics mobile robots humanoid robots pet robots, manipulator robots autonomous vehicles robot house etc.

Broad span of applications communication and entertainment robots lifestyle support robots rescue robots transportation robots medical robots etc.

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Robotic Systems RFI • An RFI may be issued to gather industry requirements and comments at the beginning of standardization phase • Any person or company may respond • OMG member decide how to proceed, based on input from both inside and outside the organization • Access the RFI documents at http://robotics.omg.org/robotic_systems_rfi.htm • Response deadline was April 4, 2006

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

RFI Responses • 1st Batch: 9 presentations in Burlingame (RTI, Systronix, SNU, ETRI * 2, NEC, NTT, ATR, Toshiba)

• 2nd Batch: 14 presentations in Tampa (Hitachi, ADA Software, SEC, Mayekawa MFG, ETRI*3, Tsukuba Univ., AIST, Coroware, IHI, PrismTech, THALES, Toshiba)

• 3rd Batch: 6 presentations in St. Louis (Samsung*2, Fujitsu, ETRI, SAIT, SEC)

Total: 29 presentations

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Organization Robotics-DTF Steering Committee

Yun-Koo Chung (ETRI䋬Korea) Tetsuo Kotoku (AIST, Japan) Hung Pham (RTI, USA䋩

All volunteers

Publicity Sub-Committee Contacts Sub-Committee Technical WGs Infrastructure WG Robotic Services WG Profile WG

Abheek Bose (ADA Software䋬Indea) Masayoshi Yokomachi (NEDO, Japan) Olivier Lemaire (AIST, Japan) Yun-Koo Chung (ETRI䋬Korea) Makoto Mizukawa (Shibaura-IT, Japan) Yun-Koo Chung (ETRI䋬Korea)

Noriaki Ando 䋨AIST, Japan䋩 Rick Warren (RTI, USA䋩 Saehwa Kim 䋨SNU䋬Korea䋩 Soo-Young Chi (ETRI䋬Korea) Olivier Lemaire(AIST, Japan) Bruce Boyes (Systronix䋬USA) Seung-Ik Lee (ETRI䋬Korea)

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Robotic Functional Services WG Co-chairs : – Soo-Young Chi (ETRI) – Olivier Lemaire (JARA/AIST)

[email protected]

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Robotics Services WG Mission Statement • The goal of the Robotics Services WG is : – Establish a clear definition of Robotic service – Identify and categorize services commonly used in robotic application and the technologies involved – Define standard interfaces that expose these technologies to robotic application developers – Coordinate with other groups within the OMG Robotics Task Force to keep specification consistent

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Robotic Device and Data Profile WG Co-chairs : – Bruce Boyes (Systronix) – Seung-Ik Lee (ETRI)

[email protected]

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Profile WG Mission Statement Application Programmer's View 1. Define scope and model of API 2. Define typical devices 3. Device hierarchies (like class hierarchies) 4. Define interfaces & Data structures 1. Consider standards such as JAUS

5. Device Profiles 1. Enumeration of available resources 2. Resource configuration and capabilities

Physical Resource View 1. Apply relevant standards (IEEE, etc) to robotics 1. Smart sensors IEEE-1451 2. Precision networked clock IEEE-1588 3. Arrange presentations on the above at OMG meetings 1. 1451 in Anaheim? 2. 1588 in Wash DC? (near NIST)

2. I/O point tagging, provides: 1. Enumeration of available resources 2. Storage of configuration and capabilities 1. on the actual device or as close to it as possible

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Infrastructure WG Co-chairs: – Saehwa Kim (Seoul National Univ.) – Rick Warren (RTI) – Noriaki Ando (AIST)

[email protected]

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Infrastructure WG Mission Statement • The purpose of the Infrastructure Working Group of the Robotics Domain Task Force is to standardize fundamental models, common facilities, and middleware to support the development and integration of a broad range of robotics applications. • This working group should collaborate with other groups within OMG. – Common facilities • Fundamental services general to wide range of robotics applications.

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Robotic Technology Component (RTC) • Adopted in the Anaheim Meeting (September 29, 2006)

• Component model for robotics – Basis for software modularization and integration at infrastructure/ middleware level in this domain – Builds on – does not replace – general-purpose component models

National Institute of Advanced Science & Technology (AIST)

Real-Time Innovations (RTI)

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Call for Participation OMG Technical Meeting in Washington DC December 4-8, 2006

Robotics-DTF meeting • WG meetings [Mon., Tue., Thu.] • Plenary meeting [Tue., Wed.] • Steering committee [Monday] (any volunteers are welcome!) http:// http://www.omg.org/registration/ p www.omg.org g g/registration/ g

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Contacts: Home Page: http://robotics.omg.org/ Mailing List: [email protected] Discussion: – [email protected] – [email protected] – [email protected] – [email protected]

※⁄⁁⁅․••
‒⁉⁡⁤⁝⁥⁚⁡⁢‒⁡⁠‒⁄⁡⁔⁡⁦⁛⁕‒⁅⁦⁓⁠⁖⁓⁤⁖⁛⁓⁦⁛⁡⁠‒

Conclusions < Key Technology of RT >

Module-base Open Architecture – Inter operability – reusability – portability – development tool

Solution BusinessDesign

Customer

Order

System Integrator

Order to make

Informatio n Component

Companies

Supplying Compone nt

Academia

Supply

manufacturer



Standardization



New New RT RT Industry Industry y

1

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Orca: Components for Robotics Alexei Makarenko, Alex Brooks, Tobias Kaupp

ARC Centre of Excellence for Autonomous Systems The University of Sydney

Uni. of Sydney: where we are coming from

• Distributed applications • Sensor networks, decentralized information fusion • Naturally leads to components

• Industry funded projects • Need for robust implementations

• Complexity of robotic software • Robotic software – the bottleneck of what is possible today (not computing hardware, sensors, algorithms) • “The PhD problem” 3

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Component-Based Software Engineering (CBSE)

• Advantages • Modularization: “collection of small problems is easier to solve than one big problem” • Minimizing source-code cross-dependencies • Maximizing software reuse • Parallel development by large distributed teams

4

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Component Model

• Fundamentals • Interface definition • Communication mechanism

• Options • • • • •

5

.NET (Microsoft) Enterprise Java Beans (Sun) CORBA (OMG and implementers) Ice (ZeroC) Custom (e.g. Player, Orca-1)

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Real Choices for Robotics

• Middleware: Off-the-shelf vs. Custom • Player uses custom and is very successful • We believe there is a limit to what can be done with custom middleware (reliability, features, performance)

• Off-the-shelf Middleware: CORBA vs. Ice • The concept is similar if not identical • For comparison of design, features, performance, etc. see ZeroC. • We’ve used Ace/Tao and found it difficult • We find Ice much more straightforward

6

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Ice – Internet Communication Engine • Background • Company: ZeroC, since 2001 • http://www.zeroc.com • Open-source, proprietary • Support • OS: Linux, Win, MacOS X, Solaris • Language: C++, Java, C#, Visual Basic, Python, PHP • Transport: TCP/IP, UDP/IP, SSL • Licensing • GPL for open-source projects • LGPL for Orca • Commercial license otherwise 7

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Orca Project: Components for Robotics • Main objective: Software Reuse • Enable: by defining a set commonly-used interfaces • Simplify: by providing libraries with a high-level convenient API • Encourage: by maintaining a repository of components

• What makes it different? • adopts a Component-Based Software Engineering approach • uses an industrial-strength library (Ice) for communication and interface definition • aims to be general, flexible and extensible • provides optional tools to assist in the development of individual components and the management of large systems • maintains a repository of free, reusable components

8

IROS’06 Workshop on Software Standardization

Alexei Makarenko

“An object-oriented middleware platform”

• Developer • Defines interfaces in an Interface Definition Language (IDL) • A bunch of remote procedure calls

• Writes server application • Implements RPC’s

• Writes client application • Calls RPC’s

• Ice • Takes care of communication, serialization, etc.

9

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Software Architecture

Component 1

Component 2

libOrcaIce

Ice C++ Linux OS

Ice Java transport

Win OS

e.g. TCP/IP

10

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Ice Services • IceGrid – grid computing • Registry (centralized) • On-demand activation, reactivation • Replication, etc.

• IceBox – application server • Internal comms are optimized to a function call (optional)

• IceStorm – event service (an IceBox service) • Publish/subscribe of any interface • Federation

• • • •

IcePatch2 – file distribution Freeze – persistence service Glacier2 – firewall Ice-E – “embedded Ice” • Light-weight version for PDA’s.

11

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Orca Project

• Hosted on SourceForge • Distribution includes • Common interface definitions (Slice IDL) • Utility libraries • Component repository

12

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Project History and Size

13

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Interface Definitions

• Slice: an Interface Definition Language (IDL) // Slice class Position2dData extends OrcaObject { Frame2d pose; Twist2d motion; };

interface Position2d { nonmutating Position2dData getData() throws DataNotExistException, HardwareFailedException; void subscribe( Position2dConsumer* subscriber ) throws SubscriptionFailedException; idempotent void unsubscribe( Position2dConsumer* subscriber ); };

14

IROS’06 Workshop on Software Standardization

Alexei Makarenko

libOrcaIce

• Aims • To set naming conventions • To simplify component development

• Provides • Classes to derive from • Classes to use • Helper functions

• Strictly optional • Can choose not to use it and call libIce directly

15

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Stand-Alone Component

16

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Component as an IceBox Service

17

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Component Repository • • • • • • • • • • • • • 18

FaithLocaliser FeatureMapLoader LaserFeatureExtractor LocalNav Logger LogPlayer OgMapLoader OrcaView RegistryView SegwayRmp SickLaser SimLocaliser Teleop IROS’06 Workshop on Software Standardization

Alexei Makarenko

Performance

19

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Example: Simple 2-D Demo with Stage

20

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Current work : ACFR Segway Project

21

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Current work: ACFR/Berkeley Team for 2007 Urban Grand Challenge

22

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Potential Issues to Be Aware of

• Developer skills • Harder than monolithic applications • Requires certain discipline in • Structuring the project • Writing individual components

• But … for large projects, CBSE is a solution not a problem

• Testing • Harder than monolithic applications • But … using a middleware package removes many testing issues.

23

IROS’06 Workshop on Software Standardization

Alexei Makarenko

http://orca-robotics.sf.net

24

IROS’06 Workshop on Software Standardization

Alexei Makarenko

25

IROS’06 Workshop on Software Standardization

Alexei Makarenko

Propose the Interaction Model Architecture for HRI Components Standardization

YWW]UGXWUGXWG

zœT€–œ•ŽGjSGwUk oœ”ˆ•Ty–‰–›Gp•›Œ™ˆŠ›–•GyŒšŒˆ™ŠG{Œˆ”

j–•›Œ•›š pUGyŒŒžG–GoypG›ŒŠ•–“–Ž  ppUGp•›™–‹œŠ›–•G–GoypG›ŒŠ•–“–Ž  p“pUGw™–—–šŒG›ŒGš›ˆ•‹ˆ™‹¡ˆ›–•G–™GoypG j–”—–•Œ•›š

TYT

z{h{lTvmT{olThy{G 9p•G›ŒG“ˆš›G Œˆ™šSG™–‰–›ŠšG™ŒšŒˆ™ŠGˆšG–ŠœšŒ‹G”–™ŒGˆ•‹G”–™ŒG•›Œ•šŒ“ G–•Gˆ——“Šˆ›–•šG –™Œ•›Œ‹G›–žˆ™‹šGšŒ™ŠŒG›–Gœ”ˆ•šSG”Œ‹Šˆ“Gˆššš›ˆ•ŠŒSGˆ•‹Gœ”ˆ•T™Œ•‹“•ŒššG•GŽŒ•Œ™ˆ“UG 9{šG“Œ‹G›–GˆG•œ”‰Œ™G–G”—™ŒššŒG™Œšœ“›šG•GˆŠˆ‹Œ”ŠGˆ•‹G•‹œš›™ˆ“GŠ–•›ŒŸ›šSG•G›ŒG ‹ŒŒ“–—”Œ•›G–G™–‰–›šG•G‹Œ™šŒGšˆ—ŒšSGˆšGœ”ˆ•–‹šSG—Œ›šSG”Œ‹Šˆ“G›––“šSG–™Gˆ——“ˆ•ŠŒšUG 9~“ŒG›ŒGˆ——“Šˆ›–•G™ˆ•ŽŒGšGž‹ŒSGˆ•‹Gš–GšG›ŒG—–šš‰“ŒGšˆ—•ŽG–G›šG’•‹G–G™–‰–›šSGˆG Š–””–•GŒ“Œ”Œ•›GšG›ˆ›G›Œ Gˆ™ŒG•Š™Œˆš•Ž“ G•›™–‹œŠŒ‹G•›–G›Œ z–ŠŒ› G–Goœ”ˆ•šUG 9j™œŠˆ“Gˆš—ŒŠ›šG–G™–‰–›G‹ŒšŽ•Gˆ™ŒG›Œ™Œ–™ŒG›ŒG”–‹ˆ“›ŒšSG”ŒŠˆ•š”šGˆ•‹G›––“šG–Gœ”ˆ•T ™–‰–›G•›Œ™ˆŠ›–•SG–GŠ–””œ•Šˆ›–•Gž›Gœ”ˆ•G‰Œ•ŽšSGž›G›ŒGŒ•™–•”Œ•›Gˆ•‹G—–šš‰“ Gž›G –›Œ™G™–‰–›šSGˆ•‹G–G•›Œ™ˆŠ›–•G–G›ŒG™–‰–›ŠGš š›Œ”šGž›Gœ”ˆ•GŒ•™–•”Œ•›šSG”–‹Œ““Œ‹ –•G —Œ™š–•šNG•ŒŒ‹šGˆ•‹Gˆ‰›šSGˆ•‹G•–G”–™ŒGˆ™™ˆ•ŽŒ‹GˆŠŠ–™‹•ŽG›–G›Œ ™–‰–›Gœ•Š›–•šU

TZT

y–‰–›GˆšG{Œˆ”GtŒ”‰Œ™ 9{–žˆ™‹Goœ”ˆ•Ty–‰–›Gj–““ˆ‰–™ˆ›–•SGŽ“Ž›šG›ŒG”—–™›ˆ•ŠŒG–GŠ™Œˆ›•ŽG™–‰–› Šˆ—ˆ‰“›ŒšGˆ•‹G •›Œ™ˆŠŒšG›ˆ›Gˆ‹‹™ŒššGœ”ˆ•GŠ–•ŠŒ™•šGšœŠGˆšGš–Šˆ“Gˆ——™–—™ˆ›Œ•ŒššSGšˆŒ› SGˆ•‹G˜œˆ“› G–G šŒ™ŠŒUG 9y–‰–›šGˆ™ŒSG–™Gš––•Gž““G‰ŒSGœšŒ‹G•GšœŠGŠ™›Šˆ“G‹–”ˆ•šGˆšGšŒˆ™ŠGˆ•‹G™ŒšŠœŒSG”“›ˆ™ G‰ˆ››“ŒSG ”•ŒGˆ•‹G‰–”‰G‹Œ›ŒŠ›–•SGšŠŒ•›ŠGŒŸ—“–™ˆ›–•SG“ˆžGŒ•–™ŠŒ”Œ•›SGˆ•‹G–š—›ˆ“GŠˆ™ŒUG 9zœŠG™–‰–›šG”œš›GŠ––™‹•ˆ›ŒG›Œ™G‰Œˆ–™šGž›G›ŒG™Œ˜œ™Œ”Œ•›šGˆ•‹GŒŸ—ŒŠ›ˆ›–•šG–Gœ”ˆ•G ›Œˆ”G”Œ”‰Œ™šbG›Œ Gˆ™ŒG”–™ŒG›ˆ•G”Œ™ŒG›––“šG‰œ›G™ˆ›Œ™G˜œˆšT›Œˆ”G”Œ”‰Œ™šGž–šŒG›ˆš’šGˆŒG›–G ‰ŒG•›ŒŽ™ˆ›Œ‹Gž›G›–šŒG–Gœ”ˆ•šUG 9y–‰–›GˆšG{Œˆ”GtŒ”‰Œ™SGŽ“Ž›šG›ŒG”—–™›ˆ•ŠŒG–G‰œ“‹•ŽGŠ–™Œ šŠŒ•ŠŒGˆ•‹Gœ•‹Œ™š›ˆ•‹•ŽG›ŒG š–Šˆ“Gˆ•‹G›ŒŠ•Šˆ“GššœŒšG•Gœ”ˆ•T™–‰–›G•›Œ™ˆŠ›–•G•G›ŒGŠ–•›ŒŸ›G–G›Œˆ”šGˆ•‹GŽ™–œ—šG

T[T

z—ŒŠˆ“GpššœŒG–•Goœ”ˆ•Ty–‰–›Gp•›Œ™ˆŠ›–• 9

{ŒG™ŒŠŒ•›G›™Œ•‹G›–žˆ™‹G‹ŒŒ“–—•ŽGˆG•ŒžGŽŒ•Œ™ˆ›–•G–G™–‰–›šG›ˆ›GŠˆ•G—ˆ™›Š—ˆ›ŒG•G –œ™G“ŒšGˆ•‹GŒŸš›G•Gœ”ˆ•GŒ•™–•”Œ•›šGˆšG•›™–‹œŠŒ‹G›ŒG•ŒŒ‹G–™G•Œš›Žˆ›•ŽG›ŒG —ˆ™ˆ‹Ž”šSG›ŒŠ•˜œŒšSGˆ•‹G›ŒŠ•–“–ŽŒšG–™G›ŒG•›Œ™ˆŠ›–•G‰Œ›žŒŒ•G—Œ–—“ŒGˆ•‹G™–‰–›šUG

9

h•G”—–™›ˆ•›GŽ–ˆ“G–™G›ŒGŒ“‹G–Goœ”ˆ•Ty–‰–›Gp•›Œ™ˆŠ›–•GšG›–G‹ŒŒ“–—Gˆœ›–•–”–œšG ˆ•‹GšŒ”Tˆœ›–•–”–œšG™–‰–›šG›ˆ›G–—Œ™ˆ›ŒGž›•Gœ”ˆ•Gš—ˆŠŒšGˆ•‹G—“ˆ GˆG‰Œ•ŒŠˆ“G ™–“ŒG•G›ŒG‹ˆ“ G“ŒšG–G–™‹•ˆ™ G—Œ–—“ŒUG

9

p•›Œ™ˆŠ›–•G‰Œ›žŒŒ•G—Œ–—“ŒGˆ•‹G™–‰–›šG”ˆ G—–›Œ•›ˆ““ Gš—ˆ•G— šŠˆ“SGŠ–Ž•›ŒSG›ˆš’T ‰ˆšŒ‹SGš–Šˆ“SG–™GŒ”–›–•ˆ“G‹”Œ•š–•šUG

9

oœ”ˆ•T™–‰–›G•›Œ™ˆŠ›–•G—–šŒšG”œ“›TˆŠŒ›Œ‹G—™–‰“Œ”šSG™Œ˜œ™•ŽG•–›G–•“ G›ŒŠ•Šˆ“G ‰œ›Gˆ“š–GŠœ“›œ™ˆ“SGš–Š–“–ŽŠˆ“SG—š Š–“–ŽŠˆ“SG—“–š–—Šˆ“SGˆ•‹GŒŒ•GŒ›Šˆ“G Š–•š‹Œ™ˆ›–•šUG

0LWVXELVKL:DNDPDUX URERW

T\T

z—ŒŠˆ“GpššœŒG–•Goœ”ˆ•Ty–‰–›Gp•›Œ™ˆŠ›–• 9

o–žG›–G”–‹Œ“G›ŒG•›Œ™ˆŠ›–•G–GˆGœ”ˆ•G‰Œ•ŽGž›GˆG™–‰–›fG

9

o–žG›–G”ˆ•ˆŽŒG›ŒG— šŠˆ“SG•›Œ““ŒŠ›œˆ“SGˆ•‹GŒ”–›–•ˆ“GŒŸŠˆ•ŽŒ ‰Œ›žŒŒ•Gœ”ˆ•G ‰Œ•ŽšGˆ•‹G™–‰–›šfG

9

o–žG›–G™Œˆ“¡ŒGˆ•GŒŒŠ›ŒGŠ–””œ•Šˆ›–•G–G™–‰–›šGž›G›ŒGœ”ˆ•šGˆ•‹G›Œ™G Œ•™–•”Œ•›šfG

9

{ŒšŒG˜œŒš›–•šGˆ•‹G”ˆ• G–›Œ™šGˆ™ŒG›ŒGš›”œ“œšG–™G›šGz—ŒŠˆ“GpššœŒG–G›ŒGœ”ˆ•T y–‰–›G•›Œ™ˆŠ›–•SGˆ”Œ‹Gˆ›GŽˆ›Œ™•ŽG›ŒG“ˆ›Œš›G™Œšœ“›šG‰ G™–‰–›ŠšG™ŒšŒˆ™ŠŒ™šGˆŠ•ŽG ›ŒG‹Œ™šŒG—™–‰“Œ”šG™Œ“ˆ›Œ‹G›–Gœ”ˆ•Ty–‰–›G•›Œ™ˆŠ›–•U

0LWVXELVKL:DNDPDUX URERW

T]T

nŒ››•ŽG›–Gr•–žGz–Šˆ““ Gp•›Œ““ŽŒ•›Gy–‰–›š 9

oœ”ˆ•Ty–‰–›Gp•›Œ™ˆŠ›–•Gˆ•‹Gj–””œ•Šˆ›–•GšGˆG˜œŠ’“ GŽ™–ž•ŽG™ŒšŒˆ™ŠGˆ™ŒˆGˆ›G›ŒG •›Œ™šŒŠ›–•G–G™ŒšŒˆ™ŠGŒ“‹šGšœŠGˆšG™–‰–›ŠšSGŒ•Ž•ŒŒ™•ŽSG—š Š–“–Ž SGŒ›–“–Ž  ˆ•‹G Š–Ž•›ŒGšŠŒ•ŠŒUG

9

zŽ•Šˆ•›G•›ˆ›ŒšGˆ™ŒGŠœ™™Œ•›“ Gœ•‹Œ™žˆ Gœ•‹Œ‹G‰ G—œ‰“ŠS ˆŠˆ‹Œ”ŠSGŽ–Œ™•”Œ•›ˆ“G ˆšGžŒ““GˆšG•‹œš›™ˆ“G•›ˆ›ŒšSGŒŸ—“–™•ŽGˆ•‹Gˆ”•ŽGˆ›Gˆ‹ˆ•Š•ŽG›šG™ŒšŒˆ™ŠGŒ“‹G ˆ•‹G–—Œ••ŽGœ—G•–Œ“Gˆ•‹GŠˆ““Œ•Ž•ŽGˆ——“Šˆ›–•šUG

9

y–‰–›šG”–•ŽG–œ›G–G“ˆ‰–™ˆ›–™ Gˆ•‹G”ˆ•œˆŠ›œ™•ŽGŒ•™–•”Œ•›šGˆŠŒGˆ™‹G—™–‰“Œ”šG –G—Œ™ŠŒ—›–•SGˆŠ›–•Gˆ•‹GŠ–Ž•›–•UG

9

m–™G™–‰–›šG›–G‰ŒGˆŠŠŒ—›Œ‹GˆšGˆššš›ˆ•›šG–™GŠ–”—ˆ•–•šG•G—Œ–—“ŒNšG—™ˆ›ŒG–”ŒšGˆ•‹G ŒŒ™ ‹ˆ GŒ•™–•”Œ•›šG›ŒŠ•–“–ŽŠˆ“Gš–“œ›–•šG‹–G•–›GšœŠŒaG{ŒG‡œ”ˆ•G•G›ŒG“––—NSG ˆšG›ŒG—–›Œ•›ˆ“GŠœš›–”Œ™Gˆ•‹GœšŒ™Gž““G‹ŒŠ‹ŒG–•G›ŒGœ“›”ˆ›ŒGšœŠŠŒššG–GˆG‡–”ŒG ™–‰–›NGˆšGˆG—™–‹œŠ›UG

9

h——“Šˆ›–•Gˆ™ŒˆšG›ˆ›GŒˆ“ G•–“ŒGœ”ˆ•GŠ–•›ˆŠ›Gˆ™ŒGˆG—ˆ™›Šœ“ˆ™“ GŠˆ““Œ•Ž•ŽG ‹–”ˆ•UG

0LWVXELVKL:DNDPDUX URERW

T^T

nŒ››•ŽG›–Gr•–žGz–Šˆ““ Gp•›Œ““ŽŒ•›Gy–‰–›š 9

oœ”ˆ•Gš–ŠŒ›ŒšGˆŒGŒˆš“ Gˆšš”“ˆ›Œ‹G•ŒžG›ŒŠ•–“–ŽŒšSGšœŠGˆšG”–‰“ŒG—–•ŒšSG‰œ›G ›GšG“ŒššGŠ“Œˆ™G•GžŠGˆ——“Šˆ›–•Gˆ™ŒˆšG™–‰–›šGž““G‰ŒGˆŠŠŒ—›Œ‹UG

9

y–‰–›šGˆšGŒ”‰–‹Œ‹G‰Œ•ŽšSG— šŠˆ“SG—–šš‰“ Gœ”ˆ•–‹G–™Gˆ•‹™–‹GŒ•››ŒšG›ˆ›Gšˆ™ŒG –œ™G“•ŽGŒ•™–•”Œ•›šGˆ•‹GˆŠŠ–”—ˆ• G–œ™G“ŒšGž““GˆŒGˆGŠŒ™›ˆ•G‹ŒŽ™ŒŒG–G ˆœ›–•–” SG•›ˆ›ŒSGŠ–Ž•›ŒGš’““šGˆ•‹Gž““GŠ–””œ•Šˆ›ŒGˆ•‹G•›Œ™ˆŠ›Gž›G—Œ–—“ŒG•G žˆ šG•š—™Œ‹G‰ Gœ”ˆ•Tœ”ˆ•GŠ–•›ˆŠ›UG

9

p•›Œ™ˆŠ›–•Gˆ•‹GŠ–””œ•Šˆ›–•G–GŒ”‰–‹Œ‹G— šŠˆ“G™–‰–›šGž›Gœ”ˆ•šGšG”œ“›T”–‹ˆ“SG ˆ•‹G•–“ŒšG‹ŒŒ—GššœŒšG–Gš–Šˆ“G•›Œ““ŽŒ•ŠŒSGŠ–””œ•Šˆ›–•Gˆ•‹G•›Œ™ˆŠ›–•G›ˆ›G ˆŒG›™ˆ‹›–•ˆ““ G‰ŒŒ•Gš›œ‹Œ‹G—™”ˆ™“ G•G—š Š–“–Ž Gˆ•‹G–›Œ™Gˆ™ŒˆšUG

9

{ŒG‹ŒšŽ•G–GˆG™–‰–›NšG‰Œˆ–œ™SGˆ——Œˆ™ˆ•ŠŒSGˆ•‹GŠ–Ž•›ŒGˆ•‹Gš–Šˆ“Gš’““šGšGŽ“ G Šˆ““Œ•Ž•ŽSGˆ•‹G™Œ˜œ™ŒšG•›Œ™‹šŠ—“•ˆ™ GŠ–““ˆ‰–™ˆ›–•šGˆŠ™–ššG›ŒG›™ˆ‹›–•ˆ“G ‰–œ•‹ˆ™ŒšG–GŒš›ˆ‰“šŒ‹G‹šŠ—“•ŒšU

0LWVXELVKL:DNDPDUX URERW

T_T

pUGyŒŒžG–GoypG{ŒŠ•–“–Ž  XUGkŒ•›–•G YUGjˆ™ˆŠ›Œ™š›Šš ZUGj“ˆššŠˆ›–•

T`T

pTXUGkŒ•›–• Human-Robot Interaction (HRI) is a core technology that can naturally interact between human and robots through robot camera, microphone, and various sensors for intelligent service robots.

0LWVXELVKL:DNDPDUX URERW

TXWT

pTYUGjˆ™ˆŠ›Œ™š›Šš HRI technology is different from Human-Computer Interaction (HCI) in that robots have an autonomous movement, bidirectional feature of interaction, and diversity of control level. To development an effective HRI, the system with module architecture to implement convenience between human and robots, cooperation, and friendship should be needed.

TXXT

pTZUGj“ˆššŠˆ›–• Vision-based HRI : face recognition, gesture recognition, behavior recognition, facial expression recognition Audio-based HRI : speech recognition, speaker recognition, sound localization, emotional recognition Others : PDA-based HRI interface, emotion generation and expression

TXYT

ppUGp•›™–‹œŠ›–•G–GoypG{ŒŠ•–“–Ž  OoypG{Œˆ”GyŒšŒˆ™ŠGhŠ››ŒšG•Gl{ypP XUG}š–•T‰ˆšŒ‹GoypG›ŒŠ•–“–Ž  YUGhœ‹–T‰ˆšŒ‹GoypG›ŒŠ•–“–Ž  ZUG–›Œ™šG

TXZT

ppTXUG}š–•T‰ˆšŒ‹Goyp Face detection/tracking/recognition: the face detection/tracking allows robot to detect a human face from images obtained through robot camera and to track a movement of the human face during natural conversation with the robot. The face recognition/verification allows robot to recognize a member of family and to verify the identity of the human face.

1(&3DSHUR URERW TX[T

ppTXUG}š–•T‰ˆšŒ‹Goyp User identification using semi-biometrics ƒ

The face recognition/verification is furthermore allows robot to know the direction, distance, and identity of the user using semibiometrics information such as user’s cloth color and height.

TX\T

ppTXUG}š–•T‰ˆšŒ‹Goyp Gesture recognition: Meaningful gestures could be represented by both temporal hand movements and static hand postures. It can be efficiently used in noisy environment and can also give active services by recognizing user’s intension for intelligent robots

+RQGD$6,02URERW TX]T

ppTXUG}š–•T‰ˆšŒ‹Goyp Facial expression recognition: Robotic systems for natural user interaction performs facial expression recognition, since facial expressiveness is regarded as a key component to developing personal attachment. The six basic facial expressions are the expressions of the six basic emotions of humans: anger, disgust, fear, happiness, sadness, and surprise.

0,7.,60(7URERW TX^T

ppTXUG}š–•T‰ˆšŒ‹Goyp Caller identification ƒ

The robot recognizes the face image of caller with hand gesture. The hand gesture detection is obtained by hand shape around face image of caller. After recognizing hand gesture, robot moves forward caller.

TX_T

ppTXUG}š–•T‰ˆšŒ‹Goyp Human following ƒ

The robot follows specific person by using color histogram of cloth and information of depth map obtained from stereo camera of moving robot. Human following obviously requires real-time ability to respond to the changing position of the person to be followed.

TX`T

ppTXUG}š–•T‰ˆšŒ‹Goyp Human detection and tracking Object recognition Human following Behavior recognition Posture recognition

TYWT

ppTYUGhœ‹–T‰ˆšŒ‹Goyp Speaker recognition: the text-independent speaker recognition allows robot to recognize the identity of a speaker during natural conversation with robot. Furthermore, this technique allows a speaker to communicate with robot through spontaneous speech.

(75,:HYHU URERW

TYXT

ppTYUGhœ‹–T‰ˆšŒ‹Goyp Sound localization: the main goal of this technique is to find the direction of the call voice by uttering its name or hand clapping sound. After that, robot moves toward the position where the sound is generated,

7RVKLED$SUL VKDUSHDUURERW

TYYT

ppTYUGhœ‹–T‰ˆšŒ‹Goyp Speaker and speech recognition ƒ

This technique allows a speaker to communicate with robot through spontaneous speech. With text-independent speaker recognition, it is able to provide useful information such as daily life schedule and TV program suitable to the speaker.

TYZT

ppTYUGhœ‹–T‰ˆšŒ‹Goyp Emotion recognition Sound source separation Speech understanding Speech synthesis

TY[T

ppTZUGv›Œ™š PDA-based HRI interface ƒ

PDAs can be used to interact with robots. Furthermore, they provide a suitable interaction device for teleoperation and a touchscreen interaction capability

TY\T

ppTZUG–›Œ™š Emotion generation and expression ƒ

This is a technique that the robots generate and express their emotion corresponding to human’s behavior

.,7(&+(YH5

.$,67$OEHUW+8%2

TY]T

p““UGw™–—–šŒG›ŒGš›ˆ•‹ˆ™‹¡ˆ›–•G–™GoypG j–”—–•Œ•›š

TY^T

~ˆ›G‹–GœšŒ™šGžˆ•›f

yŒ˜œ™Œ”Œ•›šG™–”GœšŒ™š

~ŒG”ˆ G•ŒŒ‹GUUG–™Gš›ˆ•‹ˆ™‹š }–ŠŒG–™ }š–•OnŒš›œ™ŒSGˆŠŒSGŒ›ŠUPG•›Œ™ˆŠŒš

z”—“ŒGˆ•‹G•ˆ›œ™ˆ“G•›Œ™ˆŠŒš

j–””–•G–™G•ˆ›œ™ˆ“Gž–™‹šG•›Œ™ˆŠŒšG uˆ›œ™ˆ“GŽŒš›œ™ŒG•›Œ™ˆŠŒš

z”—“ŒGˆ•‹G•ˆ›œ™ˆ“GŠ–””ˆ•‹š O‹–•’›Gžˆ•›G™Œ”Œ”‰Œ™G–ŠŒG–™GŽŒš›œ™ŒGŠ–””ˆ•‹šP

pG›Œ GˆŒG›–G™Œ”Œ”‰Œ™G’Œ ž–™‹S {Œ Gžˆ•›G›–G™Œ”Œ”‰Œ™Gšˆ”ŒGž–™‹G–G›Œ™G“ˆ•ŽœˆŽŒG –™G›ŒGšˆ”ŒGšŒ™ŠŒG–G‹Œ™Œ•›G™–‰–›G”ˆ’Œ™šU

rŒ ž–™‹ š›ˆ•‹ˆ™‹š

OpG –œGžˆ•›G›–G’•–žG›ŒGžŒˆ›Œ™SG –œG‘œš›GœšŒG“žŒˆ›Œ™”P

pG›Œ GˆŒG›–GœšŒG’Œ ž–™‹Gˆ•‹GpG›Œ Gˆ™ŒGr–™Œˆ•S {Œ Gžˆ•›G›–GœšŒGr–™Œˆ•G’Œ ž–™‹šU OpG›Œ Gžˆ•›G›–G’•–žG›ŒGžŒˆ›Œ™SG›Œ Gžˆ•›G›–GœšŒG“⇔㙜” ”Œˆ•šGžŒˆ›Œ™P

TY_T

rŒ ž–™‹ p•›Œ™•ˆ›–•ˆ“¡ˆ›–• z›ˆ•‹ˆ™‹

~ˆ›G‹–G™–‰–›G”ˆ’Œ™šGžˆ•›f

yŒ˜œ™Œ”Œ•›šG™–”G™–‰–›G”ˆ’Œ™š

~ŒG”ˆ G•ŒŒ‹GUUG–™Gš›ˆ•‹ˆ™‹š

{Œ Gžˆ•›G›–G’•–žG•›Œ™ˆŠŒGŽœ‹ŒG“•Œ m–™GoypGœ•Š›–•šGOš—ŒŒŠSGš–•SGŒ›ŠUP

z›ˆ•‹ˆ™‹G•›Œ™ˆŠŒš

{Œ Gžˆ•›G›–GœšŒGŒŸš›•ŽGœ•Š›–•š ˆ‰–œ›GoypG›ŒŠ•–“–ŽŒš

z›ˆ•‹ˆ™‹GŠ–”—–•Œ•›š

{Œ GŠˆ••–›GˆŒGŒ•–œŽG”ˆŠ•ŒG—–žŒ™G›–GœšŒ oypGœ•Š›–•šGˆ•‹G›Œ Gžˆ•›G›–GœšŒG‘œš›GoypG šŒ™ŠŒšG™–”G™Œ”–›ŒGšŒ™Œ™UG

yŒ”–›ŒGoypG zŒ™ŠŒGj–”—–•Œ•› z›ˆ•‹ˆ™‹š

{Œ GGžˆ•›G›–Gœ—Ž™ˆ‹ŒGœ•Š›–•šG–™Gˆ“Ž–™›”šG Œˆš“ Gž›G“–žG”ˆ•›Œ•ˆ•ŠŒGŠ–š›šU

h“Ž–™›”G p•‹Œ—Œ•‹Œ•› ˆ™Š›ŒŠ›œ™Œ

TY`T

~ˆ›G‹–Gh——UGkŒŒ“–—Œ™šGžˆ•›f

yŒ˜œ™Œ”Œ•›šG™–”Gh——U v™Gœ•Š›–•G‹ŒŒ“–—Œ™š

~ŒG”ˆ G•ŒŒ‹GUUG–™Gš›ˆ•‹ˆ™‹š

{Œ Gžˆ•›Gšˆ”ŒG•›Œ™ˆŠŒG•‹Œ—Œ•‹Œ•›G–•G™–‰–›G › —ŒšGˆ•‹G”ˆ’Œ™šG›–G‹ŒŒ“–—Gˆ——“Šˆ›–•šU

{Œ G‹–G•–›Gžˆ•›G›–G‹ŒŒ“–—G•ŒžGˆ——“Šˆ›–•šG–™G ›ŒGšˆ”ŒGšŒ™ŠŒšG–™Gœ•Š›–•šGˆŽˆ• –™GŒˆŠG‹Œ™Œ•›G™–‰–›

TZWT

z›ˆ•‹ˆ™‹Gp•›Œ™ˆŠŒš

z›ˆ•‹ˆ™‹G j–”—–•Œ•›š v™ ™Œ”–›ŒGšŒ™ŠŒ j–”—–•Œ•›šGš›ˆ•‹ˆ™‹

z›ˆ•‹ˆ™‹Gp›Œ”š

m™–”GœšŒ™š }–ŠŒG–™ }š–•OnŒš›œ™ŒSGˆŠŒSGŒ›ŠUP •›Œ™ˆŠŒš j–””–•G–™G•ˆ›œ™ˆ“Gž–™‹š •›Œ™ˆŠŒšG uˆ›œ™ˆ“GŽŒš›œ™ŒG•›Œ™ˆŠŒš rŒ ž–™‹ š›ˆ•‹ˆ™‹š

rŒ ž–™‹ p•›Œ™•ˆ›–•ˆ“¡ˆ›–• z›ˆ•‹ˆ™‹

m™–”G”ˆ’Œ™š

m™–”Gh——UG‹ŒŒ“–—Œ™š

z›ˆ•‹ˆ™‹G•›Œ™ˆŠŒš z›ˆ•‹ˆ™‹Gp•›Œ™ˆŠŒš

z›ˆ•‹ˆ™‹GŠ–”—–•Œ•›š yŒ”–›ŒGoypG zŒ™ŠŒGj–”—–•Œ•› z›ˆ•‹ˆ™‹š

z›ˆ•‹ˆ™‹G j–”—–•Œ•›š v™G™Œ”–›ŒGšŒ™ŠŒ j–”—–•Œ•›šGš›ˆ•‹ˆ™‹

h“Ž–™›”G p•‹Œ—Œ•‹Œ•› ˆ™Š›ŒŠ›œ™Œ

TZXT

z›ˆ•‹ˆ™‹Gp›Œ”š

m™–”GœšŒ™š }–ŠŒG–™ }š–•OnŒš›œ™ŒSGˆŠŒSGŒ›ŠUP •›Œ™ˆŠŒš j–””–•G–™G•ˆ›œ™ˆ“Gž–™‹š •›Œ™ˆŠŒšG uˆ›œ™ˆ“GŽŒš›œ™ŒG•›Œ™ˆŠŒš rŒ ž–™‹ š›ˆ•‹ˆ™‹š

rŒ ž–™‹ p•›Œ™•ˆ›–•ˆ“¡ˆ›–• z›ˆ•‹ˆ™‹

m™–”G”ˆ’Œ™š

m™–”Gh——UG‹ŒŒ“–—Œ™š

z›ˆ•‹ˆ™‹G•›Œ™ˆŠŒš z›ˆ•‹ˆ™‹Gp•›Œ™ˆŠŒš

zŒ•šŒVlŸ—™Œšš–•Gp•›Œ™ˆŠŒš z›ˆ•‹ˆ™‹GŠ–”—–•Œ•›š yŒ”–›ŒGoypG zŒ™ŠŒGj–”—–•Œ•› z›ˆ•‹ˆ™‹š

zŒ•šŒVlŸ—™Œšš–• p•–™”ˆ›–•G”–‹Œ“

z›ˆ•‹ˆ™‹G j–”—–•Œ•›š v™ ™Œ”–›ŒGšŒ™ŠŒ j–”—–•Œ•›šGš›ˆ•‹ˆ™‹

h“Ž–™›”G p•‹Œ—Œ•‹Œ•› ˆ™Š›ŒŠ›œ™Œ zŒ•šŒVlŸ—™Œšš–• h™Š›ŒŠ›œ™Œ TZYT

t–‹Œ“G–GoypGj–”—–•Œ•›Gmœ•Š›–•

TZZT

p•›Œ™ˆŠ›–•Gt–‹Œ“Gh™Š›ŒŠ›œ™ŒG–™Goyp ~ˆ›G‹–GžŒGšœŽŽŒš›G–™G›ŒGš›ˆ•‹ˆ™‹

zŒ•šŒG•›Œ™ˆŠŒ y–‰–›GhpG–™GšŒ™ŠŒGh——U OG••Œ™G–™G–œ›Œ™G–G™–‰–›P zŒ•š–™G•—œ›

lŸ—™Œšš–• i Gœš•Ž ˆŠ›œˆ›–™šG–™GšŠ™ŒŒ•

p•–™”ˆ›–•G”–‹Œ“š OzŒ•šŒSGlŸ—™Œšš–•P

TZ[T

ŒŸ—™Œšš–•G•›Œ™ˆŠŒ

p•–™”ˆ›–•Gt–‹Œ“

z Our proposed standard for HRI consists of one model and two interfaces – information model and recognition and expression interfaces

z Information model designs the types and structures of information objects as well as their management methods. z The objects are two types 9 recognition and expression.

z First of all, information model consists of the types structures and protocols of information from sensors and applications of robot.

TZ\T

zŒ•šŒGp•–™”ˆ›–•Gt–‹Œ“

z Sense information model says When, Who say, What, Where from Human to Robot information z Sense Information model consists of 4W Model 9 Who information : face recognition, speaker recognition, etc. 9 Where information : vision based location information, landmark based location info., etc. 9 What information : speech recognition, gesture recognition, etc. 9 When information: scheduler, clock, etc.

TZ]T

zŒ•šŒGp•–™”ˆ›–•Gt–‹Œ“GkŒ›ˆ“Œ‹ j–•›•œŒ…

z Who information model 9 Generally it can be obtained by face recognition , speaker recognition, etc. 9 We define whoinfo as path + userId information • Ex) “//korea/south/taejon/etri/mrsong” : “//korea/south/taejon/etri/” Is path, “mrsong” is id 9 If the robot do not know the correct user id, they send WhoHint(face image or voice sound) to server • Ex) WhoHint = { Face Image or Voice sound } • if server do not know location • WhoHint = { (Image or Sound), Location path}

TZ^T

zŒ•šŒGp•–™”ˆ›–•Gt–‹Œ“GkŒ›ˆ“Œ‹ j–•›•œŒ…

z Where information model 9 Generally it can be obtained by location finding algorithm – We define whereinfo as address(or path) + position information • Ex) Can be direct type or near type • direct type: “//korea/south/taejon/etri/7thbuilding/L864/x=10;y=1 3” • near type : Æhave to translate direct position. • “//korea/south/taejon/etri/robotdivion/hriteam/song”

TZ_T

zŒ•šŒGp•–™”ˆ›–•Gt–‹Œ“GkŒ›ˆ“Œ‹

•

j–•›•œŒ…

What information model – Generally it can be obtained by speech recognition, gesture recognition, any other command recognition methods. – We define whatinfo as natural string or path command • Ex)

• Natural string • “change the TV channel”, or “show today schedule” • Command format with path information • TV (“//korea/south/taejon/etri/7thbuilding/L864/38TV”). channel.changeUp(); – If the robot do not understand a correct command, the robot send WhatHint( gesture image, voice or any other command data) to server • Ex) WhatHint = { Image, voice or other command data(IR remote controller info, etc.) }

TZ`T

zŒ•šŒGp•–™”ˆ›–•Gt–‹Œ“GkŒ›ˆ“Œ‹ j–•›•œŒ…

• When information model – Generally it can be obtained by clock and scheduler, etc. – WhenInfo show current time – We define wheninfo as time information

• Ex) • Date Time : from clock • “2006-09-13-14:33:23” • If command use when info( in case users say “response until tommorow”), it can be processed robot AI with Whatinfo and Wheninfo. – If the robot do not send wheninfo, servers use their internal clock.

T[WT

zŒ•šŒGp•›Œ™ˆŠŒš

• Sense Interfaces use sense information model as their arguments and results. • include interfaces for 4W info model and Hint model • include sensor type interfaces and processor type interfaces – Sensor type: active, event style – Processor type: passive, it can be used in a application.

T[XT

zŒ•šŒGp•›Œ™ˆŠŒšGkŒ›ˆ“Œ‹

• Sensor(Event) type Interface: – On… interfaces • OnWho(WhoInfo), OnWho(WhoHint) • OnWhere(WhereInfo), OnWhere(WhereHint) • OnWhat(WhatInfo), OnWhat(WhatHint) • OnWhen(WhenInfo) • OnSee(ViewImage), OnSound(SoundData) – If robot cannot sense anything, In some case, robot send all images or sounds to be processed to server.

T[YT

zŒ•šŒGp•›Œ™ˆŠŒšGkŒ›ˆ“Œ‹

• Processor(Query) type Interface : – Get… interfaces • GetWho(), GetWhoHint() • GetWhere(), GetWhereHint() • GetWhat(), GetWhatHint() • GetWhen() • GetSee(), GetSound() – Get… interfaces have block mode and non-block mode

T[ZT

lŸ—™Œšš–•Gp•–™”ˆ›–•Gt–‹Œ“

• Expression Information model consists of – How information • Just say abstract expression commands: first step • It can be translated each methods for each robots: second step • Therefore, Expression information model require some special architecture lŸ—™Œšš–•G”–‹Œ“GOˆ‰š›™ˆŠ›P lŸ—™Œšš–•o•› ”–‹Œ“GO‹Œ›ˆ“Œ‹GˆŠ›–•šP lŸ—™Œšš–•yŒ•‹Œ™ ˆ™Š›ŒŠ›œ™Œ T[[T

lŸ—™Œšš–•Gp•–™”ˆ›–•Gt–‹Œ“GkŒ›ˆ“Œ‹

• Expression Model – It can be produced from Robot AI or service application in server – It can be processed on robot side or server side. • In case of robot side, robots process expression words • In case of server side, servers process words by calling robot control interfaces.

T[\T

lŸ—™Œšš–•Gp•–™”ˆ›–•Gt–‹Œ“GkŒ›ˆ“Œ‹

• Expression Model – It have to be defined with standard wordÆ we have to define some robot expression word. • Ex) “say()”, “approach()”, etc. • It may match to same interface – “say(hello)” can match to robot.say(“hello”) – Some case, an expression model may not matched to an expression interface • Ex) expression can be “greeting” but a robot may not do “greeting” because they do not know the meaning of “greeting” – In this case, a robot use expression hint. T[]T

lŸ—™Œšš–•Gp•–™”ˆ›–•Gt–‹Œ“GkŒ›ˆ“Œ‹G

• Expression Hint Model – If a robot cannot process expression, they want expression hint. – ExpressionHint shows action flow instead of abstract words.

T[^T

lŸ—™Œšš–•Gp•–™”ˆ›–•Gt–‹Œ“GkŒ›ˆ“Œ‹G

• Expression Hint Model – It can be differently implemented by each robot makers or robot service application developer . • ExpressionModel: Greeting – A maker: – Expression(Greeting) = ExpressionHint( TTS(“Hello”)) – B maker – Expression(Greeting) = ExpressionHint( robot.arm.shakeHands());

T[_T

lŸ—™Œšš–•Gp•›Œ™ˆŠŒš

• Expression Interfaces use expression information model or expression hint model as their arguments and results. – Expression Interface… • ExpressionRenderer.Expression(“some expression”) • ExpressionRenderer.Expression(“expression”,Expr essionHint hint)

T[`T

lŸ—™Œšš–•GyŒ•‹Œ™Œ™

• Expression Renderer process Expression, ExpressionHint • If Renderer cannot process Expression, it try to get ExpressionHint from robot makers. • Renderer is a process engine or task engine. • It can be placed on application server or robot side.

T\WT

j–•Š“œš–•

~ŒG•ŒŒ‹G›–G”ˆ• G–—•–•šG ˆ•‹G‘–•G™–”G–›Œ™š

{ˆ•’G€–œ T\XT

Integration of Ubiquitous Robotic Systems using Open Standards IROS IROS 2006 2006 -- Workshop Workshop on on Robotic Robotic Standardization Standardization Beijing Beijing (China) (China) – – October October 10 10thth ,, 2006 2006

Olivier LEMAIRE National Institute of

Advanced Industrial Science and Technology (AIST)

Usage of Robotics Technology

Robotics @ AIST / ISRI AIST

…

- Largest public research organization in Japan Intelligent Systems - 3200 Employees

…

Research Institute

- 50 Research 3D Centers and Research Institutes Vision Systems Field Systems Safety Intelligence

Distributed Systems

Research Group : - Research fields including

Design Research Gr. Autonomous Behavior *

Research Group

LifeHumanoid Science & Technology Ubiquitous Functions

Control Research Gr. * Information Research Group Research Group Technology

Research Group

Task Intelligence

Lighter-than-Air

Research Group

Research Group

Joint Japanese-French Robotics Lab.

* Nanotechnology, Materials & Manufacturing * Environment & Energy * Geological Survey & Applied Geoscience * Metrology and Measurement Technology

- From Fundamental Research to Applied Research Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Robotics @ AIST / ISRI / UFRG – Robotic Space Chandelier Light

Chandelier Fan Security System

• •

Auto Door

• Portable Switch

Decorative Light

Multi Consent

Multimedia Syst.

•

Robotic Room for Daily life support The Room is the Robot where each device provides a simple function Sum of functions provides more intelligent functions Constraints : – – – –

Home Server

•

Need High Flexibility/Customization Event Driven No a priori design time decision More about Human /system Interaction than control

Use of wireless Active RFID tags with Digital I/O to control appliances and gather sensor states – Use I/O tagging to achieve PnP Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Usage of Robotics Technology – Ubiquitous Robots

Environment with embedded ID Tags

• Librarian Robot • Tidy books back to bookshelf

Mobile Robot Visio Sensor (Laser, Camera)

ID Tag in the Books

Gripper

• Minimal on board processing (only sensors) • All intelligence distributed

Force Sensor

ID Tags in the Floor

Arm

Database

Mobile Base (Wheeled)

Bookshelf with Tag Reader

Interface between Robot and Tag System Rack with Tag Reader

Tag Reader for Localization

• Taking advantage of infrastructure (ID Tags) for many usual robotic tasks – Localization, Book Recognition…

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Usage of Robotics Technology – Integration Internet Infr astr uct ur e (Human Interface)

Human Interface Management

Behavior Management

Door RT Component Switch RT Component Lock RT Component

Light RT Component IDTagIdentifier RT Component Fan RT Component

RTM Infr astr uct ur e (High Level Contr ol)

Wir eless UFAM Infr ast ructur e (Low Level Contr ol)

Action Management

Camera RT Component Speech RT Component Identifier RT Component

UFAM Gateway

Web Ser vic e Infr astr ucture

Library Robot RT Component

Motor RT Component PD Controler RT Component

Database WS Component

Encoder RT Component

Book Shelf WS Component

Traj. Generator RT Component

Tag Reader WS Component

Camera RT Component

SLAM WS Component

Gripper RT Component

Wheels WS Component

Arm RT Component

Tag Reader WS Component

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Needs for Standardization of Robotics Technology

Technology Re-use • Develop new robots faster and cheaper by integrating COTS Generic

Robot

Robot Motion Controller

Robot Sensor Data Analyzer Human Human Interface Other Other

Algorithm

Robotic Technology Component Catalog

Monitoring Specifically ng Specifically Console developed developedfor for Application ApplicationAA

Head Communication Communication Controller via Proprietary via Proprietary closed closedprotocol protocol

One Onebig bigapplication application that manages

Fingerprint Sensor

Light Robo Robot R Hum uRobo Human H Hum Interf e Interface In n Ap f IInterf for oAppli for Ap B

Application A

Integration IntegrationTools Tools Admin AdminTools Tools

This kind of Modular and Flexible Application : : Monolithic and Specific Application •• Emphasize reuse of smaller specialized components Cannot address a wide enough market to be very profitable •• Takes Timetotodevelop Integrate Takes aa short long time and to maintain •• Can be distributed over a network to share resources Is Error-prone •• Is totallyresources open to other systems customization Reaches saturation forand hightointelligent system •• As moreeasily accessible, canwith generate Cannot integrate otherinterest systemsfrom different business actors and the hardware market manufacturer • Is stimulate bound to the Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Technology Re-use •

Expand the Robotics Business by sharing the burden of development

H/W Maker

H/W Maker (Academia, Business)

H/W Dvlpt. H/W Dvlpt.

S/W Developer (Academia, Business)

S/W W Dvlpt. S/W Dvlpt. ion H/WIntegration Integration H/W

Tool Vendors

Develop Robot

S/W W Integration on S/W Integration

Robot R obotIntegration Integration Integratio tion Robot Robot Maker

Testing Testing Marketing Marketing Maintenance Maintenance

System Integrators Independent Testing Org. Service Provider

Customization n Customization End User

End User

Use Use ExtraServices Services Extra

Contents Providers

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Manage complexity of a broad and complex Field •

Robotics is a multi-dimensional discipline – Integration of existing sciences – Dealing real world environment Real World Environment

Software Systems

Robotics

Mechanical Systems

Electrical Systems

•

A roboticist must master all Mechanical, Electrical, Software sciences AND deal with the complexity and uncertainties of a real world uncontrolled environment -> Only Superman can do that !! Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Manage complexity of a broad and complex Field Artificial Communication cation Artificial Communication Agent ControlLoop Loop Position Agent Intelligence Robot Position Control Intelligence Robot t Mapping g Mapping Mobile Unmanned Mobile Unmanned Neural Robot Neural Vehicle Grasping Robot Grasping …. Military Vehicle …. Synchronization Military Hospital Synchronization Hospital Network Robot Real-Time Real Time e Concurrency Network Robot Real-Time Robot Concurrency c sion Decision Robot Decision Mission on …. Mission …. Safety Sequencing Safety Sequencing Vision Speech Vision Speech Humanoid Data Processing Data ggHumanoid Processing Robot Communication n Manipulation Welfa f Robot Welfare Communication D Deployment Manipulation Welfare Deployment Reptile b t Robot Reptile Robot Robotics Robot Biped Robot Robot e Biped Robot Man/Machine Semantics Man/Machine Semantics …. Interac Interaction …. Interaction Sensor Sensor Home Composition Home Management Map Composition Management Map p Robot Robot Kinematics Image eKinematics Ma anagem nagem me entt Management Image Management Request Re s st PathTrajectory Digital Request Trajectory PathAnalog Digital Analog Planning Control Control Planning Localization Localization Security y e e Wheeled Entertainment Security Wheeled Entertainment r Sensor Robot Robot …. Sensor Robot Robot Recognition …. Recognition Fusion Fusion Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Manage complexity in multi-robot systems Group Communications

Global

Group

Group

Reasoning

Rules

Local

Robot

Local

Map

Reasoning

Rules

Map

Robot Sensors and Controls Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Everything but Static (Dynamic (Dynamic Configuration) Configuration)

We cannot shutdown a Robotic System every time an entity is added or removed

Specialized Specialized Monitoring Monitoring Console Console

System configuration needs to adapt to hardware mobility in a real-time manner We need a way to handle dynamic connection between components as well as their modeling We need a way to manage dynamic properties of components

Generic Generic Monitoring Monitoring Console Console

Robot Robot Head Head Controller Controller Robot Robot Controller Controller

Robot Robot Sensor Sensor Data Data Analyzer Analyzer Fingerprint Fingerprint Sensor Sensor

Light Light Robot Robot Human Human Interface for Interface Hospital a for A Hospital B B A

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Question of Point of View (Inheritance, (Inheritance, Composition Composition and and Granularity Granularity of of Components) Components)

The robot can be considered as a whole or as the sum of its components... Or anything else We need a way to let the same entity provide different views

+

Light with a dimmer is still just a light We need a way to define and advertise complex services without breaking the semantic of the underlying basic services A camera on a mobile robot becomes a mobile camera We need a way to compose, at integration time, functionalities defined at design time

= A mobile Robot Or A Camera Or A Mobile Camera Or A Robot with Camera

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Relationships are important (Spatial (Spatial and and Temporal Temporal Integrity) Integrity)

Software distributed entities are location independent, but the pieces of hardware they control are not We need a shared representation of the Robotic World and a way to clearly define robotic entities physical relationship Data generated from Robotic Entities may go through several other entities before reaching their destination - Delay may be induced - Information concerning the origin of the data might be lost

㻛

We need a way to ensure spatial and temporal integrity of the data

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

To go or not to go ? (Conflict (Conflict Management, Management, Prioritization Prioritization of of Tasks) Tasks)

Do that ! Do this ! In a distributed Robotic System, we cannot assume anything about the synchronization of task Conflicting requests to (especially hardware) entities may lead to incoherent behavior of the system

????

Some request may be more important than others (especially safety related ones) Go there !

Come Here !

We need a way to manage conflicting requests to robotic entities and to prioritize them in an homogenous way

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Safety First (Conformance) (Conformance)

Even if a Robotic System cannot be 100% reliable, it has to be 100% safe ! It is NOT possible to assert that a robotic system is 100% safe in an unmanaged real world environment Companies are reluctant to put on the market a robotic system that could cost them fortunes in potential product liability claims

Moving Hardware

+ Unskilled User

Standards in robotics could help developing a conformance framework defining acceptable safety and performance levels

=

Conforming Robot makers, protected against abusive PL claims, can develop more challenging robots User’s trust in the technology improves

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Tools for the people ! Robotic Systems will be more often used by people with minimal technical skills Robotic Systems must become easier to program and to manage Many tools are necessary to manage the different aspects of the construction of a Robotic System Functionality design Application design Application configuration Simulation Monitoring

We need a way to let all these tools homogenously and interchangeably work together

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

We need a Framework,… not a Patchwork ! There are standard frameworks addressing problems related to distributed objects Most of them are either too general - Hard to understand and support … or too limited - Some specific aspects of the robotic technology are not covered They can’t always integrate well - Based on different technologies - Overlapping and conflicting

Robotic Technology Fault ault Fault Fau Tolerance rance Tolerance erance

… … Real-Time Real-Time Real ea Time Communication C ommuninicatio cationn Communication Commun

Home Home tomation Automation utomation utomatio Automation

… …

Service Serv Se Service Management Manag Management Mana

Data Data Distribution nn D Distribu Distributed Distribution Distributio on o Distrib s b Distributed Simulation SSimula Simulat Simulation Relationship Relationship Relationship Management M Man ann Man na nagemen agement Management

We need a framework that would homogeneously address the problematic of robotic system

Concurrency Concurren ncy Concurrency Concurre nccyy Management M t Management

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Some frameworks do exist ... Many research groups around the world have been and are still trying to address Most of them take a similar approach which indicates that a consensus could be reached, but... Most of them are research oriented Technically correct, unusable as is None of them as yet been backed up by the industry

Some companies are trying to develop their own solution which usually

Robotic Robotic Technology Robotic Technology Technology

Robotic Technology Robotic Technology Robotic Technology

Cover only their needs Is proprietary

None of these solutions is even close to reach the volume to make it a defacto standard

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Requirements for Interoperability

Infrastructure Paradigms

Semantics

Common Mechanisms

Interoperability in Robotics Technology

Common Data / Objects

Protocols

Architecture Mediators Common Facilities

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Mapping

Mapping

Call / Support

Call / Support

Adapter

Development API Device Driver API

Hardware

Hardware Layer

Hardware

(optional)

-External platform APIs supporting Interface Definition extension, the Need Standardization : (by InCommunication real,for the basic model of independent as many concepts asmapping possibleto the concept mechanisms defined in the customization model (by Adapter itAND depending on the communication technology adopted) strong and guidelines for model extension, -extension, Hardwarethe capability description, functional tagging mapping to it depending on the Capability Descriptors programming adopted) - Mechanical /language Electrical Interconnections (out of scope) Internal Communication External CommunicationAPI APIs and protocol format (usually, Robotic Paradigms APIs this is already standardized within the communication platform specifications) Call / Support

Call / Support

Call / Support

Internal Communication API

Control Logic

Hardware Layer

OS

Implementation Layer

Adapter

Implementation Layer

Programming Language

Mapping

External Communication API

Device Driver API

Standard Paradigms –

Platform Semantics Layer Concept Model Robot – provides the concept related to robotics ActorsScientist : Implementation Layer and defines the type of requirements. Robotic System Integrator – Select the communication Actors : System Architect Hardware Layer Robotic – build a robotic system from a strategies Format to match the functional requirements and deploy catalog of available component definitions. Transformation Robotic External Developer Interface Definition – Develop the logic / the robotic application. Actors : Component (optional) Skill Descriptor algorithms. Relies on : Relies on :Maker – Develops Hardware and provides Hardware Static / Dynamic Relies onLanguage : hardware functional descriptors Modeling (like UML or easier adapted) Composer Communication Technology (like Socket,/ more CORBA, Web OS, programming language Services, RMI, possibly (N)DDS…) Relies onStandardization : Need for : External Communication NeedProtocol for Standardization : Hardware Ideally, the complete model API should be standard. Adaptation Mapping

Mapping

Capability Descriptor

Robotic Paradigms APIs

Actors :

Customization Parameterization

Mapping

Mapping External Interface Definition

Mapping

Platform Layer

Middleware Technology

Customization Parameterization

Platform Layer

Modeling

Model Layer Standard Paradigms Semantics Concept Model

Model Layer

Model Layer

Standards at all Levels of Abstraction

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Standardization of Robotics within the Object Management Group

What is the Object Management Group (OMG) •

The Object Management Group (OMG) is – an open membership, not-for-profit consortium that produces and maintains computer industry specifications for interoperable enterprise applications

– an ISO PAS (Publicly Available Specification) submitter, able to submit our specifications directly into ISO's fast-track adoption process.

• •

Any company may join OMG and participate in the standards-setting process. The one-company-one-vote policy ensures that every company, large and small, has an effective voice in our process.

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

OMG Organization & Technical Committees Finance DTF

OMG Space DTF

Government DTF

Architecture Board Business Modeling DTF

Life Science DTF

Platform Technology Committee

Consultation, Command, Control, Communications and Intelligence

Domain Technology Committee

Healthcare DTF

(C4I) DTF

Newly created in December 2005

Software Communication DTF

Robotics DTF

Manufacturing DTF

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

The OMG Robotics DTF : Its Mission •

The purpose of the Robotics Domain Task Force is to foster the integration of robotics systems from modular components through the adoption of OMG standards.

•

To realize this purpose, we will: – Recommend adoption and extend OMG technologies that apply to the specific domain of robotics systems where no current baseline specifications exist, such as MDA for Robotics. The object technology is not solely limited to software but is extended to real objects. This effort promotes the use of OMG technologies in various markets. – Promote mutual understanding between the robotics community and the OMG community. – Endeavor to collaborate with other organizations for standardization, such as the one for home information appliances, and make an open effort to increase interoperability in the field of robotics. – Coordinate with the appropriate OMG subgroups and the Architecture Board, for technology areas that overlap with other OMG Task Forces, to determine where the work will be accomplished.

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Ab Lev st el ra o f ct io n

Working Groups that reflect the Domain Practices

OMG Robotics Domain Task Force

Robotic Platform Working Groups Robotic Infrastructure WG Define standard infrastructure necessary to develop robotic applications

Provides feedback on usability and applicability of definition & Request new necessary features

Provide Platform Abstraction Layer

Tools / Facilities WG Define standard facilities and tools (including configuration, composition, languages) useful for robotics

Level of Functional Abstraction

Provide Tool Abstraction Layer Provides general guidelines for defining models and fundamental concepts definitions

Propose / Request features become available at infra level

Robotic Domain Working Groups Robotic RoboticProfiles ProfilesWG WG

Define standard hardware definitions (including device definition and their associated data structures) on which the Service Layer can rely. Provide Hardware Abstraction Layer

Provides feedback on usability and Robotic RoboticServices ServicesWG WG applicability of definition Define standard abstractions of services & Request new necessary features found in robotic applications (including interfaces to call services, mechanisms and associated data structures) Provides data structures

Provide Algorithm Abstraction Layer

Specialization Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Sorting out the concepts Artificial Communication cation Artificial Communication Agent Control: Loop Loop Position Agent Intelligence Robot Control This is what you mayMapping hear people talk about when yout joinPosition their Working Groups Intelligence Robot g Mapping le e Mobile U Unmanne ed ed Unmanned Mobile Unmanned U Robotic Tools WG Neural Robot Robotic R obo cInfrastructure Infrastructure INeural f t tu WG WG W F Tools/ /Facilities Facilities WG Vehicle Robot Grasping …. Military Vehicle …. Grasping Synchronization y ynchroniza a Military Hospital Synchronization aModel / Encapsulation Component / /Event Management Hospital Behavior Language Network Robot Component Model e / Encapsulation a Event Management na m Real-Time Real Time e en Decision Concurrency Behavior Language or L a anguage Network Robot Real-Time R evior Data / /Data Distribution Flow Robot Concurrency sion sion nstrributioon/ /Command Internal Component Configuration DataFlow Flow Data Distribution Flow D Distributi Command C Robot w Decision Internal Component Configuration m e a Security Application Mission …. n S c it Management M g m / /Activity tyMonitoring Monitorin ng Security Management Activity Monitoring A li ati Deployment D Application Deployment Mission …. Code Mobility / Deployment Safety Physical World Repository Code Mobility Deployment Mob o obi / Safety Physical World Repository y al o Sequencing ySequencing Execution Synchronization / /Prioritization Evaluation Metrics / /Conformance Test n h o Execution Synchronization Prioritization E t i cs C onformanc f a Evaluation Metrics Conformance Test Vision Speech Capability Modeling Vision Speech Modeling Language Capability Modeli g/ /Description Description/ /Advertisement Advertise ver ve ert er e rt rtisement rtiti Modeling Advertisement Humanoid oid d Modeling Language Capability Composition / Hierarchization Data Humanoid Processing ab om cch chi hi Capability Composition / Hi Hierarchization Data esRemote sing ggteMonitoring Processing M Mon ng Remote Monitoring Resource Allocation / /Management Robot Simulation ...... i c ti Communication a em Resource Allocation Management Manipulation f Robot Welfare Simul Simulation ommunica a Communication Deployment D Manipulation Safety Management / Safety Procedure / Safety Policies Welfare a Deployment D p y y Management g ent / Safety Procedure d /S Safety Safety Policies Reptile b t Robot Fault Tolerances / Recovery Strategies Reptile Robot Recover Strategies rat Robotics Faultt Tolerances / Recovery System ...... Robot BipedRobot Robot o a / /Dynamic R onf SystemConfiguration Configuration DynamicReconfiguration Reconfiguration Robot Biped Man/Machine B Semantics Man/Machine Semantics …. Inte Interac ctiic sWG Interaction Robotic Robotic …. Interaction Robotic b ti Profiles botic Profiles P fil WG WG W R Robotic b In ti Services S Services WG G Sensor Sensor Home Common Device Definition Localization / Positioning Composition Home Common Management Co om o mmon Device D efi finiitio ti Definition Localization Localiza c / Positioning o Composition Map Map p Management Composite Device Management Navigation tmp Robot mp Composite Device M Management Naviigation/ /World World dMapping a Navigation Mapping Robot e Kinematics Sensing Devices Image Ma annagem nagem gem men e/ n/Motion t Control Management Path-Planning cs v Kinematics Sensing Devices Image Management M M st Co Ki ti Path-Planning Motion Control/ /Kinematics Kinematics Request Re R e s Pathj t y Actuating h Trajectory Devices Task Digital Request t PlanningAnalog Trajectory PathPath vices s Actuating Devices assPlanning Task Digital Devicess Analog Object Planning Control Rendering Rendering Devices O j Recognition g on/ /Person rs Recognition Object Recognition Person Recognition n Control Planning Localization Communication Devices Object Tracking / Person Localization Co o ion Devic ices Communication Devices Object b Tra cki g / Perso son onTracking Tracking Tracking Tracking Person Visual Security Visual Processing Pro s Entertainment Processing Wheeled Security yy Common DataWheeled Entertainment Structure Audio oSensor t Common Data Structure o s AudioProcessing Processing Robot Robot …. Data Adaptation /rData Sensor Sensor Robot Robot Recognition …. a Mechanism M t Relation nManagement mRecognition Data Adaptation Mechanism / Data Relation Management F s SensorFusion Fusion Fusion Data Fusion Semantics / /Onthology ... Human Interface tth l Data Semantics On Onthology ... n Human Interface Energy Management ... Energy Management ...

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Prioritize the work In order to prioritize standardization work, we took the following survey :

– Given a list of • around 50 of the most common concepts found in robotic systems • and around 20 possible applications of robotic systems

– Each organization was asked for each given item its position related to its : • Participation to the Standardization Process of this item – – – –

I want/will propose Technology I want/will influence Technology I have nothing to say / Que sera sera I think it's impractical

• Utilization of the Standard for each item – – – –

I Think it will help me a lot I will comply with existing standards I don't need such standard This is against my interests Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Survey Results in Brief

• 17 organizations participated – Japan : 8, Korea : 5, US : 2, France : 1, India : 1

• The result of the survey showed that : – Needs for standards are High • Especially for Robotic System Infrastructure

– Intention of Participation seems High • Not only research centers but also businesses

– Needs match proposals rather well • Some unaddressed needs Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Probable Activities Activities are considered probable if : - At least 3 organizations are willing to propose technology - At least 2/3 of organization think it will help their business

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Possible Activities Activities are considered probable if : - At least 2 organizations are willing to propose technology - At least 1/3 of organization think it will help their business

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Improbable Activities Activities are considered improbable if : - less than 2 organizations are willing to propose technology - or less than 1/3 of organization think it will help their business

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Working Groups : Their Roadmap St. Louis Boston Anaheim Was. DC Item

Status Apr-2006

Jun-2006

Sep-2006

Dec-2006

Rev. Sub

Adoption

Final.

San Diego Mar-2007

Robotics Infrastructure Component Model

On going Rev. Sub

Final.

Deployment & Config.

On going Proposed Discussion Discussion Draft RFP Draft RFP

Robotics Profiles Typical devices abstract interfaces and hierarchy Hardware level Resource Profiles

On going Proposed Discussion Draft RFP Draft RFP

RFP

Proposed Proposed Discussion Discussion Draft RFP Draft RFP

Robotics Services Localization Service

On going Discussion Discussion Draft RFP Draft RFP

RFP

User Identification Service Proposed Proposed Discussion Discussion Draft RFP Draft RFP

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

How to participate to the process •

Attend to the Robotics Plenary sessions during the OMG Technical Meetings (approx. once every 2 months) • The best way to keep up-to-date with the latest information concerning the standardization for Robotics is to be part of it. By joining the meeting and giving your opinion, you will not only know where the standardization process is heading to, you will actually influence it to suit the needs of your business.

•

Respond to RFPs • Responding to RFPs is THE way for your organization to make its ideas and technologies become an international standard by proposing your solutions.

•

Propose a theme and lead a sub-working group • If your organization has interests in any given aspect of the Robotic Technology and that would benefit from standardization, leading a sub-working group could give you recognition in your field and provide you with a great competitive advantage.

•

Subscribe and participate to our Mailing Lists [email protected] [email protected] [email protected] [email protected] Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Contacts Robotics Domain Task Force Contact : [email protected] Co-chairs : Hung PHAM (RTI) Tetsuo KOTOKU (AIST) Yun Koo CHUNG (ETRI)

Publicity Drive WG Contact : [email protected] Co-chairs : Abheek BOSE (ADA Software) Masayoshi YOKOMACHI (NEDO) Olivier LEMAIRE (JARA / AIST)

Robotic Infrastructure WG Contact : [email protected] Co-chairs : Noriaki ANDO (AIST) Rick WARREN (RTI) Saehwa Kim (Seoul National Univ.)

Tools / Facilities WG Contact : [email protected] Co-chairs : Abheek BOSE (ADA Software)

Robotic Profiles WG

Robotic Services WG

Contact : [email protected] Co-chairs : Bruce BOYES (Systronix) Seung-Ik LEE (ETRI)

Contact : [email protected] Co-chairs : Olivier LEMAIRE (JARA / AIST) Soo-Young CHI (ETRI)

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

The RT Middleware

Robot Technology Middleware (RTM) : what is it anyway ?!?!

Joint national project between :

AIST JARA MEW

(Panasonic)

Sponsored by :

NEDO Objective Objective Increase Increase the the competitivity competitivity of of the the Robotic Robotic Industry Industry by by facilitating facilitating the development of large scale multi-vendor Robotic the development of large scale multi-vendor Robotic systems systems through software modularization and standardization through software modularization and standardization Intelligent Systems Research Institute – Ubiquitous Functions Research Group

2 Organizations, 2 Approaches… 2 Middlewares ?

• AIST considered mostly the Component Developer case and emphasized : – – – – –

Component Model and Data Exchange Component Internal Execution Integration by weakly typed components through data exchange Control Systems, High Performance, Real-Time Debugging/Visualization tools, Static System Composition Tools

• MEW considered mostly the System Integrator case and emphasized : – – – – –

Component Model and Data Exchange Component Integration Integration by strongly typed components through well defined interfaces Event Driven, Sequential Behaviors, Flexibility, Scalability Dynamic Application Composition Tools, Configuration Tools Intelligent Systems Research Institute – Ubiquitous Functions Research Group

The RT Middleware Concretely RTM Configuration Service

RTM Monitoring Service

CORBA Naming Service

CORBA Event Service

CORBA Trading Service

RTM Script Service RTM Services Access RTM R RT MC Component omponent Interface Standard Interf rface c

e Management API

RTM Base Objects Standard Interfaces

ce Access

(OS A

Specific Hardware Driver orr Technol ogy yS DK Technology SDK

State Management API

RTM R RT MB Base ase e Objects Ob bjects Standard Interfaces Interf rfaces

Security API

Concurrency API Componentt Specific Specific Interf rface Interface

Positioning API

Calls & Supports

Geometry API

Uses Multimedia API

Business Logic

Monito

Component Specific Interface

Loggi Logg

Componentt Specific Specififc Standard r RTM M Interface Interf rface

Data Exchange API

Access

ACE (OS Abstraction Layer)

ce Access

(OS A

Specific Hardware Driver orr Technol ogy yS DK Technology SDK

Specific Hardware Driver or Technology SDK

Componentt Specific Specififc Standard r RTM M Interface Interf rface

RTM R RT MB Base ase e Objects Ob bjects Standard Interfaces Interf rfaces

Componentt Specific Specific Interface Interf rface

RTM R RT MC Component omponent Interface Standard Interf rface c

Componentt Specific Specififc Standard r RTM M Interface Interf rface

RTM R RT MB Base ase e Objects Ob bjects Standard Interfaces Interf rfaces

Component Interface Definition IDL Component Interface Definition IDL (What the Component can do) (What the Component can do) Receives Requests from …

Sends Requests to …

Component Implementation Component Implementation (C++, Java,…)(How it does it) (C++, Java,…)(How it does it) Is Supported by …

Componentt Specific Specific Interface Interf rface

Controls …

Code common to all Components in an RTM System (Implementation included in RTM Framework)

Part of code of the Component automatically generated by an IDL Compiler

Code common to the same family of Components in an RTM System (Implementation included in RTM Framework, See RTM Foundation Interfaces for more details)

Part of code of the Component coming from third parties libraries

Libraries of often used data structures (Implementation included in RTM Framework)

Access from Remote Component

Hardware / Local Resource Access Logic

RTM R RT MC Component omponent Interface Standard Interf rface c Access from Remote Component

Business Logic

mponent Access

Multim

Discovery API

Remote Component Access

Component Specific Standard RTM Interface

Configuration API

Logging API

Access from Remote Component

ccess ess

Monito

Monitoring API

Calls

Access from Remote Component

RTM Component Standard Interface

Part of Code of the Component to be implemented by the component developer

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

The RT Middleware : Concretely A set of standard mechanisms: State Machine Data Exchange Mechanism Uniform Global Positioning Activity Monitoring A set of supporting services: Centralized Configuration Manager Component Repository Event Dispatcher Event Reactor Application Execution

A set of standard interfaces : Common Sensors (IO, Laser, Camera, Mobile Base…) Common Actuators (Motor, Arm, Home Appliances…) Common Rendering Devices A set of tools for : Component Configuration Management Real-time System Monitoring Graphical Application Composition Component Deployment

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

Intelligent Systems Research Institute – Ubiquitous Functions Research Group

New Zealand The University of Auckland

Application specific programming languages and tools for robotics Geoffrey Biggs and Bruce MacDonald Robotics Research Group Department of Electrical and Computer Engineering http://robotics.ece.auckland.ac.nz

10 October 2006

Outline Problem description Literature survey

The University of Auckland

New Zealand

Tools standardisation Conclusions

Robotics Lab

The University of Auckland

New Zealand

10 October 2006

Problem Description How to describe Robot behaviour? Robot programming is not identical to programming in other domains Many tools involved in the programming process Integrated Development Environments Debuggers Programming languages Libraries Software architectures Human Robot Interaction tools ...

The University of Auckland

New Zealand

10 October 2006

Problem Description ... Goal: useful robotic assistants for humans Previously: Some work on CORBA frameworks for Robotics Dynamic software reconfiguration for robot software Contributions to Player 2

Today: programming language and tools

The University of Auckland

New Zealand

10 October 2006

Existing Robot Programming Systems

Manual and automatic programming distinguished by the method used Software architectures provide the underlying support and access to the hardware

The University of Auckland

New Zealand

10 October 2006

Manual Methods Program is created by hand Text or graphical methods of input Finished program loaded onto robot (or simulation) for testing

The University of Auckland

New Zealand

10 October 2006

Controller-Specific Languages e.g. ABB, KUKA Common on industrial robots Proprietary Most advances in design focussed on the programing environment

The University of Auckland

New Zealand

10 October 2006

Generic Languages Generic language extended with robotspecific capabilities e.g. C++, Java, Python

Often used in research environments Robot abstractions common e.g. Player, CARMEN, Orca Object oriented concepts popular

The University of Auckland

New Zealand

10 October 2006

Behaviour-Based Languages Reactive rather than deliberative Subsumption's Behaviour Language Functional Reactive Programming e.g. Yampa, Frob rcFollowLeftWall :: Velocity->Distance>SimbotController rcFollowLeftWall v d _ = proc sbi -> do let r = rfLeft sbi dr Display

< device > Keyboard

< processor > Agent-Server (windows2000)

< device > Sensory Glove

< on board> 㨯wireless LAN 㨯PCI 㨯IP7000

‫ޟ‬TCP/IP‫ޠ‬

CAN

< device > Mouse

< processor > Client-Server (windows2000)

< management > 㨯 Preservation of task and re-execution command 㨯 Communication with Client-Server(TCP/IP) 㨯 Communication with Data-Server(TCP/IP) 㨯 Communication control of CAN bus 㨯 Process of camera image

Agent Robot

< device > Joystick

Operation

< on board> 㨯input device

< management > 㨯 Simulator system 㨯 Input control 㨯 Display the interface to an operator

‫ޟ‬CAN‫ޠ‬ (Controller Area Network)

< device >

< device >

< device >

< device >

H8S for Power management

H8S for Arm control

H8S for Crawler control

H8S for Camera/LP control

Real-time OS

< on board> 㨯 Arm 㨯 H8S

< on board> 㨯 H8S

< management > 㨯Task control 㨯Get Current’s value 㨯Store Current/Voltage logs 㨯Command in another device

Oct. 10, 2006, Beijing, China

Real-time OS

< management > 㨯Arm control 㨯Command in another device

Real-time OS < on board> 㨯 Machine 㨯 PSD 㨯 Ultrasonic Sensor 㨯 H8S

< management > 㨯Machine control 㨯Get Sensor’s value 㨯Command in another device

Real-time OS < on board> 㨯 Camera 㨯 LP 㨯 H8S

< management > 㨯Camera/LP control 㨯Get Angle’s value 㨯Command in another device

International Conference on Intelligent Robots and Systems

16

Physical Agent Robot (PAS) PAR04R

Oct. 10, 2006, Beijing, China

International Conference on Intelligent Robots and Systems

17

Aichi Expo 2005 Robot week, June 9-19

Oct. 10, 2006, Beijing, China

International Conference on Iintelligent Robots and Systems

18

Robot Week (Jun 9-19) Aichi Expo

PAR04R Shibaura Inst. Tech.

Intelligent Wheel Chair, 㵰Komawari-kun㵱 the helper robot UEC

Oct. 10, 2006, Beijing, China

Echono-vehicle Meijo Univ

Apri-Alpha & Apri-Attenda Toshiba & SUT

Collaboration using RT middleware-ORiN Component based on RT Middleware from AIST 19 ApriAlpha using ORCA International Conference on Intelligent Robots and Systems

Layered Implementation of Middlewares „ „

ORiN for Robot Collaboration LwRTC for Embedded controller

Oct. 10, 2006, Beijing, China

International Conference on Intelligent Robots and Systems

20

RT middleware-layers PART1

PART2

application

PART3 controller

㪸㫇㫇㫃㫀㪺㪸㫋㫀㫆㫅 common API

service level Model, data-framework

RTMW1

task level Model, data-framework

mediator RTMW2

Robot profiles

Vision Ctrl.

Common controller I/F

motion level Model, data-framework

Internal NW

Sensor Ctrl.

Dev.Profile

Dev.Profile

controller

Robot A

Robot B

Body Ctrl.

Arm Ctrl.

Dev.Profile

Dev.Profile

motors camera

Oct. 10, 2006, Beijing, China

arm devices

sensors

International Conference on Intelligent Robots and Systems

21

Toshiba robot Voice operation operator

projector visitors Robot profiles XML(CRD) robot control & management system

Robot status management

robot status storage & display system

robot control & status acquisition using RAC

monitor and refer robot status and location

cameras

Oct. 10, 2006, Beijing, China

UEC robots Meijo Univ robots PAR04R International Conference on Intelligent Robots and Systems

22

System integration using ORiN control data robot control & management system

operator

GUI Robot control application operation VB.NET

command table

CAO Engine

CRD provider

RAC provider

Robot profiles XML(CRD)

operator Oct. 10, 2006, Beijing, China

RAC requests

Voice operation International Conference on Intelligent Robots and Systems Robots

23

ORiN: Open Resource/Robot Interface for the Network Application ORiN Provider

Robot Controller Oct. 10, 2006, Beijing, China

International Conference on Intelligent Robots and Systems

24

ORiN structure Application

Engine

App. X

App. Y

Abstract Device

Dev. B

App. Z

㪦㪧㪚

㪬㪧㫅㪧

㪛㪛㪜

Non-CAO Application (ex. OPC Client, UPnP Control Point)

Application IF

CAO Engine Device IF

Provider

A Co.

B Co.

Proprietary Protocols

CAP

CRD

(ex. OPC Server)

SOAP

Internet

Device

CRD File (XML)

Oct. 10, 2006, Beijing, China

Other Standards

Non-ORiN Device

International Conference on Intelligent Robots and Systems

25

Key Technologies of ORiN 䋨Open Robot Interface Model䋩 ¾ &RQWUROOHU$FFHVV2EMHFW&$2 9

A middleware providing a standard program interface for the applications and the devices.

9

It’s based on the distributed object technology, DCOM or CORBA.

¾ &RQWUROOHU$FFHVV3URWRFRO&$3 9

A standard communication protocol for the Internet to allow data-exchange over firewall.

9

It’s based on the SOAP technology.

¾ &RQWUROOHU5HVRXUFH'HILQLWLRQODQJXDJH&5' 9

A standard data schema for representing the data in a device.

9

It’s based on the eXtensible Markup Language, XML-Schema.

Oct. 10, 2006, Beijing, China

International Conference on Intelligent Robots and Systems

26

RAPI The Robot Information Framework and Application Program Interface ISO TC184/SC2 New Work Item

Oct. 10, 2006, Beijing, China

International Conference on Intelligent Robots and Systems

27

Objective of RAPI to provide standard methods for accessing robot/device system information when developing a multi-vendor Mfg. system •a unified access method •a common application platform •a common API •a unified robot/device profile description •networking capability using distributed object Oct. 10, 2006, Beijing, China

International Conference on Intelligent Robots and Systems

28

Relationship between ORiN and RAPI ORiN ORiN::PC PCtechnology technologyof ofthe thestates statesof ofart art •Implementation •Implementationdepending depending on on Microsoft Microsoft Technology Technology

but but

••Available Availableand andReady-to-apply Ready-to-applytechnology technology RAPI: TheRobot RobotInformation InformationFramework Frameworkand andApplication ApplicationProgram ProgramInterface Interface RAPI:The Independent Independentfrom fromsoftware softwareproducts productsfrom fromthe thespecific specific vendors vendors •• Abstraction Abstractionof ofORiN ORiN •• Subset Subsetof ofORiN ORiNin infunctionality functionality Limit Limitto tofunctions functionsthe theleast leastrequirement requirement Exclude RAP 29 Exclude RAP Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems

Merits of RAPI End Users &RVWHIIHFWLYHRSHUDWLRQ ಴ 0XOWLYHQGRU0IJV\VWHPWRPHHW XVHUQHHGV ಴ 0IJ0RQLWRULQJDQG5HPRWH 0DLQWHQDQFHLVHDVLHU

Software Vendors &RPPRQ3DFNDJHVIRU,QGXVWULHV ಴ &RVWUHGXFWLRQE\VWDQGDUGL]LQJ VRIWZDUHSDFNDJH ಴ 1HZURERWVRIWZDUHPDUNHWLV JHQHUDWHG

5$3,6WDQGDUGV &RPPRQDSSOLFDWLRQSODWIRUPDQG$3, Independent from the Applications and the Robots System Integrators (DV\FXVWRPL]LQJRI)$ VHUYLFHV ಴ 5HGXFHGHYHORSPHQWFRVWE\ XVLQJDFRPPRQSODWIRUP ಴ ,QGHSHQGHQWIURPWKHURERW PDNHUV Oct. 10, 2006, Beijing, China

Robot Makers ([SDQGWKH5RERW$SSOLFDWLRQ)LHOG ಴ 5HGXFHGHYHORSPHQWFRVWIRU 5RERWFRPPXQLFDWLRQ,QWHUIDFH ಴ 1HZURERWPDUNHWIRUYDULRXV W\SHVRIDSSOLFDWLRQLVJHQHUDWHG

International Conference on Intelligent Robots and Systems

30

ISO TC184/SC2 Schedule   UG 4 WK 4 VW 4 QG 4 UG 4 &UHDWLQJWKHVW GUDIW

WK 4

VW 4

 QG 4 UG 4 8SGDWLQJ

5$3,VW UHYLVLRQ

,62PHHWLQJLQ3DULV

 VW 4 QG 4 )LQDOL]LQJ

WK 4

&RPPLWWKH1:,3

5$3,GUDIWE\WKH25L1IRUXP 6WDUWWKHGLVFXVVLRQE\WKH37

Objective: To create a standard framework for Robot communications. Term of Activity-XQHWR-XQH