Special issue on networked intelligent robots through ... - IEEE Xplore

Scanning the Issue Special Issue on Networked Intelligent Robots Through the Internet The use of robots has dramatically expanded the potential of e-services. Intelligent robotic systems have been extensively applied in many different ways, and also in our daily life. Various remote control methods have been utilized for intelligent robotic systems. Consequently, it is very convenient to use the Internet to control intelligent robots. In the past few years, many researchers have been using the Internet as a command transmission medium, which can control the intelligent robot and obtain feedback signals. Although the Internet has many advantages in a variety of fields, using the Internet to control intelligent robots also has some limitations, such as uncertain time delay, uncertain data loss, and data transmission security problems. The issues for controlling intelligent robots through the Internet are broadly discussed in terms of direct control, behavior programming control, supervisory control, and learning control. Future research trends and applications are also important issues for the Internet robot. The first paper, “Networked Intelligent Robots Through the Internet: Issues and Opportunities,” by Luo and Su, describes the methods and analyzes the effects on the remote control systems of such problems as uncertain time delay, uncertain data loss, and data transmission security problems. There are some control architectures that combine computer and robot. The issues for controlling intelligent robots through the Internet are discussed in terms of direct control, behavior programming control, supervisory control, and learning control. Next, the paper points out some applications for networked intelligent robots. Some previous results have been used in real life. In the future, the Internet will become an important part of our life with the growth of computation power. It is expected that the networked intelligent robot will be developed quickly to improve the welfare of society. The next two papers, by Tan and Clapworthy, explore “Virtual Environments for Internet-Based Robots: (I) Modeling a Dynamic Environment and (II) Path Planning.” “Modeling a Dynamic Environment” addresses the issues of parameter acquisition for single-image-based modeling of virtual environments (VEs). By studying the properties of basic three-dimensional (3-D) features such as point sets and edge corners, Manuscript received December 26, 2002; revised December 29, 2002. Digital Object Identifier 10.1109/JPROC.2003.809197

a parameter-searching method was developed that employs virtual objects as feature-matching templates. The approach is particularly valuable if the environment is subject to unpredictable change. Instead of using the templates to pick out the basic image features that will enable the 3-D models to be constructed, they use the projection of virtual objects as masks and scan the parameter space of the virtual object to find the best match between the features of the image and those of the virtual object. They also investigate some standard problems associated with particular situations, including “the three-point problem,” “the three-edge-corner problem,” and “parameter acquisition for cuboids.” Finally, they have developed a method that enables VEs to be integrated into Internet-based teleoperation systems, based on information acquired from only a single image of the physical environment. The second of these two papers covers “Path Planning.” It addresses the issue of path planning in a VE for the control of an Internet-based robot manipulator. A hybrid method is proposed in which the dimension of the path-planning problem is reduced by separating the problem of planning the position of gripper from that of planning the pose of the gripper. A global configuration-space technique is used for the former and a local operational-space technique is used. This topic includes four parts: “Path Planning in Virtual Environments,” “Mapping of Graphical Models,” “Pose Planning,” and “Path Modification.” They have verified the feasibility of applying traditional robot-control techniques, such as collision detection, obstacle avoidance, and path planning to the network-based robot, which enhances the possibilities for implementing network-based robots capable of competing with traditional teleoperated systems. The fourth paper, “Supermedia-Enhanced Internet-Based Telerobotics,” by Elhajj et al., discusses the development of supermedia enhanced internet-based telerobotics. This paper introduces new planning and control methods for real-time telerobotic operations via the Internet. Supermedia is the collection of video, audio, temperature, and other sensory feedback. However, when the communication medium used introduces random communication time delay, difficulties arise. Due to the complexity and diversity of such systems, the first challenge is to develop a general and efficient modeling and analysis tool.

0018-9219/03$17.00 © 2003 IEEE

PROCEEDINGS OF THE IEEE, VOL. 91, NO. 3, MARCH 2003

367

This paper also proposes the use of Petri Net modeling to capture the concurrency and complexity of Internet-based teleoperation. The modeling not only combines with the event-based planning and control method, but also provides an efficient analysis and design tool to study the stability, transparency, and synchronization of such systems. In addition, the concepts of event transparency and event synchronization are introduced and analyzed. The modeling and control method has been applied to the design of several supermedia enhanced Internet-based telerobotic systems, including the bilateral control of mobile robots and mobile manipulators. These systems have been experimentally implemented in a three-site test bed consisting of robotic laboratories in the USA, Hong Kong, and Japan. The experimental results have verified the theoretical development and further demonstrated the stability, event transparency, and event synchronization of the systems. It was shown theoretically and experimentally that this approach results in a stable teleoperation system regardless of the time delay encountered. The fifth paper, “Control of Robot Arm with Virtual Environment via the Internet,” by Safaric et al., introduces a new platform-independent Web-based virtual laboratory for robotics engineering students and other types of industrial trainees, which provides users with on-line access to real-world hardware for remote experimentation. The approach requires the user to develop tasks off-line, using their remote computing resources, before submitting the experiment to the virtual laboratory server for execution on the actual device. It is a new type of virtual lab server, called VLAB. The virtual laboratory approach is based on the concept that it provides a working facility for off-line programming of actual working robot lines in an industrial environment and hands-on training while reducing the need for high-cost physical devices in educational institutions. To avoid robot arm and real-world collision, this platform also includes the collision detection software between a virtual robot arm and a VE in the virtual model. The collision detection software is used as a uniform space subdivision method to accelerate the collision detection. The user invokes the experiment with a simple start in the browser with the VLAB server’s Web address, which runs on the user side. Then, the collision detection code and Virtual Reality Modeling Language (VRML) code are downloaded to the user computer, and the user sees the teach pendant in his VRML browser and is able to communicate with an “on-line execution part” of the VLAB server. This reported approach offers significant education and training experience while minimizing the use of expensive or time constrained physical resources. The sixth paper, “Collaborative Teleoperation Using Networked Spatial Dynamic Voting,” by Goldberg et al., describes a networked teleoperation system that allows participants to collaborate using a “spatial dynamic voting” (SDV) interface. They present an Internet-based multiple-operator single-robot system that averages multiple vector inputs to control the position of an industrial robot arm. They analyze system performance with a uniform 368

ensemble of well-behaved deterministic sources and then model malfunctioning sources that go silent or generate inverted control signals. They found that performance is surprisingly robust even when a sizable fraction of sources malfunction. The SDV interface is displayed on the browser of all active voters. Using the SDV interface, voters participate in a series of 30–60 s voting images. Each voting image is a single image with a textual question. When the voting cycle is completed, SDV analysis algorithms analyze the voting pattern to determine a consensus command that is sent to the tele-actor. The tele-actor is a skilled human with cameras and microphones who navigates and performs actions in the remote environment. As an alternative to semantic analysis of the image, they consider votes as spatial distributions and identify preferred “consensus” regions in the image. They then use these regions to define two metrics for individual and group performance in terms of leadership and collaboration. The SDV interface differs from multiple-choice polling because it allows spatially and temporally continuous inputs. The seventh paper, “REAL—An Internet Accessible Mobile Robot Laboratory,” by Guimarães et al., presents the Remotely Accessible Laboratory (REAL), a virtual laboratory accessible through the Internet. The architecture of REAL departs from the commonplace Web application, since it employs a sophisticated software architecture based on software components. To build a reliable architecture for REAL, a component model suitable for telematic services is designed and implemented: CCM-tel. CCM-tel is neutral, based on Common Object Request Broker Architecture (CORBA), and shares many common features with CORBA Component Model (CCM), such as the ability of components to send asynchronous notification messages through a reliable notification channel (both unicast and multicast notification schemes are supported). In this paper, CCM-tel components were developed for REAL and grouped into three categories: access, service, and communication. An access session allows the user to subscribe/unsubscribe to the services, to authenticate subscribed users, and to start a subscribed service. A service session is established when the user starts a service, and it allows the user to perform the permitted actions (e.g., to manipulate the robot). The communication session is responsible for establishing media streaming connections to the service. New Internet applications such as virtual laboratories and Internet robots are closer to telecommunication services than to commonplace Web applications. Virtual laboratories and Internet robot services demand a high volume of software that must operate efficiently and reliably. The eighth paper, “Force-Reflecting Teleoperation Over the Internet: The JBIT Project,” by Oboe, reports the application of real-time closed-loop control over Internet in the Java-Based Interface for Telerobotics (JBIT) system. Here, the Internet users can access and command a two-degrees-offreedom robot in real time, and receive both visual and force feedback. One can use the Internet for the remote closed-loop control of a slave robot, the controlled system with on-line identification of the connection characteristics, adaptation PROCEEDINGS OF THE IEEE, VOL. 91, NO. 3, MARCH 2003

of the data rate to the available throughput, recovery of the missing packet, handling of the variable delay is operability and safety, and developing the new tools for adaptive data flow management used to prevent congestion. The system has an improved coordinating force control scheme, aimed at enhancing the transparency of the teleoperator. Telerobotics equipment can benefit from its decentralized structure, in which a controller on the remote equipment is usually already present, thus facilitating the implementation of local autonomous control, needed when the connection with the master controller is lost. This paper demonstrates that it is possible to design force-feedback telerobotics equipment in which Internet is used as communication channel, even if the present network capabilities severely limit the achievable performance. Improvement of the system is expected with the new network technologies—the throughput, losses, and delays will be available. The new technologies will ease the design of reliable systems for closed-loop control of remote robots. The ninth and final paper, “Internet-Based Solutions in the Development and Operation of an Unmanned Robotic Airship,” by Ramos et al., describes the conceptual and implementation aspects of an Internet-based software architecture for the Autonomous Unmanned Remote Monitoring Robotic Airship (AURORA) project, showing its use in the different project phases. Some problems associated with Internet connections in aerial robots are also discussed, such as quality of service and security issues. These problems limit the use of the Internet in aerial robots. The main objective of Project AURORA is to establish the initial technological basis for semiautonomous airship navigation and to validate them through low-demanding monitoring applications. The major physical components of AURORA are the airship, a mobile

ground station, and a communication system. The integrated software environment covers the on-board ground station and communication infrastructure. The AURORA phases include the airship simulator; computer-aided design for control and navigation, sensory and flight data visualization, mission programming, and data visualization; a 3-D virtual viewer of the airship and the surroundings; setup control system parameters on flight; flight data storage; and flight and data playback. The Project AURORA software architecture is built around a Transmission Control Protocol/Internet Protocol environment, allowing an easy integration of the software tools. The generic tools available in AURORA include 3-D viewers, plotting utilities, avionics panel, mission visualization and programming tools, simulators, and Web clients. The AURORA Internet interface to the operation site corresponds to the software and hardware components associated with the reception of flight data and their distribution. In Project AURORA, some of these solutions have been extensively tested in practice whereas others are under development, in an evolving procedure of creation of Internet solutions for aerial robotics. REN C. LUO, Guest Editor National Chung Cheng University College of Engineering Chiayi, 62107 Taiwan, R.O.C. TOSHIO FUKUDA, Guest Editor Center for Cooperative Research in Advanced Science and Technology Nagoya, 464–8603 Japan

Ren C. Luo (Fellow, IEEE) received the Ph.D degree from Technische Universität Berlin, Berlin, Germany, in 1982. He was an Assistant, an Associate, and a Full Professor in the Department of Electrical and Computer Engineering and Director of the Center for Robotics and Intelligent Machines, North Carolina State University, Raleigh, and was the Toshiba Chair Professor in the Institute of Industrial Science, University of Tokyo, Tokyo, Japan. He is currently a Professor in the Department of Electrical Engineering and the President of National Chung Cheng University, Chia-yi, Taiwan, R.O.C. He has published more than 200 papers in internationally renowned journals and conference proceedings. His research interests are concerned with sensor-based intelligent robotics systems multisensor fusion and integration, micro- and nanotechnologies, computer vision, rapid prototyping, and advanced manufacturing systems. Prof. Luo is President of the Chinese Institute of Automation Engineers. In 1996, he received the Alcoa Foundation Outstanding Engineering Research Award at North Carolina State University. In 1998–1999, 2000–2001, and 2002–2004, he also received National Science Council Outstanding Research Awards. He was President of the IEEE Industrial Electronics Society, and is Editor-in-Chief of IEEE/ASME TRANSACTIONS ON MECHATRONICS. PROCEEDINGS OF THE IEEE, VOL. 91, NO. 3, MARCH 2003

369

Toshio Fukuda (Fellow, IEEE) received the M.S. and D.Eng. degrees from the University of Tokyo, Tokyo, Japan, in 1973 and 1977, respectively. He also studied at the graduate school of Yale University, New Haven, CT, from 1973 to 1975. In 1977, he was with the National Mechanical Engineering Laboratory, Tsukuba, Japan. From 1979 to 1980, he was a Visiting Research Fellow at the University of Stuttgart, Stuttgart, Germany. From 1982 to 1989, he was with the Science University of Tokyo, Tokyo, Japan. He joined Nagoya University, Nagoya, Japan, in 1989, where he is currently a Professor with the Center for Cooperative Research in Advanced Science and Technology, the Department of Micro System Engineering, and the Department of Mechano-Informatics and Systems. He has written six books, has edited five books, and has published more than 1000 technical papers in the areas of microsystems, robotics, mechatronics, and automation. His current research interests include intelligent robotic systems, cellular robotic systems, mechatronics, and microrobotics. Dr. Fukuda is a Fellow of the Society of Instrument and Control Engineer and Vice President of the International Fuzzy Systems Association. He has been awarded the IEEE Eugene Mittlemann Award, the Banki Donat Medal from Polytechnic University, Budapest, Hungary, the Medal of the City of Sartillo, Mexico, and the IEEE Millennium Medal. He has been the Vice President of the IEEE Industrial Electronics Society, and is Secretary of the IEEE Neural Network Council, President of the IEEE Robotics and Automation Society, Editor-in-Chief of IEEE/ASME TRANSACTIONS ON MECHATRONICS, and Director of IEEE Division X.

370

PROCEEDINGS OF THE IEEE, VOL. 91, NO. 3, MARCH 2003