An Exclusive Course for Computer Scientists and Engineers - CiteSeerX

© CORBIS & JOHN FOXX

An Exclusive Course for Computer Scientists and Engineers

T

he field of human–robot interaction (HRI) HRI theory to practice. However, as would be expected with an addresses the design, understanding, and evaluation emerging field, HRI courses are largely ad hoc. of robotic systems, which involve humans and Teaching HRI is challenging because the subject is multidisrobots interacting through communication [1]. As ciplinary, and there is lack of educational materials, such as textthe field matures, education of stubooks and resources such as robots and interfaces. dents becomes increasingly important. This article summarizes the discussion and Courses in HRI provide the canonifindings from the “Teaching Humans BY ROBIN R. MURPHY, cal set of knowledge and core skills About Human–Robot Interaction” TATSUYA NOMURA, that represent the current state of the workshop on the development of an field and permit the evolution of HRI course for computer scientists and AUDE BILLARD, knowledge and methods to be transengineers. This half-day workshop was AND JENNIFER L. BURKE ferred from research to a broad set of held at the IEEE/Robotics Society of students. In addition, coursework in HRI Japan International Conference on Intellicreates a workforce capable of transferring gent Robots and Systems (IROS), 22 September 2008, in Nice, France. The motivation for the workshop was a Digital Object Identifier 10.1109/MRA.2010.936953 direct response to a key finding from the National Science

JUNE 2010

1070-9932/10/$26.00ª2010 IEEE

IEEE Robotics & Automation Magazine

85

The IEEE Robotics and Automation Society sponsors a Technical Committee on Human– Robot Interaction. Foundation (NSF)-sponsored HRI Young Pioneers Workshops [2], held in conjunction with the annual Association of Computing Machinery (ACM)/IEEE Conference on Human–Robot Interaction. The findings consistently emphasized the need for an interdisciplinary course or curriculum in HRI to be taught at the university level. However, until the IROS workshop, there has been no reported venue for faculty to gather and discuss such a curriculum or teaching methods. Although this workshop was limited in both time and the number of participants, it offers a starting point and some insight into HRI education. The objectives of the workshop were to identify what is essential in an HRI course by leveraging the experiences to date in teaching HRI and then use this list of fundamentals to define course content. The workshop was also expected to create a community of educators within the emerging HRI research community, foster the exchange of best practices and pedagogical methods, and provide reference materials, if any, for instructors teaching HRI. The rest of this article is organized as follows. It first describes the workshop in terms of participants and activities. The challenges for a course in HRI identified by the participants follow next. Suggested course content, both in terms of the set of candidate topics for a course, and a sequence of lectures for advanced students in artificial intelligence (AI) and robotics follows. Next, possible course projects and assignments are discussed. The article then concludes with a distillation of the workshop into a set of six major findings.

Workshop Description The workshop was attended by 18 participants, representing France, Germany, India, Japan, Korea, Switzerland, and the

United States, and was organized around group discussions. Graduate students slightly exceeded the number of professors and industry researchers. The workshop consisted of four parts; beginning with each participant positing what they believed should be included in a HRI course and what is currently missing from HRI education. Second, a discussion on available resources for HRI education was initiated with an invited talk by Dr. Kojiro Matsushita from the University of Tokyo and demonstration of his two low-cost robot kits [3]. One kit was made from servomotors and plastic water bottles and can be constructed by beginners in less than 6 h after 2 h of instruction, making it suitable for nonrobotics students. The resulting robot can take many configurations, including legs and snake structures, and more emotive shapes similar to puppets. The robot can be controlled directly, learn motions from the user guiding the robot, or controlled by noninvasive contact sensors measuring muscle strain. Another kit is based on a toy hand. Dr. Matsushita is working on an English translation of his book on how to build and use these robots. The next discussion, led by Dr. Aude Billard from the Ecole Polytechnique Federale de Lausanne (EPFL), concentrated on lessons learned from both instructor and student experiences with HRI. Three professors described HRI classes at the Indian Institute of Information Technology Allahabad (Prof. G.C. Nandi), EPFL (Prof. Aude Billard), and at the University of South and Texas A&M (Prof. Robin Murphy). Rod Gutierrez, a graduate student at the University of South Florida, presented feedback from the 2008 Young Pioneers Workshop at the 2008 ACM/IEEE International Conference on Human–Robot Interaction with amplifying comments from that workshops attendees. The fourth discussion took the form of breakout groups. Participants were split into two groups: one to discuss the perfect syllabus, or sequence of lectures, for an HRI course, and the other to determine the perfect set of assignments and projects (Figure 1). The groups then gave reports summarizing their thoughts. The fifth component was a short recap and discussion of future activities, primarily increased involvement in the annual HRI conference.

Challenges

Figure 1. Participants in one of the two breakout groups. (Photo courtesy of Robin Murphy.) 86

IEEE Robotics & Automation Magazine

Creating a new course is always challenging, but the field of HRI provides three additional challenges for education. First, HRI is multidisciplinary, incorporating contributions from communications, computer science, engineering, psychology, and theater, creating challenges in creating course content that covers the field in sufficient depth without requiring a large number of prerequisites. Balancing coverage depth while minimizing prerequisites is particularly hard because the background between the engineering sciences and the human sciences was felt to be large. In response, the workshop participants quickly restricted discussion to teaching students in the engineering sciences; even within that restriction, the differences between individual engineering disciplines and computer science were significant. JUNE 2010

Second, the diversity of the HRI field also extends to resources, and as a result, there are no dedicated HRI resources, although possible materials can be extracted from mature fields. For example, HRI does not have a journal or textbook. There is a dedicated conference, the annual ACM/ IEEE International Conference on HRI now in its fourth year, but the majority of participants were not aware of the conference. A related conference, the IEEE International Symposium on Robot and Human Interactive Communication (Ro-Man), also publishes HRI research. The IEEE Robotics and Automation Society sponsors a technical committee on HRI. Third, there is a lack of cost-effective, pedagogically appropriate robots and rich interfaces. As detailed below, hands-on projects in HRI are highly desirable. Robots such as Lego Mindstorms are inexpensive and do not require extensive programming expertise but, as noted by the students, may not provide sufficient capability to support key HRI topics. Humanoid robots vary in price but often have significant limitations for general HRI topics. For example, the design of the HOAP-3 robot prevents the camera in the head from seeing the hands, curtailing physical interaction and learning manipulation tasks. Few robots in any price range support human–computer interfaces such as haptics, touch screens, or gestures. Speech recognition remains unreliable, obviating the easy application of natural language to a survey course. One promising robot resource that addresses the third challenge is the robot kits presented by Dr. Matsushita and shown in Figure 2.

Course Content Starting from the discussion and throughout the workshop, topics for inclusion in course emerged. The objectives of an HRI course were proposed, and a subset of these topics was arranged into one possible sequence of lectures aimed at advanced robotics or AI students. The individual topics were not rated as to relative importance because of time constraints. The topics not only largely borrowed from robotics, AI, and psychology themes but also included more unique HRI subjects and applications. Robot control and humanoid robot design and control were two robotcentric topics suggested for inclusion, along with user interfaces. Skill acquisition, often associated with traditional robot learning, has been experiencing a renaissance with the new emphasis offered by HRI. In particular, it was noted that students often do not understand limits on the range of motion or degrees of freedom in humanoid robots and thus become confused when trying to generate naturalistic motions. Natural JUNE 2010

(a)

Robot control and humanoid robot design and control were two robotcentric topics suggested for inclusion. language processing and machine learning, staples of AI, were also deemed important. Psychology and cognitive engineering topics were tools and methods to measure HRI, joint attention theory, teams, and user-centered design. The participants noted that there was no concise list of qualitative and quantitative evaluation methods or tools, nor was there a clear mapping of particular techniques to desired outcomes, e.g., what technique would be best to measure X? Social behaviors, emotion or affective expressions, interaction modalities, social learning, user expectations, safety, the Uncanny Valley, and ethics emerged as unique HRI topics. The topic of social behaviors actually is composed of two topics: one is “what are social behaviors?” and the other is “how can robots be programmed to generate social behaviors?” Rehabilitation and therapy was singled out as major HRI application areas. Course objectives should include at a minimum: u definition of HRI u the basic modalities for interacting with a robot u the key issues in HRI u the current applications u the process of making robots into social platforms u the importance of social skills in robots (role of learning, a theory of mind).

(b)

Figure 2. Demonstration of walking robot and robot hand kits by Dr. Matsushita. (a) Legged robot from water bottles and (b) robot hand. (Photo courtesy of Rodrigo Gutierrez.)

IEEE Robotics & Automation Magazine

87

Psychology and cognitive engineering topics were tools and methods to measure HRI, joint attention theory, teams, and usercentered design. The prerequisites for an HRI course depend on the target audience and scope of material, although probability and statistics was considered a universal prerequisite. In addition to probability and statistics, related concepts such as regression analysis and experimental design would be helpful for a course focused on methodology. Robotics and AI (capturing control and automation), sensors, and machine vision are starting points for robotics students to study HRI. In addition, having signal processing and machine learning might be very helpful, although participants noted that machine learning was a topic that should also be covered in the course. Assuming an advanced robotics student with a background in AI, a set of possible lectures spans robot inputs to ethics. These are: u modalities and types of knowledge acquired through interactions, including vision, speech, and haptics u representing the world and the intentions of others u case studies of social learning and interaction u evaluation methodologies, both qualitative and quantitative u ethics.

Course Projects and Assignments Having a hands-on component to an HRI class was strongly recommended by those who have taught HRI, who wish to teach HRI, and students. The recommended pattern was to have a series of small assignments either directly related to the current course material or scaffolded in complexity, then a final project chosen by the students. Assignments and projects directly involving robots and users were seen as the most desirable. However, working with users and robots raises many issues. Availability of platforms and of users is a concern. User-studies often requires a great deal of planning and preparation, including getting any institutional human–subject protocol approvals. Working with robots is costly, and there are concerns that sufficient robots will not be operational when needed. Robot simulations may prove to be a viable alternative to directly using a robot. Simulations such as Microsoft Robotics Studio can be programmed at a high level of abstraction, allowing the students to move and direct the robot without having to focus on the details of the robot or robot programming. Regardless of whether real or simulated robots are used, two applications are particularly attractive for a course. Search and rescue robotics has a strong societal benefit, whereas social robots are engaging and entertaining.

88

IEEE Robotics & Automation Magazine

Summary of Findings The workshop focused on teaching roboticists (computer science and engineers) at graduate level, generally discussing issues from an instructor’s viewpoint (e.g., pedagogy and resources) with a presentation and feedback from students. The six findings from the workshop are summarized below. u Finding 1: Students prefer HRI courses with a high degree of interaction between students and between students and robots over courses that are primarily lecture based. Interaction, both through discussion and hands-on projects, appears to be the desired style for teaching HRI. u Finding 2: Candidate topics for coverage in an HRI course include emotion, ethics, humanoid robot design and control, interaction modalities, joint attention theory, machine learning, natural language processing, robot control, safety, skill acquisition, social behaviors, social learning, teams, tools and methods to measure HRI, the Uncanny Valley, user interfaces, user-centered design, and user expectations. The choice of topics to include depends on the course prerequisites. On one hand, course prerequisites permit content to go deeper or free up time in the course schedule to include more of these topics. On the other hand, prerequisites may exclude students from the human sciences or even from a particular engineering science discipline. This could undermine the benefits of interdisciplinary courses and the discussionoriented teaching style desired by the students. u Finding 3: The most prominent deficits for creating course content in HRI are the lack of: 1) a set of key principles of HRI, 2) a survey of mechanisms on how to generate social behaviors, and 3) a succinct synopsis of user evaluation methods. We note that the fist deficit in the list reflects the lack of consensus in the HRI community over HRI. However, the second and third deficits highlight gaps in robotics that must be filled by multidisciplinary work; the second deficit shows the need to connect control theory with the behavioral sciences, whereas the third deficit necessitates a transfer of quantitative and qualitative methods pioneered outside of robotics. u Finding 4: The major missing pedagogical tools for instructors are cost-effective robots and a corpus of case studies, illustrating key principles of HRI. Cost is viewed as a major driver of a robot that can be adopted by a large number of universities for teaching HRI. u Finding 5: Course development should consider industry needs as well as instructor constraints and student learning preferences, as not all students will become HRI researchers. This includes understanding anthropomorphic robots as well as nonanthropomorphic forms. u Finding 6: Regardless of the target audience, an HRI course will most likely require students to have a background in statistics and will, at a minimum, cover interaction modalities, issues, social interactions, and applications.

JUNE 2010

The workshop briefly touched on the way ahead. In terms of facilitating general progress in HRI education, there was a hope that the HRI conference would become a clearing house for HRI-specific resources. In terms of continuing the discussion on HRI education, it would be interesting to elicit the viewpoints of other disciplines, especially psychology, on what they believe are fundamental topics and how HRI should be taught.

Acknowledgments The authors thank Dr. Matsushita for his demonstration of low-cost robots, Rod Gutierrez for his presentation and general assistance during the workshop, Dr. Ephriam Glinert for his support of the HRI Young Pioneers Workshop (NSF Grant IS-0813909), and the IROS 2008 tutorial chairs, Dr. Rachid Alami and Dr. Roland Siegwart.

Keywords

The prerequisites for an HRI course depend on the target audience and scope of material, although probability and statistics was considered a universal prerequisite. Technology and Research Intelligent Robotics and Communication Laboratories, Japan. He is a member of the Japanese Psychological Association, the Japanese Cognitive Science Society, and the Mathematical Society of Japan. He is a Member of the IEEE. His research interests include intelligent robots and human—robot interaction.

Human–robot interaction, robotics education.

References [1] M. A. Goodrich and A. C. Schultz, “Human-robot interaction: A survey,” Found. Trends Hum.-Comput. Interact., vol. 1, no. 3, pp. 203–275, 2007. [2] J. Burke, R. Murphy, and C. Kidd, “Young researchers in HRI workshop 2006,” Interact. Stud., vol. 8, no. 2, pp. 343–358, 2007. [3] K. Matsushita, H. Yokoi, and T. Arai, “Plastic-bottle-based robots in educational robotics courses—Understanding embodied artificial intelligence,” J. Robot. Mechatron., vol. 19, no. 2, pp. 212–222, 2007.

Robin R. Murphy received a B.M.E. degree in mechanical engineering, and M.S. and Ph.D. degrees in computer science in 1980, 1989, and 1992, respectively, from Georgia Tech, where she was a Rockwell International Doctoral Fellow. She is the Raytheon Professor of Computer Science and Engineering at Texas A&M. In 2008, she was awarded the Al Aube Outstanding Contributor Award by the Association for Unmanned Vehicle Systems International Foundation for her insertion of ground, air, and sea robots for urban search and rescue at the 9/11 World Trade Center disaster, Hurricanes Katrina and Charley, and the Crandall Canyon Utah mine collapse. She is a distinguished speaker for the IEEE Robotics and Automation Society and has served on numerous boards, including the Defense Science Board, U.S. Air Force Scientific Advisory Board, NSF Computer and Information Science and Engineering Advisory Council, and the Defense Advanced Research Projects Agency Information Science and Technology Study Group. She is a Senior Member of the IEEE. Her research interests include AI, HRI, and heterogeneous teams of robots. Tatsuya Nomura received the M.S. degree in mathematics from Osaka University, Japan, in 1989, and the D.E. degree in engineering from Kyoto University, Japan, in 1998. From 1989 to 2000, he was with the Corporate Research and Development Group at Sharp Corporation. From 2000 to 2004, he was with Hannan University, Osaka, Japan. He is currently an associate professor in the Department of Media Informatics, Ryukoku University, Otsu, Japan, and a researcher in the Advanced

JUNE 2010

Aude Billard received a B.Sc. degree in physics from EPFL, with specialization in particle physics at the European Center for Nuclear Research (CERN) in 1994. She received her M.Sc. degrees in physics from the same university, with specialization in particle physics at the CERN and in knowledge-based systems in 1996 and a Ph.D. degree in AI from the Department of Artificial Intelligence at the University of Edinburgh in 1998. She is an associate professor and head of the Learning Algorithms and Systems Laboratory at the School of Engineering, EPFL. Before this, she was a research assistant professor at the Department of Computer Sciences at the University of Southern California, where she retained an adjunct faculty position to this day. She is a Member of the IEEE. Her research interests focus on machine learning tools to support robot learning through human guidance. This extends also to research on complementary topics, including machine vision and its use in human–machine interaction and computational neuroscience to develop models of learning in humans. Jennifer L. Burke received the B.A. degree in business from Florida State University, the M.S. degree in counseling from the University of North Florida, and the M.S. and Ph.D. degrees in industrial-organizational psychology (minor: man– machine interaction) from the University of South Florida, in 1980, 1990, and 2006, respectively. She is a practicing human factors engineer at SA Technologies, specializing in robotic interface design. She is active in the robotics and psychology/ human factors communities and is the author of more than 30 publications in fields of robotics, human performance, and workplace studies. She is a member of the ACM, the American Psychological Society, and the Human Factors and Ergonomics Society. Her research interests include team processes and human–robot interaction. Address for Correspondence: Robin R. Murphy, Computer Science and Engineering, Texas A&M University, College Station, TX, USA. E-mail: [email protected].

IEEE Robotics & Automation Magazine

89