Wearable Computing Editor: Thad E. Starner
■
Georgia Institute of Technology
■
[email protected]
Wearables in 2005 Bradley Rhodes and Kenji Mase
I
n July 1996, one year before the first International Symposium on Wearable Computers, DARPA sponsored a workshop entitled “Wearables in 2005” (www.darpa.mil/MTO/Displays/Wear 2005). Attendees predicted how wearable computers might be used in 2005 and identified key technology gaps that needed to be filled to make their vision a reality. In October 2005, the 9th Annual International Symposium on Wearable Computing was held in Osaka, Japan, the first to be held in Asia. Participants presented a wide range of research from both industry and academia, spanning 13 countries and weaving together such diverse fields as interface design, hardware and systems, gesture and pattern recognition, textiles, augmented reality, and clothing design.1 Many of the themes would have sounded familiar in 1996, with continuing improvements in ergonomics and power management as well as gesture recognition and augmented reality. As you would hope, the field has also developed in new directions in the past decade, with a much greater emphasis on large-scale recording and annotation of everyday activities, on the science and engineering of clothing design, and on performing thorough quantitative evaluations of potential input devices. We have also seen a large increase in the use of accelerometers, smart phones, and RFID readers as researchers leverage continuing drops in cost and size in the consumer electronics world. As the largest primary conference for
92
PERVASIVE computing
wearables researchers, ISWC provides a good snapshot of the state of the field. So, with the benefit of hindsight, here are some highlights of how wearables research actually looked in 2005.
RFID FOR CONTEXT AWARENESS RFID readers have long been used as wearable input devices in semicontrolled environments because they let computers quickly identify tagged objects without requiring lengthy user input or com-
As the largest primary conference for wearables researchers, the International Symposium on Wearable Computers provides a good snapshot of the state of the field.
plex machine-vision algorithms. This year, RFID technology was ubiquitous, with six papers presenting work that used it. Two papers used RFID tags traditionally, as a fast input method. In “Wearable Technology for Crime Scene Investigation,” Chris Baber and his colleagues used RFID-tagged evidence bags to quickly link evidence to speech annotations and other log entries. Baber and colleagues note that using the evidence bags to tag and retrieve audio annotations can provide the advantages of digital records
while retaining the advantages of physical annotations written on the bags. In “An Event-Driven Navigation Platform for Wearable Computing Environments,” Masakazu Miyamae and his colleagues demonstrated a platform that could use a wide variety of sensors to determine a wearer’s location, including GPS for outdoors and RFID and bar code scanners for indoor locations. The wearer would explicitly scan a tag to receive information about a location. The remaining four papers used RFID tags not as explicit input devices but to automatically determine the context of wearers as they went about their daily activities. In “Fine-Grained Activity Recognition by Aggregating Abstract Object Usage,” Donald J. Patterson and his colleagues described a joint University of Washington/Intel Research Seattle study in which subjects wore a glove with an embedded RFID reader while performing such everyday morning activities as making oatmeal, eating breakfast, and setting and clearing the table. This research’s ultimate goal is to automatically detect “activities of everyday living” in areas such as elder care, where staff would like a general idea of what a resident is doing so that they can tell when to move the resident to a higher level of care. In the study, researchers placed RFID tags on every object in the kitchen that subjects touched during a practice trial. The tags provided the wearable with a continuous stream of information regarding what objects the subject was touching at a given time. The
Published by the IEEE CS and IEEE ComSoc ■ 1536-1268/06/$20.00 © 2006 IEEE
researchers then fed this data into an automatic classifier that attempted to label the activities. The best classifier correctly distinguished among 11 interleaved and occasionally interrupted activities 88 percent of the time, with 60 objects tagged in the environment. This paper won this year’s Best Paper Award, the first to be awarded at an ISWC. “Hands-On RFID: Wireless Wearables for Detecting Use of Objects,” by Kenneth P. Fishkin, Matthai Philipose, and Adam Rea, described the technical details of the RFID-reader glove, dubbed the “iGlove” (see figure 1). Assaf Feldman and his colleagues from the MIT Media Lab used a similar technique in “ReachMedia: On-the-Move Interaction with Everyday Objects,” one of this year’s six best-paper nominees. This project uses a bracelet instead of a glove and focuses more on interface and application design issues and less on activity classification than the previous project. The system provides justin-time information about objects in the environment—for example, a summary of a book the wearer is holding. The wearer gives explicit input using simple gestures, which wireless accelerometers in the bracelet detect. The wearable also automatically senses what object the wearer is holding via the wireless RFID reader. Technically, the authors call this “semi-implicit” input: the wearable can identify the object you’re holding, but you might have to hold it a certain way to make sure the reader reads the tag, especially with larger objects. The system speaks information about the held objects through a cell phone ear piece. The final RFID paper, “RFID Information Grid for Blind Navigation and Wayfinding,” by Scooter Willis and Sumi Helal of the University of Florida, described a system that helps the blind navigate by embedding RFID chips under rugs, along baseboards, and along sidewalks’ edges. RFID readers integrated into a shoe and the tip of a walking cane read the tags. Location is read out on a cell phone. JANUARY–MARCH 2006
Figure 1. The iGlove has an embedded RFID reader that informs researchers what objects the wearer is touching at a given time.
INTERFACE EVALUATIONS In wearable computing’s early days, most interface evaluation studies were either highly task driven (such as the pioneering Vu-Man studies at Carnegie Mellon University that looked at wearables for vehicle repair) or preliminary evaluations of what were mostly demonstrations of new concepts or technology. As the wearables field has matured, evaluations of wearable interfaces have become more quantitative and robust. One such evaluation this year was “The Impacts of Limited Visual Feedback on Mobile Text Entry for the Twiddler and Mini-QWERTY Keyboards,” by James Clawson, Kent Lyons, Thad Starner, and Edward Clarkson, another best-paper nominee. This is the latest in a series of papers from Georgia Tech on typing speeds for mobile keyboards. Previous papers studied learning rates for the Twiddler one-handed chording keyboard, expert Twiddler-user speeds, and Mini-QWERTY keyboard speeds and learning rates. In this latest longitudinal study, the authors found that experts on the mini-QWERTY keyboard can type around 60 words per minute (twohanded). This drops, however, to 46 wpm when typing blind (for example, as
you would if you were taking notes while looking at slides or typing under the desk during a meeting). This compares to about 47 wpm (with higher accuracy) using the Twiddler one-handed chording keyboard, which is already designed for eyes-free use. (In the interest of full disclosure, Bradley Rhodes was on Kent Lyons’ PhD thesis committee.) Another input-related study was “Acceptable Operating Force for Buttons on In-Ear Type Headphones,” by Vincent Buil and Gerard Hollemans, in which the authors determined the maximum force they could apply to a button on an in-ear headphone without causing discomfort. Buil and Hollemans were motivated by a new interface for controlling portable audio players (demonstrated at the conference) where you simply tap the ear piece in a given pattern to control selection and playback. For output, Jason Wither and Tobias Höllerer of the University of California, Santa Barbara evaluated three techniques to enhance the apparent depth of augmented-reality overlays in the third best-paper nominee, “Pictorial Depth Cues for Outdoor Augmented Reality.” The first technique uses horizontal and vertical “shadow planes” to PERVASIVE computing
93
WEARABLE COMPUTING
WEARABLE COMPUTING
a specific design or piece of research, this paper lays out a framework for thinking about smart clothing’s design. It also surveys issues ranging from identifying the end user’s body, activity, and cultural needs to fiber selection and layering to integrating electronics.
RECORDING EVENTS
Figure 2. In a presentation by Joanna Berzowska combining artistry and technology, memory wire makes brooch flowers open and close. (figure courtesy of XS Labs, 2005)
indicate the depth of the 3D cursor and any virtual objects by projecting their shadows on two receding grids. The second technique displays a small topdown view of graphical objects in the screen’s bottom-left corner. The third technique uses color-coding of markers and virtual objects to indicate their depth relative to the 3D cursor position. Markers range from light blue to red to deep green as they move from closer (to the user) than the cursor to further away than the cursor. The researchers were surprised to find almost no statistically significant differences among the techniques, but they did reduce the error rate from around 23 percent to around 10 percent when they added other markers to the scene to show relative depth. The authors suggest that the head-mounted display’s relatively low video resolution might have obscured differences among the techniques, especially since subjects viewed both graphical objects and the real world through the video screen.
CLOTHING DESIGN We’ve always encouraged artistic papers at ISWC, as long as they have technical content, but it’s often hard for presenters to bridge the culture and communications gap between the technical and artistic communities. Joanna Berzowska’s presentation of her and Marcelo Coelho’s paper on animated kinetic dresses, “Kukkia and Vilkas: 94
PERVASIVE computing
Kinetic Electronic Garments,” succeeded admirably. The project’s goal was entirely aesthetic: one dress’s hemline rises and lowers as if betraying or thwarting the wearer’s secret desires, and brooch flowers open and close of their own accord on another dress’s neckline (see figure 2). But the presentation included all the technical details and lessons necessary to accomplish these creations. In particular, the authors used Nitinol (memory wire) sewn into felt to cause the motion. After trying many configurations, they determined that a tight coil was the best configuration to “set” the Nitinol, as it created the largest motion. They also determined that felt was the perfect fabric for their projects for a number of reasons: Felt is sturdy, so when the Nitinol relaxes back to its nonset shape, the felt pulls the dress or flower back to the normal position. Felt is thick, so circuitry and wires can be woven into the fabric itself. And felt is a good insulator of heat and electricity and is relatively fire-retardant, so the wearer is protected if the electronics short out. Looking at clothing design from a more general perspective was bestpaper nominee “A Design Process for the Development of Innovative Smart Clothing That Addresses End-User Needs from Technical, Functional, Aesthetic and Cultural View Points” by Jane McCann, Richard Hurford, and Adam Martin. Rather than presenting
In the past few years, an increasing number of projects have looked at recording everything a wearable user does and then presenting that information in the form of an annotated diary. The main bottleneck for such applications isn’t storage, which continues to double every year or so, but annotating and filtering the raw data to make it browseable and searchable. In addition to the University of Washington/Intel project described earlier, several papers at this year’s ISWC addressed this difficult problem. A real-world example of this annotation bottleneck is the motivation for “Recognizing Mimicked Autistic SelfStimulatory Behaviors Using HMMs,” by Tracy Westeyn and her colleagues. Children with the neurodevelopmental disorder autism often engage in repetitive behaviors such as flapping their arms or screaming. Therapists treating such children would like to know the frequency with which these events occur to gauge their treatment’s effectiveness, but logs parents keep are notoriously inaccurate. It’s not practical for therapists to browse through days of video footage looking for these socalled self-stimulatory behaviors. The eventual goal of this research, which is still in its early stages, is to provide a system with which a therapist can browse through a relatively small number of video clips, watching the salient behaviors and throwing out the few remaining false positives. Also addressing this bottleneck is “Wearable Hand Activity Recognition for Event Summarization,” by Walterio Mayol and David Murray. This project uses a shoulder-mounted active camera to capture what you’re doing with your hands, with the ultimate goal www.computer.org/pervasive
WEARABLE COMPUTING
of automatically creating video logs of how skilled artisans perform their crafts. The system automatically detects hands by looking for skin color within the camera’s field of view and automatically pans the camera to follow your hands as you reach for and manipulate objects. These objects are similarly recognized by color histogram, and the video is then segmented into individual events based on objects being held and hand motion. The final result is a set of key-frame images, each representing a salient moment in the overall video. Another paper looking at the creation of video logs was the final best-paper nominee, “A Body-Mounted Camera System for Capturing User-View Images without Head-Mounted Camera,” by Hirotake Yamazoe, Akira Utsumi, and Kenichi Hosaka. Unlike the other papers, this work focuses on the ergonomic and fashion problems associated with headmounted cameras. Cameras that hang
Bradley Rhodes is a research
Kenji Mase is a professor at
scientist at Ricoh California
Nagoya University and pro-
Research Center and pro-
gram cochair of the Interna-
gram cochair of the Interna-
tional Symposium on Wear-
tional Symposium on Wear-
able Computers 2005 (ISWC
able Computers 2005 (ISWC
05). Contact him at mase@
05). Contact him at
[email protected].
nagoya-u.jp.
from a lanyard around the neck are far more comfortable and socially acceptable than head-mounted cameras. However, head-mounted cameras have the advantage of automatically capturing whatever the wearer is looking at—likely the most important part of a scene. Built on the authors’ previous work, the system uses one wide-angled front-facing camera and a pair of infrared cameras that look up at the wearer’s head. The system uses the image of the wearer’s head to estimate the direction the wearer is looking relative to the camera’s orientation and then extracts the region of the wide-angle image that corresponds to that view.
T
hese highlights are just a small sample of the 28 papers and 17 posters presented at ISWC 2005. You can obtain the full list of papers, including abstracts and PDFs, from the IEEE Computer Society Digital Library at www.computer.org/publications/dlib. We also hope you’ll participate in the next ISWC, to be held 11–14 October 2006 in Montreux, Switzerland. Submissions are due 21 April. Details are at the conference Web site, http://iswc.net. REFERENCE 1. Proc. 9th IEEE Int’l Symp. Wearable Computers (ISWC 05), IEEE CS Press, 2005.
Welcomes Your Contribution Computer magazine looks ahead to future technologies
• Computer, the flagship publication of the IEEE Computer Society, publishes peer-reviewed technical content that covers all aspects of computer science, computer engineering, technology, and applications. • Articles selected for publication in Computer are edited to enhance readability for the nearly 100,000 computing professionals who receive this monthly magazine. • Readers depend on Computer to provide current, unbiased, thoroughly researched information on the newest directions in computing technology. To submit a manuscript for peer review, see Computer’s author guidelines:
www.computer.org/computer/author.htm JANUARY–MARCH 2006
PERVASIVE computing
95
