FEATURE
Mobile computers are increas-
Peter Froehlich Telecommunications Research Center (ftw.) |
[email protected]
Lynne Baillie Telecommunications Research Center (ftw.) |
[email protected]
Rainer Simon Telecommunications Research Center (ftw.) |
[email protected]
physical and the digital worlds. This innovation demands a more sophisticated multidisciplinary approach to the modeling of spatial interaction than has yet been developed. Our aim in this article is to open up the “black box” of mobile spatial interaction (MSI) and discuss some issues and possible approaches that could be taken. Groundbreaking concepts that proposed how to combine virtual information with the user’s direct surroundings arose as early as the 1990s. Two early examples include Egenhofer’s Smart Compasses, which point and direct people to places of interest, and Geo-Wands, virtual geographic pointers for the selection of surrounding objects and
Geo-Wands:
Using mobile phones as digital pointers to realworld objects
i n t e r a c t i o n s J a n u a r y + F e b r u a r y 2 0 0 8
Realizing the Vision of Mobile Spatial Interaction
ingly used as a link between the
15
On The Language of Interactions
The
p2d sensor prototype [11]
augmented-reality techniques,
primary activity. This imposes
enabling the continuous inter-
strong constraints on the interac-
type of technological advance-
play between virtual and spatial
tion design.
ment was implemented in
information.
attached services [1]. [1] Egenhofer, M. J. “Spatial Information Appliances: A Next Generation of Geographic Information Systems.” First Brazilian Workshop on GeoInformatics, 1999.
An early application of this
Atlanta in the late 1990s; visi-
i n t e r a c t i o n s J a n u a r y + F e b r u a r y 2 0 0 8
16
[3] Counts, S., M. Smith, and J. Zhao. “Automated Route Annotation in Support of Community.” Proceedings, MSI workshop CHI2007, San Jose, Calif., 2007.
Outside the car MSI is quickly evolving as a means of sup-
tors to the Georgia Tech campus
ally focused on three main cat-
porting mobility in pedestrian
were able to experience enhanced
egories, which have now reached
navigation, outdoor sports, and
“future visions” of spatial aware-
a level of technical advancement
traveling. However, more work
ness [2].
that makes them ideal for tech-
is required in this area to better
nology transfer into commercial
support the user undertaking
has become increasingly feasible
devices. The three categories are
these activities. For example,
and desired. New technology,
wayfinding, access and creation
Scott Counts from Microsoft
including digital cameras, GPS,
of spatial data, and augmented
Research proposed that the
accelerometers, digital compass-
reality.
“route” should be viewed as a
Since then the concept of MSI
[2] Abowd, G. D., C.G. Atkeson, A. Dey, J. Hong, S. Long, R. Kooper, and M. Pinkerton. “Cyberguide: A mobile context-aware tour guide.” ACM Wireless Networks 3 (November 1997): 421-433.
Research in MSI has tradition-
Orientation and Wayfinding.
spatio-temporal document type
being built into mass-market
A good example of successful MSI
[3]. His group is currently inves-
mobile phones, thus enabling
is the “flight-mode” visualization
tigating how runners can author
the user to interact with his or
in today’s car navigation systems.
their own routes, which can be
her surroundings in new ways.
The bird’s-eye perspective facili-
automatically annotated with
These innovations are inspiring
tates an efficient match of digital
sensor data for outdoor sports.
industries to create new contact-
information with the driver’s
less applications, such as in the
view, thereby blocking out the
Data. A common theme found
domains of ticketing and vend-
task-irrelevant rear view and
in MSI research is the ability to
ing. Spatially related research,
making closer parts of the road
access digital information that
in the form of Web-based map-
more prominent than farther-
has been attached to physical
ping and geo-browsing, is begin-
away ones. As exemplified with
places called points of interest
ning to be transferred to com-
most wayfinding systems, MSI
(POI). The Geo-Wand is a research
mercial mobile devices. MSI is
is frequently a secondary task
concept that enables tourists to
further boosted by virtual- or
that should not interfere with the
point their mobile phone at a
es, and RFID chips, is constantly
Accessing and Creating Spatial
FEATURE
historic building and ask “What
the real world. However, this tag-
square, including water pipes and
is this?”
ging could quickly get out of hand
electricity, to assist urban plan-
with a high number of postings
ners and infrastructure service
MSI includes other aspects, such
in popular areas. Lemmelä and
providers [8].
as our sense of space and of
Korhonen from Nokia Research
social interaction. For example,
provide an interesting solution
Innovation Funnel.
researchers at Fraunhofer pro-
to this, a visualization method,
sense of orientation and mean-
duced a prototype of “StreetBeat”
which indicates the density of
ingfulness to conventional loca-
[4], a location-based mobile appli-
postings by overlaying semi-
tion awareness—and it is quickly
cation that aims to make people
transparent heat maps over the
stepping out of its research stage.
feel part of the cultural experi-
conventional map view and auto-
The ftw. project Point-to-Discover
ence of alternative-music clubs in
matically displaying the “hot”
demonstrated the high technical
Berlin by playing excerpts from
topics [6].
accuracy and attractiveness of
Beyond physical spatiality,
the previous night’s band as they wander by.
Augmented Reality.
For a tour-
Getting MSI through the MSI adds a
accessing nearby points of inter-
ist in a historic city, relevant
est with a spatially aware mobile
information about a specific area
phone, featuring a combination
ment academic MSI research in
is not always easily accessible.
of GPS, a digital compass, and
consumer products may latch on
A spatially aware mobile device
3D-accelerometers [9]. Japan,
to these types of social experi-
could serve as a window to vir-
where mass-market phones with
ences, and thus this area is one
tual information, such as the his-
integrated GPS and compass have
that can benefit from extended
toric past of a building [7].
been available since 2006, is the
Companies looking to imple-
research and development.
Virtually enhanced exploration
first test bed for commercial MSI.
is also useful as a professional
The phones can provide informa-
isting information and services,
tool. Schall developed an applica-
tion on nearby restaurants and
people are also increasingly cre-
tion that enables users to view
shops [10]. It is only a question of
ating and geo-referencing their
the subsurface structure of a city
time before spatially aware hand-
In addition to accessing preex-
own multimedia content. The challenge for MSI is to make this development fruitful for mobile
[4] McCall, R., S. Ghellal, J. and Rothauer. “Mobile Phones, SubCulture and Presence.” Proceedings, MSI workshop CHI2007, San Jose, Calif., 2007.
[5] Ahern, S., S. King, M. Naaman, R. Nair, and J.H.I. Yang. “ZoneTag: Rich, CommunitySupported ContextAware Media Capture and Annotation.” Proceedings, MSI workshop CHI2007, San Jose, Calif., 2007.
[6] Lemmelä, S. and H. Korhonen. ”Summarizing location based posting activity on communication hot spots.” Proceedings, MSI workshop CHI2007, San Jose, Calif., 2007.
[7] Baillie L., H. Kunczier, and H. Anegg. „Rolling, Rotating and Imagining in a Virtual Mobile World.” Proceedings, MobileHCI’05, Salzburg, Austria, 2005.
contexts. Several concepts in recent years, such as GeoNotes,
[8] Schall, G., E. Mendez, B. Reitinger, D. Schmalstieg, and S. Junghanns “Handheld Geospatial Augmented Reality Using Urban 3D Models.” Proceedings, MSI workshop CHI2007, San Jose, Calif., 2007.
have demonstrated the feasibility and attractiveness of attaching digital information to real-world coordinates, with metaphors such as virtual post-its or graffiti. Some of the most interesting with the socially motivated inter-
[9] Simon, R. and P. Fröhlich. “GeoPointing: Evaluating the Performance of an Orientation-Aware Location Based Service under RealWorld Conditions.” Proceedings, LBS2007, Hong Kong, 2007.
action forms of sharing and tagging. One example is the Zonetag mobile application, which allows for the context-aware upload of photographs from a camera phone [5]. Zonetag aims at lowering the effort of tagging on mobile devic-
[10] GeoVector. http:// www.geovector.com
es, by automatically suggesting tags based on the user’s social network, his previous entries, as well as names of nearby places in
Prototype
interface of Street Beat [4]
i n t e r a c t i o n s J a n u a r y + F e b r u a r y 2 0 0 8
MSI activities here are concerned
1
On The Language of Interactions
sets are sold globally.
[11] The Mobile Spatial Interaction (MSI) Initiative, http://msi.ftw.at
ity wants to send you a money-off
ist attractions) and create new
To realize MSI applications
voucher (deny and delete); your
applications (e.g., augmented-
and services, the strongest chal-
buddy is nearby and wonders if
reality games). As the techni-
lenge is how to conceptualize
you want a coffee (accept, audio
cal advancements move from
and design applications that are
alert); the train you need to catch
a research stage to production,
tailored to the needs of mobile
is delayed (accept, vibrate).
new visualization methods and
users. In this respect, the follow-
here is in the acquisition and
are needed. It is important to
Dealing with uncertainty. MSI
access to contextual data—to
highlight the importance of this
applications are subject to many
understand and act appropriately
kinds of uncertainties: GPS, sen-
on the infrastructure that is
topic and drive joint initiatives to make MSI transition success-
sor inaccuracies, or incorrect geo-
available when billions of people
ful [11].
tags. There are also technical and
start to contribute MSI data every
design issues provoked when the
day. We will need new systems
user moves from indoors to out-
that can store, search, and mine
doors. These certainties cannot
geo-spatial data. This will require
always be avoided, and there is a
interdisciplinary and hybrid
need to ensure transparency and
research across different fields to
communicate these inaccuracies
enable the successful collection
to the user.
and searching of such data in
Given the plethora of different
i n t e r a c t i o n s J a n u a r y + F e b r u a r y 2 0 0 8
standards and communication
18
multimodal interaction concepts
ing things need to be considered:
Display of spatial information.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without the fee, provided that copies are not made or distributed for profit or commercial advantage, and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on services or to redistribute to lists, requires prior specific permission and/or a fee. © ACM 1072-5220/08/0100 $5.00
Spatial content. The challenge
order for it to be useable, useful, and used. Identifying the business value.
interfaces for mobile devices,
The proven feasibility and attrac-
as well as a range of varying
tiveness of conceptual research
computing power available in
scenarios like wayfinding, POI
the devices, it will be quite dif-
access, and exploration should
ficult to design in global terms.
now motivate the definition of
Presenting spatial informa-
more commercially oriented
tion and various details about
application concepts. In-depth
the location and the people in
user research is one of the stron-
the location without provoking
gest success factors to identify
cognitive overload is an inter-
the real needs of specific user
esting and challenging interac-
groups, such as tourists in a
tion-design task. To increase
variety of mobile situations. This
the bandwidth and flexibility of
market-oriented research should
information display, the speech
feed into the definition of busi-
and sound-output capabilities of
ness and role models including
mobile devices should be exploit-
all stakeholders of MSI: end-user
ed to a larger degree.
communities, service providers,
Social disclosure and privacy.
mobile operators, content provid-
There are challenges in how we
ers, and handheld manufactur-
go about protecting individual
ers.
privacy—not only in meeting leg-
We see MSI as a major oppor-
islative and safety requirements,
tunity to make mobile internet
but also in being sensitive to
services useable, because interac-
what users want and do not want
tion styles such as pointing are
to make available about their
closely matched to situational
present location. Designing with
needs of mobile users. MSI will
these issues in mind is complex.
affect both how we interact with
For example, a shop in the vicin-
existing information (e.g., tour-
About the Authors Peter Fröhlich is a senior HCI researcher at Telecommunications Research Center Vienna (ftw.). He manages the project Point-to-Discover (p2d), which is co-funded by mobilkom austria, Siemens Austria, and the Austrian competence center kplus. The project develops the foundations for mobile spatial interaction: interaction techniques, modeling of spatial information, GIS data integration, and content aggregation, as well as hardware prototyping. Together with Lynne and Rainer, he has initiated the MSI initiative. Lynne Baillie has a Ph.D. in HCI from Napier University in Scotland. She has worked as a senior researcher for the Telecommunication Research Center Vienna (ftw.). Since 2002 she has investigated existing HCI methods for their applicability in the mobile domain and as a result has developed new methods in order to design more usable applications for mobile devices. She has also undertaken studies in the home and investigated how we can improve and extend user-centered development methodologies to enable codesign with families. Rainer Simon is a researcher at Telecommunications Research Center Vienna (ftw.) and a research fellow at the Vienna University of Technology Research Group for Industrial Software. He has several years of experience in the field of mobile application research and currently works with major mobile network operators on novel location-aware applications and gesture-based interaction methods for mobile phones.
