Beyond Weiser's Vision of Ubiquitous Computing - CiteSeerX

2/11/2009

Beyond Weiser’s Vision of Ubiquitous Computing Stefan Poslad Queen Mary University of London http://www.eecs.qmul.ac.uk/people/stefan/ubicom

1

Overview • Ubiquitous Computing: achievements √ • Beyond Weiser’s model: Smart DEI model of Ubiquitous Computing • Research Examples

Ubiquitous computing:beyond Weiser: University Cambridge Talk 10-02-2009

2

1

2/11/2009

Ubiquitous Computing (UbiCom) • The term ubiquitous, meaning appearing or existing everywhere, combined with computing forms the term Ubiquitous Computing (UbiCom) – Term introduced by Marc Weiser in early 1990s – Synonyms: pervasive computing, etc

• UbiCom describes a vision for computing to – Enable computer-based services to be made available everywhere y – Support intuitive human usage but yet, appear to be invisible to the user. – Situated in physical (and human) world environments Ubiquitous computing:beyond Weiser: University Cambridge Talk 10-02-2009

3

What were ICT Environments Like? • In the late 1980s, when much of the early work on UbiComp started – people were often unreachable if away from a fixed phone point – computing was only available at a desk computer, attached to the wired Internet. – some early models of personal computers but no laptops were available – no pervasive wireless networks were available – little location-determination for non-military personnel was available

• Let’s look at the history of the main ICT development trends – A distinction is made between the availability of 1st prototypes (1) vs. 1st widespread commercial uptake of an ICT product (M). – Difference between the (1) and (M) phases is about 10 years.


4

2

2/11/2009

Short History of ICT Technology WearCom1

Weiser’s Class Tabs Room 2000 WWW1 WWWM TCP/IP Mobile M InternetMPhone-D

WLAN1 PC1

Computer1

1

1940

1970

TCP/IP WLANM GPSM Internet1 LapPDA1 GPS1 top1 LapPDAM topM PCM 1980

1985

1990

1995

2000

2005

Could also note when specific PC technologies arose, e.g., hard-disk, mouse, removal memory cards, etc Ubiquitous computing:beyond Weiser: University Cambridge Talk 10-02-2009

5

Early UbiCom Research Projects • Smart Devices: CCI – PARC Tab, MPad & LiveBoard; Active Badge, Bat and Floor

• Smart Environments: CPI and CCI – Classroom 2000, Smart Space and Meeting Room, Interactive Workspaces and iRoom, Cooltown, EasyLiving and SPOT, HomeLab and Ambient Intelligence

• Smart Devices: CPI – Unimate and MH-1 Robots, Smart Dust and TinyOS

• Smart Devices: iHCI – Calm Computing, Things That Think and Tangible Bits, DataTiles, WearComp and WearCam, Cyborg

• Other UbiCom Projects


6

3

2/11/2009

Active Badge, Bat and Floor Active Bat • Uses ultrasound, greater accuracy ~ 3 cm. • Base station asks Bat for a signal g that is then measured in multiple ceiling receivers (must add to environment), position determined using trilateration Active Floor • Unlike, Badge & Bat, identified someone by their type of walk or gait, not by carry an identifying token. Cons: scalability, accuracy • Floor design requires a careful analysis to specify an appropriate spatial resolution and robustness to allow users to walk on sensors without damaging them. Ubiquitous computing:beyond Weiser: University Cambridge Talk 10-02-2009

7

Cooltown

Car is one of 5 Cooltown demos. Here car detects it is low on fuel whilst on the way to a business meeting and guides user to a convenient filling station Ubiquitous computing:beyond Weiser: University Cambridge Talk 10-02-2009

8

4

2/11/2009

iHCI: Calm Computing • Example of calm technology was the “Dangling String” created by artist Natalie Jeremijenko, situated at PARC • 8 foot piece of plastic spaghetti that hangs from a small electric motor mounted in the ceiling. • Motor is electrically connected to a nearby Ethernet cable so that each bit of information that goes past causes a ti ttwitch tiny it h off the th motor. t • Hence the degree of twitching indicates the degree of network traffic in that Ethernet segment.


9

Tangible Bits

MetaDesk: tangible map

AmbientRoom: Light patches Bottles, Clock, Water ripples, handles

transBOARD: Networked digital interactive whiteboard Graphically view & record Drawings, tagged pens


10

5

2/11/2009

DataTiles

• Allows users to manipulate data in form of tangible “tiles” • Combinations of data streams and functions make it possible to create new applications Ubiquitous computing:beyond Weiser: University Cambridge Talk 10-02-2009

11

WearComp and WearCam • Mann’s experiments with wearable computers started in late 1970s. • Main application was recording personal visual memories that could be shared with other via the Internet. • (Photo from http://en.wikipedia.org/wiki/Wearable http://en.wikipedia.org/wiki/Wearable_computing) computing)


12

6

2/11/2009

Cyborg 2.0

Electrode array surgically implanted into Warwick’s left arm and interlinked into median nerve fibres is being monitored.


13

Analysis of Early Projects: Distributed Access Support • Mobile device model design for Tabs and Pads. • Supported communication & location-awareness for mobile users, commercial mobile ICT devices, widely available wireless networks. • Late 2000s, 2000s mobile devices such as phones and laptops are widely used and wireless networks are widely available that support data communication routing to users wherever they are • Service discovery of local network resources is weak and the discovery of other local environment resources is still virtually non-existent • Hence, much of the vision of Cooltown is not routinely available. – smart environments not yet mature or widely available. – diversity of support needed needed, cost and secure fixings fixings.


14

7

2/11/2009

Analysis of Early Projects: contextawareness • Context-awareness: mainly location awareness • Early achievements based upon (local not global) location awareness indoors with heavily instrumented environment. • Legacy: Global Location determinism (via Mobile phone, GPS) but accuracy limited to 10s of M. • Integrated into some mobile devices, e.g., phones, cameras, cars. • Location-determinism tends to be supported mainly as stand-alone devices and services that are not readily interoperable. • Mainly M i l ffor outdoor d use. • Systems for indoor use are available today based, e.g., based upon trilateration using WLAN but not ubiquitous


15

Analysis of Early Projects: iHCI • Electronic boards – Allow users to collaboratively edit text and graphics – PARC prototype in early 1990s now commercial products. – “Classroom 2000” use in 1995-1998 (Abowd) now routinely used in many educational establishments

• Wearable smart devices – several products are available but not in pervasive use.

• Robots – Heavy use in clean room manufacturing, not in open environments apartt from f cleaning, l i mowing i robots b t and d robot b t ttoys and d pets t

• iHCI – A continuing research initiative. – Very many variations – Mass market ? But Wii is popular Ubiquitous computing:beyond Weiser: University Cambridge Talk 10-02-2009

16

8

2/11/2009

Today: Living in an Increasingly Digital, Interconnected, World • • • • • • • • •

More everyday human activities heavily reliant on computers More devices per person, building, transport vehicle, etc Greater variety of general purpose vs. task specific computer devices Devices smaller/ larger?, cheaper, use less energy, reliability ↑ (or ↓?) Profusion of multi-purpose, intelligent, mobile devices can access remote services ... and more local services Physical & Human world strewn with embedded sensors & control devices More devices can interoperate in ad hoc versus planned ways High speed wire(less) networks are pervasive & accessible by all all, can be added less disruptively into the physical environment . ↑ Energy efficient devices , yet ↑ overall energy consumption


17

Everyware UbiCom Applications • Vision: ubiquitous computer systems to support people in their daily activities in the physical world tasks to simplify these and to make these less obtrusive. • People will live, work, and play in a seamless computer enabled environment that is interleaved into the world. • Bushnell (1996) coined variations of term ware such as deskware, couchware, kitchenware, autoware, bedroomware and bathware to reflect the use of ubiquitous computing for routine tasks. • Greenfeld (2006) used the term everyware to encompass the many different types of ware • Some Everyware scenarios follow Ubiquitous computing: smart devices, environments and interaction

18

9

2/11/2009

UbiComp System GPS Transmitter

Location Determination

Clock

Display

AVAV Capture

Image Processing

AV-player AV player

Projector Communication

AV database

Removable Memory Printer

Automatic face detection, recognition, etc

dd/mmyy x,y

GIS

Ubiquitous computing:beyond Weiser: University Cambridge Talk 1019 02-2009

Why is the Bus late?

Ubiquitous computing:beyond Weiser: University Cambridge Talk 1002-2009

20

10

2/11/2009

Location Determination

NO. TOA 211 14:11 107 14:17


Appliance, e.g., Food heater Ceiling

21

External Gas Grid Manual Controller e.g., Food Heating Manual Timer

Desk Lights

Sensors Temperature

Wall External Electricity Grid

WLAN

Mobile Phone

ICT Network Storage

WWAN

22


11

2/11/2009

Appliance, e.g., Food heater External Physical Environment Sensors

Ceiling

Sensors, food Temperature Digital, networked Controller & Timer

Desk Lights Wall External Electricity G id Grid

Mobile Phone

ICT Network Storage 23


WWAN

WLAN

Overview • Ubiquitous Computing: achievements • Beyond Weiser’s model: Smart DEI model of Ubiquitous Computing √ • Research Examples


24

12

2/11/2009

UbiCom: Weiser’s 3 Internal System Properties 3 main properties for UbiCom Systems were proposed by Weiser (1991) 1. Computers need to be networked, distributed and transparently accessible –

In1991, little wireless computing, Internet far less pervasive

2. Computer Interaction with Humans needs to be more hidden –

Because much HCI is overly intrusive

3. Computers p need to be aware of environment context –

In order to optimise their operation in their physical & human environment.


25

Internal System Properties: iHCI • Concept of calm or disappearing computer model: – Device too small to be visible, embedded, – Devices visible, not noticeable, part of peripheral senses

• Implicit (iHCI) versus Explicit HCI – – – – –

Much HCI is explicit, low-level, more devices -> human overload More natural, less conscious interaction Systems anticipate use -> explicit interaction ↓ Calm Computing If explicit HCI ↓, -> careful balance between several factors

• Embodied Reality as opposite of VR (people in virtual world) – Devices as embodied, env. aware, service access & execute entities – Devices aware of, & bounded by, physical, not just virtual env.


26

13

2/11/2009

Internal System Properties: contextaware • Context-based ubiquity rather than global ubiquity • Physical Environment Context: location, time, temperature, rainfall, light level etc. • Human Context (or User context or person context): interaction is usefully constrained by users’ identity; preferences; task requirements, etc. • ICT Context or Virtual Environment Context: UbiCom system is aware of the services available that are available internallyy and externally, y, locallyy and remotelyy • Active (by system) versus passive context adaptation (by user)


27

Devices: Weiser’s 3 Internal System Properties HPI Physical

Human

ICT

Environments CPI

UbiCom System

HCI implicit HCI context-awareness distributed ICT ICT

CCI

Virtual ICT


28

14

2/11/2009

Devices: Extended set of Internal System Properties 3 main properties for UbiCom Systems were proposed by Weiser 1. Computers need to be networked, distributed and transparently accessible. 2 Computer Interaction with Humans needs to be hidden more 2. –

Because much HCI is overly intrusive

3. Computers need to be aware of environment context –

In order to optimise their operation in their physical & human environment.

To which two additional properties are added: 4. Computers can operate autonomously, without human intervention, be self-governed 5. Computers C can h handle dl a multiplicity l i li i off d dynamic i actions i and d interactions, governed by intelligent decision-making and intelligent organisational interaction. This entails some form of artificial intelligence.


29

UbiCom: Different Combinations of Core Properties versus a Single Definition • No single, absolute definition for ubiquitous computing. • Instead propose many different kinds of UbiCom based upon combining different sets of core properties Here are some examples proposed by others • Weiser (1991): distributed, iHCI, physical environment context aware • Ambient Intelligence (AmI), similar to UbiCom - intelligence everywhere? •

Arts and Marzano (2003) define 5 key features for AmI to be embedded, contextaware, personalised, adaptive and anticipatory.

• Buxton (1995): ubiquity and transparency • Endres et al al. (2005) (2005): distrib distributed ted mobile mobile, intelligence intelligence, a augmented gmented realit reality • Millner (2006): autonomy, IHCI etc .


30

15

2/11/2009

UbiCom System Model: Smart DEI • No single type of UbiCom system • Different UbiCom systems applications support: – Internal properties (distributed, iHCI etc) to different degrees – Different types (phys. virtual, human) of external environment interaction to different degrees – Different form-factors (six basic forms) for devices – Multiple systems can combine to form interacting systems of systems

• 3 basic architectural design patterns for UbiCom: – smartt Devices, D i smartt Environments, E i t smartt Interaction I t ti

•

‘Smart’ means systems are: – active, digital, networked, autonomous, reconfigurable, local control of its own resources, e.g., energy, data storage etc. Ubiquitous computing:beyond Weiser: University Cambridge Talk 10-02-2009

31

Five main properties for UbiCom require a multi-disciplinary approach to R&D Networked Transparent access Openness

Automated, Independent Selfgoverned

Autonomous

Sense, adapt & control environment Physical World, World Person ICT infrastructure aware

Distributed Context-aware

iHCI

Disappearing Computer Handling Non-determinism Intelligent Implicit Interaction Knowledge & task sharing Embodied Reality Goal-based, etc. Mediated Reality Social intelligence Ubiquitous computing:beyond Weiser: University Cambridge Talk 1002-2009

32

16

2/11/2009

Ubiquitous Computing System User‐aware Location

Post‐human Immersed Hidden Anticipatory

Physical Human Time Phenomena Person Activity Sense

ICT Resources

Communal Orchestrate Cooperate Compete Mediate Reactive Env Model Goal Utility

Self‐ Model

Context‐ aware

Organisational Intelligent

Learning

Search Knowledge Uncertainty Reason Un‐ Embedded Control tethered

iHCI

Individual

Plan

Persist Auto‐ ent nomic

Autonomous

Distributed System (Single Virtual Computer) y ( g p ) Discovery

Mobile

Transparent

Openness

Secure Fault‐ tolerance

Distributed

Shared Interoperable Heterogeneous Virtual concurrency

Ad hoc vs. Wireless vs. Asynchronous vs. P2P vs. Client‐ server Fixed Wired Synchronous

Local vs. Global

Networked

Mobility: Dimensions Mobile Services Mobile Context LAS, LAS etc

Mobile Devices

Self-powered,

Mobile Code

Volatile, open, dynamic distributed services

Mobile Data

Mobile Host

Wireless Communication

Manual control Automatic Type of Mobile Host

Physical Dimensions Skins

Animate Pads

Inanimate

SurfaceSurface mounted

Tabs Artefacts

Air or Fluid

Dust Smart Phone

How device is attached to host

Accompanied

Human Laptop Computer

Smart Card

MEMS, Sensors, RFID Tags

Mobile Communication

Wearable

When Mobility occurs

Manufacture to Install Mobile between sessions

Embedded

Implants

Remove to De-install

Mobile during sessions

17

2/11/2009

Devices: Extended set of Internal System Properties Physical Environments Physical Ecological Phenomena (Living) CPI

(Survive, Adapt)

Human Environments Personal

CPI (Sense,

Public

Social HCI (Cooperate)

HCI

Adapt)

HCI (Compete)

implicit HCI ContextAware

Autonomous

Distributed

Intelligent

UbiComp System

ICT CCI ICTI Virtual Environments


Increasing Human (H) Interaction

Multi-lateral HCI H2H

Human interaction mediated by UbiComp

H2C / eHCI

Minimum

Increasing Physical (P) World Interaction

Humans use model of ICT H d l f ICT to explicitly interact. H personalises C

C2H / iHCI

C aware of H’s context & adapts to it. C is personalised itself

C2C

0

35

Minimum

Autonomous ICT system interaction system interaction

Increasing Ubiquitous Computing (C) Interaction


36

18

2/11/2009

Multi-lateral CPI

Increasing Physical (P) World Interaction

P2P

Increasing Human (H) Interaction

Physical interaction (No ICT mediation) C Senses P C Senses P C Aware of P’s Context

C2P / CA C Augments or Mediates P’s reality. C Adapts to P’s context

P2C/AR/MR

Virtual Reality Virtual Reality facilitated by C

C2C /VR

Minimum

0

Increasing Ubiquitous Computing (C) Interaction Ubiquitous computing:beyond Weiser: University Cambridge Talk 10Minimum

37

02-2009

Form Factors for Smart Devices Spatial Dimensions 3D Volumes

Clusters of 1D technologies

Ear Receiver

3D Surfaces

Organic UIs 2000s Mobile Phones

2D Planar

1980s Mobile Phones

Laptops

Electronic Whiteboards

ebooks

1D Points

Nanotechnology

Nano

MEMS

Milli Micro

Centi

Deci

Meter


Macro

Device Size 38

19

2/11/2009

UbiCom System Model: Smart DEI Device Trends Increasing capability to manufacture low power, micro, more complex devices

Use more complex, multifunctional, mobile, personalised (& private) smart devices to ease access to & embody services rather than just to virtualise irt alise them e.g., phone is also a camera, music player, is also a printer??

Increasing capability to embed devices in the physical h i l environment i

Use smarter environments to sense and react to events such as people, with mobile devices , entering & leaving controlled spaces e g walls can sense camera e.g., is recording and modify lighting to improve recording

Increasing capability for more interoperable distributed mobile devices

Use more service access devices with simpler functions and allow them to interoperate – smarter interaction between devices ee.g., g camera can interconnect to phone to share recordings, direct to printer to print

UbiquitousWeiser: Computing Ubiquitous computing:beyond University Cambridge Talk 1002-2009

Smart Device

1-1 Interaction

Smart Mobile Device

39

Smart DEI Model 1-M, M-M Interaction

Smart Environment

Mobile, Personalised, MTOS, Remote 1-1 interaction

Fixed vs. Untethered, ASOS, sense-control, local 1 1-1 1 interaction

Design: Fat-clientproxy-server ?

Design: Autonomous Peer-to-Peer interaction?

Smart Interaction

1-M, M-M, richer (coop, compete, semantic, etc), P2P interaction

Design: Appliance Model, Service Pool, Remote Service Access Point, Service Design Thin-clientContract, Ubiquitous computing:beyond Weiser: University Cambridge Talk 10- on-demand, 40 02-2009 proxy-server interaction? autonomic?

20

2/11/2009

Overview • Ubiquitous Computing: achievements • Beyond Weiser’s model: Smart DEI model of Ubiquitous Computing • Research Examples √


41

CRUMPET, CReation of User-friendly Mobile Personalised for Tourism Project • The main objectives of the CRUMPET project was to: – –

Implement and trial tourism- related value-added services for nomadic users: Evaluate the use of agent technology as a suitable approach for the fast creation & composition of such services.

• Hence based upon a Multi-Agent System ( MAS design), to – – – – –

ACL to factor out common service actions to ease interoperability across services Semantic approach to service mediation to integrate multi-service content Content adaptation for mobile use & heterogeneous ICT, wireless network, terminals Location-aware map service Personalisation

• Two designs – Active A ti iintelligent t lli t mobile bil client li t ((micro i agentt platform) l tf ) aids id content t t adaptation d t ti – Passive mobile client (micro-browser)


42

21

2/11/2009

CRUMPET Project integrates 4 key technologies Multi-agent technology Location-aware services

Multimedia iinformation f ti access over

Personalised user interaction

mobile terminals

08.10.2001

CRUMPET Project IST-1999-20147

43

Architecture Variation A: This deployment architecture has a larger client-side footprint and is suitable for deploying in high end PDAs and PCs

08.10.2001


44

22

2/11/2009

Architecture Variation B: This deployment architecture has a very small client-side footprint and is suitable for deploying in low end PDAs and suitably equipped mobile 'phones TA Satellite Wireless Station

Service Provider

GSA

HTTP HTTP - no control over link

Fixed Network SCA JAVA process very little control over GPS and browser

Wireless Station

Service Provider MAPA

ExplorerCE MA

DCA

CASA

Service Provider

CRUMPET Services

CCA

08.10.2001

CA

SA

UMA


45

CRUMPET ICT Adaptation Map components: • Map of the nearby “world” • Start/Edit tour • Status bar with proactive “bulb”

My IP address and port are...

Ok, here are your nearby points of interests.

Here is my new location.

Components: • Map of the “world” • Diagnostics information • Client status (Agent and network status) • Points of interests

08.10.2001


46

23

2/11/2009

CRUMPET System


08.10.2001

47

Crumpet System: conclusions • Low resource terminals find it challenging to parse rich semantic messages • Multi-lateral approach to adapt content: – E.g., variable image compression, text headings vs. whole body, etc

• Context acquisition & composition design: – What is an event: user stopping? Etc? – Ordering: location -> personalisation -> terminal or different – Generic versus specific adaptation e.g., personalisation

• Agent design – Simple reactive and proactive – ACL need? Ubiquitous computing:beyond Weiser: University Cambridge Talk 10-02-2009

48

24

2/11/2009

Pearson (2000) Predictions for Smart Environments Automatic body measurement scanners

Nano devices roaming within blood vessels

Food Electronic packaging outnumber chips tell state Solar reflector organic pets satellites increase of food Technology to Nanotechnology sunlight Emotional enhance illusion garden plants objects, switches of tradition Micromechanical in home garden gnomes Mood sensitive Reconfigurable light bulbs Space Automatic body g buildings robots outnumber solar measurementt people in some power Anti-noise Self- driving scanners countries stations technology cars on smart Digital highways homes bathroom mirror 5 years ago to present time

2010

2015

2020

2025

2030

Ubiquitous computing:beyond Weiser: 49 University Cambridge Talk 10-02-2009

Conclusions • Good opportunities for mass use of smart devices and context-ware devices began relatively recently – these opportunities are expanding. • References • Poslad S. (2009) Ubiquitous Computing: Smart Devices, Environments and Interaction. Wiley, ISBN 9780470035603 http://www.elec.qmul.ac.uk/people/stefan/ubicom/ • Pearson (2000) Pearson II.D. D (2000) Technolog Technology Timeline Towards Life in 2020. BT Technology J., 18(1): 19 – 31


50

25