Seungpyo Hong, Daeil Seo, Gyuwon Song, Sangchul Ahn,. Donghoon Kang, Jinwook ... services running on a ubiquitous computing environment will be how to find ... have technologically advanced as technologies developed. Now with the ...
Follow Me Follow You: Location Based Ubiquitous Communication Service Framework Seungpyo Hong, Daeil Seo, Gyuwon Song, Sangchul Ahn, Donghoon Kang, Jinwook Kim, Heedong Ko Imaging Media Research Center, Korea Institute of Science and Technology, 39-1 Hawalgok-dong, Sungbuk-gu, 136-791, Seoul, Korea {junon , xdesktop, sharp81, prime, choco, jwkim, ko}@imrc.kist.re.kr
Abstract As sensing, actuating and computing devices are becoming more available around us, more intelligent communication services considering a user context attract broad interests. One of the important problems related to communication services running on a ubiquitous computing environment will be how to find sensing and actuating devices around users and establish the communication automatically. We suggest Follow Me Follow You service, a framework to past-prototype an autonomous and pervasive communication service for a ubiquitous computing environment. Our framework consolidates the software architecture with various UPnP devices to discover proper sensing and actuating devices, establish the peer to peer communication, and enable dynamic communication flow transition.
Keywords Human-human communication, Context-awareness, Proximity selection.
INTRODUCTION The means of communication between human beings have technologically advanced as technologies developed. Now with the ubiquitous computing environment shaped around us, a demand for new possibilities of communication has emerged. Nowadays, sensor network enables to perceive environments around us and to track the people or vehicles. Cell phones, televisions, automobiles and other devices used in our daily lives are getting stronger computing power, more useful peripherals and smarter intelligence. Evolution of these technologies gives us an opportunity to make new applications that enrich human’s life. There have been several attempts to develop new ways of communication using ubiquitous computing technologies. Family Intercom[2] and Geocast[3] use person’s location as a context, and establish a communication or send a message based on that context. However their approach shares similar problems with the traditional communication methods in a sense that an explicit intention of the caller such as pushing a button or speaking callee's name is required. There are situations where these limitations become critical. Babies or small children, for example, have little
ability to manipulate a communication device. Nevertheless, they are highly mobile so that it is hard to keep track of them by using the conventional devices. Patients who are staying in their homes have similar kinds of issues. In the case of emergency, he or she may not be near a communication device or is perhaps unable to use it, but the doctor still has to be notified as quickly as possible. In a ubiquitous computing environment where a number of computing and media devices interact with human beings and collect a variety of information about the people and the surrounding environment, it is possible to overcome the limitations. In this paper, we suggest a new communication service that is made possible with the ubiquitous computing environment: Follow Me Follow You Service. To make this service possible, several premises must be satisfied. We need sensors to track the person’s location and devices to capture and deliver corresponding messages. Also we need a location model to represent person’s location, device’s location and device’s working range. A service agent that is a software application running on a middleware is required to establish, maintain and terminate a connection.
FOLLOW ME FOLLOW YOU SERVICE 1.1 Concept To overcome the limitations of traditional communication devices mentioned at the introduction, we need an autonomous system that is not only embedded in the environment to provide input/output functionalities, but has its own intelligence to enable communication between human beings. In a ubiquitous computing environment, there are numerous computing devices with sensors and actuators. In comparison to the conventional communication means, the sensors are the input devices, and the actuators are the output devices. In Follow Me Follow You (FMFY in abbreviation) service, sensors of our interests are microphones and cameras. These sensors collect information about the environment and the user. Likewise, we use several display devices and speakers as an actuator which is a device that affects the environment and the user. Considering a type of modality, there exists a pair of a sensor and its associated
actuator such as {microphone and speaker} and {camera and display device}. In a limited environment, our service may not find a proper pair of sensor and actuator of the same type of modality. To solve this problem, media conversion process such as text-to-speech and speech-to-text can be applied. Also, there is a location provider that can keep track of a user’s position and has a capability to send the position to the others in network. Using the location provider, our FMFY service agent finds locations of the users, look for available sensors and actuators near the users, and establish a connection between the sensors and the actuators.
A` is same place with room A, but camera’s coverage shrinks as light condition get worse. In this case, camera on the left wall is better choice than camera on the upper wall.
Figure 2. Possible Problems in Device Selection Figure 1. Flow change in FMFY Service.
Figure 1 explains how FMFY service works in the environment with sensors, actuators and a location provider. If a person in space A moves to space B, our location provider reports changes of a user position to FMFY service agent which is responsible for establishing a communication flow. Then FMFY service agent changes a source of the flow from sensor1 to sensor2. Similarly when a counterpart of the communication moves to space C from space D, a destination of the flow changes from actuator2 to actuator1.
1.2 Device Selection Mechanism
The second problem arises when a pair of sensor and actuator has a different level of quality. Since each sensor or actuator has its own resolution and quality of sensing or actuation capability, mismatching of sensor and actuator pair may occur. For example, instead of 1600*1200 resolution display, 800*600 resolution display is suitable for render the images from 50M pixel camera in the room A. Moreover transmitting data acquired from a microphone to a general display system is a nontrivial problem, since modalities of sensing and actuating devices are incompatible to each other in the case of room C to room B. It reveals us that the proximity selection for a communication service requires finding the right pair of sensor and actuator.
Basically, FMFY service is an extension of proximity selection application. Proximity selection is one of the widely used methods to develop ubiquitous computing applications[4]. FMFY service also uses the proximity selection to choose appropriate sensors and actuators nearby the person. Unfortunately, proximity is not sufficient condition of appropriateness. There exist other considerable points more than proximity, such as coverage, data quality and modality. Figure 2 describes possible problems with coverage, data quality and modality. We define coverage of a device as an effective volume of the device where information can be sensed or an action of the device can be delivered. In the room A, if we only consider the proximity, we will select camera on the right wall even though only half of person can be captured. Not only a position of the device but also its coverage should be considered for proximity selection. Moreover device’s coverage can be changed by environment, or by device’s movement such as pan and tilt. Room
Figure 3. Evaluating Sensor-Actuator Pairs
By evaluating scores for proximity, coverage, data quality matching and modality, and by merging them, we can calculate the scores for each pair as shown in Figure 3. Then,
the highest scored pair is assumed to be most appropriate pair for the communication.
1.3 Multi-layered Service Architecture
cluding washing, cleaning, and cooking. However, it is hard to baby sit while doing other jobs because a baby is likely to move around. In this case, we can simplify FMFY service as following. Camera follows the baby, and video follows the baby sitter: Service agent can keep track of the baby and let the camera record the baby while the baby sitter watches the baby through a display device that is closest to him or her. The babysitter can cook in the kitchen while the baby plays with a toy horse in the other room as shown in Figure 5.
Figure 4. Roles of Place and Person Layers
In a ubiquitous computing environment, a connection of the communication can be triggered by a person who wants to communicate with another person, by a third person or even by an autonomous computing system. Users of communication can be mobile people, fixed devices, places and computers. To generalize these kinds of communications, we introduce two layers, “place-to-place layer” and “person-to-person layer”. FMFY service defines persons involved with the communication first at the person-toperson layer and then asks the place-to-place layer to establish the connection by finding a proper pair of sensor and actuator as shown in Figure 4. At the person-to-person layer, FMFY service identifies individuals involved with the communication and request spatial information of the persons to the location provider. This layer needs information about who has which tracking device, relation between people, characteristic of the people and all the environmental information to recognize the intention of the people. More than one person-to-person communication can exist in this layer and be connected to place-to-place layer. Place-to-place layer is responsible for finding a proper pair of sensor and actuator, establishing a connection. Place-to-place Layer does not care about who is the owner of location tracking device and who is user of sensors and actuators. This layer is more close to resources, while upper layer (person-to-person layer) is more close to human. In this layer, common module exists to handle various commands from upper layer and to avoid resource conflicts.
Figure 6. Doctor-Patient Service
Patients can have emergency situations anytime and anywhere. In these cases, they often cannot take care of themselves or call for help. FMFY service can also be used for these cases. In this case, communication can be established by combining two single-way communications. Camera follows the patient, video follows the doctor: The service agent may conclude that the patient is in an emergency state, then it records the patient’s images using the nearest available camera and shows it to the doctor using a display device that is nearby. Microphone follows the doctor, speaker follows the patient: When the doctor gives emergency treatments, the FMFY service agent records the voice using the nearest microphone and plays it on the speaker that is closest to the patient.
1.5 Implementation For the implementation of FMFY service, we used three UPnP[6] devices: location provider, network camera, and media renderer. The UPnP devices communicate with our software agent through CAIM (Context Aware Interaction Manager) [1] used as a middleware.
1.4 Applications FMFY Service can be applied to more sophisticated Ubicomp. applications, such as babysitting service and medical service. Figure 7. Layout of our Ubicomp. Infrastructure
To establish ubiquitous computing environment, we deployed 10 Network Cameras, 8 Media Renderers and 35 Cricket beacons as shown in Figure 7. These devices are installed to maximize their working range. Figure 5. Babysitting Service
A baby sitter may need to work on other house chores in-
1.5.1 Location Provider Location is an important context in ubiquitous computing applications. Therefore the location provider also plays
important role in our application. In our application, indoor location tracking is performed by using Cricket Location System[5]. Cricket beacons are installed on the ceilings of corridors and rooms as shown in Figure 8(a) and (b). A Cricket listener is connected to a PDA via Bluetooth module (see Figure 8(c)).
(a)
(b)
(c)
Figure 8. (a) The Cricket beacons in the Corridor (b) The snap shot of Location Provider (c) a Cricket listener
1.5.2 Network Camera and Media Renderer We used multiple AXIS 210 cameras to capture the user’s movement and to provide MPEG2/MJPEG video streaming service in the network. When the IP address and the location of the cameras are advertised to the network, our middleware, CAIM recognizes the existence of network cameras and types of available services. Media Renderer is capable of playing MJPEG video. In our implementation, it provides not only the basic functionalities of media control such as play, stop, and pause, but also the location information of itself. This is a kind of software running at the distributed computers which are connected to network and display device such as PDP and LCD.
1.5.3 Middleware Cricket Beacon
Cricket Beacon
Cricket Beacon
Cricket Beacon
Cricket Beacon
Cricket Listener
Cricket Listener
Location Provider
Location Provider
Cricket Beacon
1.5.4 Service Agent Although FMFY Service Agent is the key player in FMFY Service, it is rather simple. In this implementation, place-to-place layer and person-to-person layer are implemented in same agent. They just distinguished as a class level, because Agent Manager can handle only one agent now. Service agent queries information about sensors, actuators and location providers. After collecting and updating the information, device selection mechanism is applied for all the possible pairs. Finally, service agent sends the command to devices through the CAIM to make data streaming from sensor to actuator. This process is iterated during communication is valid.
CONCLUSION AND FUTURE WORK In order to have an autonomous and pervasive communication service, we proposed FMFY Service in a ubiquitous computing environment. It can be applied to Baby Sitting Service and Doctor-Patient Service as previously explained. To implement FMFY Service, we have developed three UPnP devices and used CAIM to orchestrate between the devices and the service agents. Device selection mechanism is quite restricted for one to one pair selection. There are several cases which cannot be covered with current mechanism such as multiuser cases and one to N cases. In the future work, these factors will be considered. For the implementation, more actuators, sensors and users must be participated for the scalability and robustness test. Various type of sensors and actuator are needed to examine modality issue.
ACKNOWLEDGMENTS This research is supported by the ubiquitous Automation Computing and Network Project, the Ministry of Information and Communication(MIC) 21st Century Frontier R&D Program in Korea
REFERENCES
See
[1] Ahn, S., et al., “The Interaction Framework for Context Aware System”, Int. Conference on HCI( 2005)
C
UPnP
UPnP
Video Streaming
CAIM
UPnP Media Renderer
person from certain location, request for deliver a command to a certain device, and so on) from service agents
Response
UPnP
Network Camera
Query/Command
Service Agent Figure 9. How Components Works Together.
The middleware, CAIM, plays two important roles in our applications. First one is discovery, control and management of distributed UPnP devices. Second one is responding to the queries (request for subscribing changed status of a certain devices, request for finding the nearest device or
[2] Nagel, K., et al., “The Family Intercom: Developing a Context-Aware Audio Communication System”, Proceedings of UbiComp (2001) [3] Durr, F., et al., “On a Location Model for FineGrained Geocast”, Proceeding of UbiComp(2003) [4] Schilit, B. N., et al., “Context-Aware Computing Applications”, IEEE Workshop on Mobile Computing Systems and Applications(1994) [5] Cricket Location System, http://cricket.csail.mit.edu/ [6] UPnP Forum, http://www.upnp.org
