servers or lists, or to reuse any copyrighted component of this work in other works must be ..... processes can support dedicated transactional guarantees, even in case of failures and ... replication can be made affordable. First, updates on cer-.
In: Proceedings of the 2nd International Conference on Web Services (ICWS’2004), pages 358-366, San Diego, CA, USA, June 2004.
Hyperdatabases for Peer-to-Peer Data Stream Processing Gert Brettlecker, Heiko Schuldt, and Raimund Schatz University for Health Sciences, Medical Informatics and Technology (UMIT) {gert.brettlecker, heiko.schuldt, raimund.schatz}@umit.at
Abstract New sensor technologies, powerful mobile devices, and wireless communication standards strongly proliferate ubiquitous and pervasive computing. This is particularly true for healthcare applications. Modern non-invasive unobtrusive sensors enable sophisticated monitoring of the human body. As a result of this monitoring a growing amount of elderly people suffering from chronic diseases will benefit from an enhanced quality of life and improved treatment. Healthcare applications process a vast amount of continuously generated data, this has become a serious challenge. Data stream management addresses this problem in general. However, healthcare applications have additional requirements, e.g., flexibility and reliability. As a consequence, we propose the combination of a sophisticated information infrastructure, called hyperdatabase, and data stream management. Hyperdatabases already allow for reliable process management in a peer-to-peer fashion. In this paper, we elaborate the close relation between distributed process management and data stream management. Further, we show that data stream processing can significantly benefit from an infrastructure for reliable process management. We present the architecture of our prototype infrastructure that supports processing of continuous data streams with a high degree of flexibility and reliability.
1. Introduction With the advent of new sensor technologies, powerful mobile devices, wearable computers, and wireless communication standards, new generations of applications emerge. Ubiquitous and pervasive computing are starting to infiltrate people’s daily life. As a consequence, an increasing amount of data is continuously generated. These developc
2004 IEEE. Personal use of this material is permitted. However, permission to reprint/publish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. To appear in ICWS’04
ments raise new challenges in the field of information management. A proposed solution for the task of managing continuously generated data in real-time is data stream management (DSM). Various activities currently aim at providing new paradigms and techniques to handle and process information flows like a DBMS does with static data.
1.1. Monitoring in Healthcare An important area of new data stream management applications is monitoring in healthcare. Chronic ailments such as cardiovascular diseases, hypertension, and diabetes affect a significant number of the western population [1]. In particular, if we consider our aging society, the amount of elderly people suffering from one or more chronic diseases is increasing. Telemonitoring (TM) applications enable healthcare institutions to take care of their patients while they are out of hospital, which is especially useful for managing various chronic diseases as well as for measuring the effects of treatments under real-life conditions.
1.2. Example Scenario The following fictitious monitoring scenario will serve as illustration of the examples used throughout the course of this paper and as a brief summary of our vision for the future: Fred, aged 63, has already been suffering from congestive heart failure (CHF) for 1.5 years. CHF is a disorder causing the heart to lose its ability to pump sufficient blood through the body. Additionally, Fred has chronically high blood pressure which is related to CHF. Due to these facts, Fred has a high risk of heart failure. Fred’s treatment includes many emergency department visits and hospitalizations, which have physical and emotional impact. In this scenario, disease management can only be improved by reduced time between physician visits. As a vision for the future, Fred’s caregiver will decide to equip him with a wearable health monitoring system consisting of a smart shirt [9], a ring sensor [2], and a PDA for computation and wireless communication. This setup will allow for unobtrusive monitoring of ECG, heart rate,
blood pressure, oxygen saturation as well as motion activities, sensed with an inbuilt accelerometer. Fred’s PDA will wirelessly communicate with his home automation system in order to exchange data. The system will also aggregate additional measurements (such as location, activities, body mass) taken by Fred’s smart home environment. The different sensors will be integrated in a telemonitoring infrastructure, that will analyze the data accumulated, extract and forward relevant information to the care provider in charge. The infrastructure will provide a flexible platform for different kinds of monitoring applications and will guarantee a high level of reliability. Supported with this infrastructure Fred will feel safe to live independent at home. In case of changes of his health condition, his physician in charge will be automatically informed, and is able to retrieve all medical important data. Also, the infrastructure will detect critical events, which require immediate intervention (e.g., heart attack, fall, or unconsciousness), and will invoke reliable processes for calling the emergency service. As a consequence, Fred’s disease will be better managed with less hospitalization and more quality of life.
1.3. Issues in Health Monitoring Applications The following challenges are relevant to monitoring in healthcare but not restricted to this field of applications. Applications for traffic management, financial tickers, or object tracking have similar challenges on data management. Nevertheless, healthcare applications in contrast have to meet a highly ambitious combination of challenges. The large number of sensors, components, devices, information systems and platforms connected by different network technologies forms a complex distributed system with increased failure probability. Furthermore, existing systems and components are well in place and need to be incorporated into new applications. Reliability and provable correctness are new challenges that are of utmost importance in TM applications, where failures may have perilous consequences. Healthcare applications require a high degree of flexibility because of the vast amount of different diseases and the peculiarity of every individual. Usability supports flexibility by allowing medical specialists to adapt the healthcare application to their personal needs. Adaptability supports the collaboration of heterogeneous systems, components, and devices. Adaptability also supports mobility and intermitted connectivity, where disconnected parts of system are still able to operate locally in peer-to-peer style (e.g., when Fred is leaving the house data of his body sensors has to be processed and stored on his wearable PDA until Fred returns). As the example scenario shows, a core task of monitoring applications in healthcare is to coordinate and process various distributed data flows. These data flows are generated by different sensors. Sensor data has to be propa-
gated, combined, processed, and stored in a complex distributed system. This kind of applications will greatly benefit from DSM. DSM is an important part of healthcare applications, but also process management is needed to handle user defined workflows. Based on results generated by a DSM system, processes have to be executed (e.g., if Fred’s vital signs are critical, the physician in charge has to be informed and the event has to be stored in his patient record). Also healthcare processes are able to influence DSM (e.g., new guidelines for CHF treatment must immediately lead to an update of the way Fred’s sensory data is processed). Extracting important medical information out of continuous sensory data is a complex task, which generally involves multiple processing steps and combination of different data sources (like different kinds of sensors, patient records, laboratory results, etc.).
1.4. Infrastructure for Health Monitoring All these challenges necessitate an infrastructure that combines the processing of data streams and process management, i.e., the possibility to combine services and to execute composite services in a reliable way. Standards like SOAP, WSDL, and UDDI allow for service description, discovery, and invocation and support the proliferation of web services. Hyperdatabases [11, 12] address the composition user-defined processes over existing services and provide an infrastructure for reliable process execution. Characteristic features of hyperdatabases are i.) the possibility to add transactional guarantees to the execution of processes and ii.) the peer-to-peer execution of processes, without need for global control. Transactional guarantees support reliable execution with provable correctness. Peer-topeer process execution supports high reliability and availability, decentralized process execution in network areas of intermitted connectivity (e.g. personal area network between body sensors and PDA), and high scalability. Hyperdatabases and processes rely on the invocation of discrete services in request-response style. Continuous invocation of services as it would be needed by DSM is not supported yet. In this paper, we propose the extension of hyperdatabases for executing continuously running processes as needed in DSM. In general, we think of stream processes as sets of activities over discrete services as in conventional process management, and continuously running operators as needed for DSM. The strength of our proposed solution is integrated process and data stream management in peer-to-peer style. This enables new applications in healthcare and addresses scenarios like the one presented in section 1.2. We will show in the course of the paper that our new hyperdatabase infrastructure in general jointly addresses the two different process types (conventional processes and stream processes) and will support
in particular the special requirements imposed by healthcare applications. Flexibility enables execution of different kinds of processes over different kinds of data sources (both static and streams), to face the vast variety of medical problems (e.g., Fred might suffer from diabetes in the future and therefore also new types of data sources have to be integrated then). Also, our extended hyperdatabase infrastructure features high reliability characteristics in order to guarantee that information gathered from Fred’s sensors is appropriately and timely processed. Outline of this Paper: In section 2, we present our system model. In section 3 we introduce the concept of hyperdatabase systems and present details of the implementation of a concrete hyperdatabase prototype we used as starting point of our new infrastructure. Section 4 describes our extensions of hyperdatabase systems for data streams. Section 5 surveys related work and Section 6 concludes.
2. System Model 2.1. Data Streams A data stream is defined as a continuous transmission of data elements. Each data element, also called tuple or record, is a collection of data items. Data items may be of different data types. Data elements have a discrete time, which is realized by designated data items containing sequence numbers (stream relative). Additional time stamps have the advantage to allow for time correlation between data streams. These sequence numbers allow for recognition of missing data elements and correct ordering in a stream. In a data stream processing environment (also called data stream management system, short DSMS) each data stream is identified by a unique stream ID.
2.2. Operators Operators are the processing units of DSM and shown in Figure 1. Operators process incoming data streams and feed the results into outgoing data streams. Sensors are the primary sources of data streams (e.g., Fred’s ECG sensor). Sensors can be considered as operators without incoming data streams. Data stream processing is done by combining operators. An important constraint is that an operator is compatible to the type of incoming data elements (e.g., an operator for temperature data is not able to deal with acceleration data). Outgoing streams are incoming streams for one ore more following operators. At the end of a processing branch, there is a final operator, which transfers the results of stream processing to external systems or users, also called outside world. A specialized operator for transfer tasks is a web service operator (WSO), which encapsu-
Internal State Backup
Internal State Management
Internal State Input Streams Processing Unit
Output Streams
DSM Operator
Outside World
Figure 1. DSM Operator
lates a discrete web service for use in DSM. The WSO converts incoming data elements into request messages and invokes the web service. Response messages may be reconverted into data elements for outgoing data streams. The main difference between a data stream and a set of service invocations is the sequence in time, therefore data elements of response messages derive their sequence number from data elements providing the corresponding request messages. Due to the nature of stream processing, data is continuously generated by sensors, which usually are physically separated from computation and storage devices. Therefore, distribution of stream processing over multiple nodes is necessary. A node is a hardware unit which provides resources like computation, storage, transmission, sensors, or user interface. Operator classes implement different semantics of stream processing (e.g., operator class A realizes noise reduction on ECG sensor readings.). One operator class may have multiple operator types, which all have the same processing semantics, but different device requirements (processor load, memory consumption, power consumption, network bandwidth, etc.) and also different quality of service (precision, delay, etc.). For example there may be two noise reduction operator types for Fred’s ECG data, one with high precision and demanding device requirements and another with less precision for mobile devices, that can also be used in case of heavy system load to increase stream throughput. The precision degradation is limited by quality of service definitions. Operator instances are running instances of an operator type on a particular hardware. In the remainder of the paper, operator is the short notation for operator instance. A node hosting an operator instance is also called provider. Usually, the processing of a data stream subsumes an internal state in the operator. This internal state is information about recent processing results, which is needed for future processing but not directly available in the incoming data elements (e.g., the average of Fred’s blood pressure in the last hour). If it is needed to migrate the operator execution to another provider, incoming data streams have to be redirected and the internal state has
2.4. Interfaces to the Outside World Fred‘s PDA
Fred‘s PC
ECG Aquisition
ECG Variability
Blood Pressure Aquisition
Blood Pressure Variability
Caregiver‘s PC
Analysis
Critical Detection
Invocation of Alarm Processes
Figure 2. A Stream Process to Monitor Fred also to be moved to the new provider. At the new provider, the internal state is needed to initialize the newly created operator instance. Operators offer an internal state management for doing backups and initialization of internal states. Internal states are only valid for operator instances of the same type.
2.3. Stream Processes A stream process is a well defined set of logically linked operators continuously processing the selected input data streams, thereby producing results and having side effects. Side effects are effects on external systems imposed by processing results (e.g., the times when Fred’s blood pressure exceeds a certain threshold are stored in his patient record). Outgoing data streams can be used as input streams for other stream processes, or sent to the outside world by a special operator. Stream processes are also identified by globally unique process IDs. Operators of a stream process have process-unique operator IDs. Continuous queries are special stream processes without producing side effects. Figure 2 shows a stream process monitoring our patient Fred. Sensors are gathering Fred’s heart activity, his ECG, and his blood pressure. The sensors are wirelessly connected to Fred’s PDA, which is hosting the operators for sensor data acquisition. The PDA has also a wireless connection to Fred’s PC, which hosts operators for variability processing on ECG and blood pressure. The ”Critical Detection” operator is responsible for matching Fred’s vital signs to alarm thresholds and invocation of appropriate processes in case of dangerous conditions (e.g., calling the ambulance). The caregiver’s server is hosting an analysis operator, which is appropriate to Fred’s CHF disease. Solid arrows are data streams between the operators and dotted arrows characterize external interaction. We consider stream processes as continuously running distributed processes whereas their operators are similar to activities from a process management point of view. But in contrast to activities operator instances, the former processes are running continuously.
Operators may offer interfaces to the outside world (see Fig. 1). WSOs realize encapsulation of external web services. Therefore, operators can also issue services that correspond to transactions on databases, send e-mails, etc. Generally, operators may start user-defined processes for result delivery. Transactions for permanent storage of processing results or retrieving static data needed for combination with data stream elements are examples for interaction. The outside world can also invoke services on operators. Services in this context may offer actions like changing operator parameters or entering data elements. In general, service invocation in both directions can be realized by web service standards.
3. Hyperdatabases We propose an integrated information management infrastructure supporting user-defined processes, both conventional and streaming processes as needed for DSM applications. Stream processes are composed of operators (e.g., medical sensors, processing tasks, WSOs, etc.) whereas conventional processes are based on discrete services such as data processing services, data storage services, or event notification services, to mention only a few. Common in both cases is that operators or services, respectively, have to be combined in a given order (control flow) and that data has to be passed between them (data flow). Moreover, the two are tightly connected to each other since processes can be invoked by operators (e.g., an operator of Fred’s stream process detects a heart attack), where also reliability becomes a life-saving requirement. Also processes can affect stream processes (e.g., Fred’s physician is setting up new threshold for blood pressure events within the regular treatment process). In all these cases, an infrastructure is needed that supports processes, that provides dedicated execution guarantees for processes, and that is able to scale with the number of processes and services or operators. Because of this strong relation between stream processes for DSM and processes for workflow management, we propose the extension of hyperdatabase HDB systems [11, 12] which already provide a sophisticated infrastructure for conventional processes supporting (web) service compositions. In what follows, we briefly introduce the HDB concept and present a prototype HDB implementation.
3.1. HDB Concept for Process Execution In the last years, the prerequisites for application development have changed dramatically. Usually, specialized ap-
plication systems or components are well in place. Applications are no longer built from scratch but rather by combining services of existing components into processes or workflows. These components implement their own data management and provide application services to the outside. Hence, similar to supporting access to shared data, current infrastructures need to provide access to shared services. HDB systems provide such an infrastructure that supports the execution of processes on top of distributed components using existing services. In contrast to traditional process support systems, hyperdatabases execute processes in a peer-to-peer style based on the locally replicated metadata, yet without contacting any central instance. With this peer-to-peer process execution, a component works off its part of a process and then directly migrates the instance data to nodes offering a suitable service for one of the next steps in the process. Sophisticated routing strategies allow for dynamically choosing among the available components and balancing the overall load among them. Hence, the infrastructure is able to scale well with the number of processes. Moreover, the HDB provides transactional semantics for processes [13]. By this concept, the execution of processes can support dedicated transactional guarantees, even in case of failures and concurrent access to shared services. OSIRIS [18, 14] (Open Service Infrastructure for Reliable and Integrated process Support) is a prototype of a hyperdatabase, that has been developed at ETH Zurich. Due to its good support for reliable and scalable distributed applications, OSIRIS is a excellent base for our extensions towards an infrastructure for data stream processing.
3.2. OSIRIS Architecture Main emphasis of the HDB design was to avoid any central component for process navigation. Rather, the execution of a process instance involves only nodes that provide a service for that process, hence takes place in a peer-topeer style. Therefore, the HDB prototype OSIRIS consists of two parts. First, each node of the HDB network runs an HDB layer, that is responsible for the invocation of requested services, execution and routing of process instances, and failure handling. The HDB layer enables independent execution of processes, exchanging and processing messages, which are stored in persistent queues. Each service provider requires global meta information (e.g., about available service providers and their load). This global meta information leads to the second part of the HDB architecture, global repositories for meta information maintenance. Each HDB layer only contains replicas of locally needed pieces of global meta information.
3.3. Replication Management Meta data replication is based on a hierarchical organization with central repositories and clients replicating from them. The local replication manager, which is part of the HDB layer, is keeping local replicas consistent. The following three constraints on meta information explain, how replication can be made affordable. First, updates on certain information (e.g., process definitions) are infrequent. Second, nodes only require parts of the global information (e.g., only about pieces of processes that the provider is able to host). Lastly, changes on global meta information are not always critical (e.g., small changes on load information). Replication management is based on publish/subscribe techniques. The primary copy resides at the global repository. Each client that needs replicas has to subscribe for the selected information. As a result of this subscription, the repository publishes the current information. Whenever the primary copy changes, updates are published to all subscribed clients.
4. Hyperdatabases for Data Streams In this section, we describe our proposed infrastructure for DSM, which is currently being implemented on top of the existing HDB infrastructure OSIRIS. The goal is to support integrated stream processing and process management in order to achieve flexibility and reliability for monitoring applications in healthcare. We consider stream processes in a similar way as processes with respect to important aspects like distributed execution, load balancing, or fault tolerance. As for processes, we need for DSM to distribute meta information about stream processes. For this task the replication management of OSIRIS can be exploited. Replication management is extended by additional repositories, holding necessary information to establish and run DSM (like information about stream process definitions, providers of operators, operator load, backup information, etc.). Also the HDB layer has to provide additional functionality, like transport of data streams, activation and routing of stream processes, execution of operators, load and failure handling for stream processes, to mention the most important. Subsequently, we describe the basic architecture for a DSM enabled hyperdatabase system.
4.1. Global Repositories Global HDB repositories for DSM are the basis of the new platform for the execution of continuously running queries over data streams in a peer-to-peer fashion. Process Repository holds the global definitions of all processes, both traditional and streaming. Stream process definitions are decomposed into pairs of subsequent opera-
PID 1 1 1 1 1 1
PO Type ECG Acquisition Blood Pressure Acquisition ECG Variability Blood Pressure Variability ECG Variability Blood Pressure Variability
POID 1 3 2 4 2 4
CO Type ECG Variability Blood Pressure Variability Analysis Analysis Critical Detection Critical Detection
COID 2 4 5 5 6 6
R X X X
A A B
Process Repository
X
PID .. process ID, POID .. producing operator ID COID .. consuming operator ID, R .. routing flag
...
E
...
F
P
R
...
...
G
...
...
Process Join Repository HDB Layers
...
A Fred‘s PDA
B Fred‘s PC
C
K
Operators & Services
tors, also transmission units (TU) similar as for process definitions where pairs of subsequent activities are called execution units. TUs are tuples containing process ID, producing operator with type and ID, and consuming operator with type and ID. TUs are identified by a TUID and published to all providers that are able to host the producing operator. Additionally, TUs contain routing flags indicating if the producing operator provider is responsible for routing the consuming operator. Table 1 shows all TUs of Fred’s stream process according to Figure 2. Since Fred’s PDA is able to host the operator types ”ECG Acquisition” and ”Blood Pressure Acquisition” the TUs 1 and 2 are locally replicated on this node. Subscription Repository records a list of all available services and operator types, offered by all providers in the HDB network. Providers hosting operators only need to know providers offering subsequent operators of the corresponding stream process. Operator Backup Repository is storing information on how to retrieve a backup of the internal state of an operator. This information is published to providers that are potential destinations of an operator migration. Join Repository holds current providers of all running operators with more than one input stream, also known as joins. Since preceding operators may be running on different providers, a designated provider is needed to make the routing decision for the join such that the different streams actually arrive at the same provider for being joined. The designated provider is selected by the routing flag of the corresponding TU. After routing, the decision has to be published to the other providers via the join repository. A repository entry contains process ID, operator ID, and actual provider address. As defined by the hyperdatabase concept, these repositories do not require a centralized implementation. If the number of nodes in the HDB-network is increasing, additional repositories of same kind can reduce the load of repositories (i.e. by applying horizontal partitioning). Figure 3 illustrates the concept of a hyperdatabase architecture for data stream management. Different operator types are characterized by different shapes (e.g., star, triangle, etc.). This example describes the distributed execution of Fred’s stream process (see Figure 2). Where Fred’s PDA serves streaming data information of two body sensors, ECG (pentagon), and blood pressure (star). The sen-
...
H
...
C
Operator Backup Repository
Table 1. TUs of Fred’s Stream Process
Subscription Repository
Load Repository
E
C
HDB Repositories
TUID 1 2 3 4 5 6
Caregiver‘s PC
Figure 3. HDB Architecture for DSM sory information is pre-processed at Fred’s home PC (rectangle and triangle). From there the data streams are transmitted to the caregiver for final processing (parallelogram). Additionally critical conditions are detected via the hexagon operator running on Fred’s PC.
4.2. HDB-Layer Each service provider has a HDB layer running locally for meta data replication and management, operator invocation, etc. For reliably dealing with data streams, this layer has to be extended by the following additional modules. Figure 4 illustrates the HDB layer modules necessary for data stream management. Stream Communication (SC) realizes the exchange of communication messages between service providers containing one or more data stream elements. This module can leverage network dependent features to improve transmission of data streams (e.g., produce optimal message sizes by combining or splitting messages). Stream Transport (ST) implements a reliable data stream connection between two service providers based on the functionality provided by SC. ST handles the in- and output queues for data stream elements, enforces the correct order of data stream elements based on their sequence numbers, requests missing data elements from senders, and eliminates duplicates. In case of loss on transmission or failure of output providers, elements are re-sent. These issues are realized by acknowledgement protocols between sender and receiver. Additionally, ST checks as by-product whether all receivers are alive. Operator Management (OM) connects the incoming and outgoing data streams with the corresponding local operators, establishes internal connections between local operators. Internal connections are needed if more operators of the same stream process are running on the same provider. Internal connections are established directly without need of ST and SC. Necessary routing information is provided
Global Repositories Network Layer HDB Layer
Replication Manager
Continuous Query Manager Operator Management
Operators
Stream Transport Stream Communication
Output Streams Input Streams
External Connection
Figure 4. HDB Layer for DSM by the Stream Process Manager. OM also takes snapshots of internal states of local operators, which are necessary for migrating operator execution to other service providers in case of high load or failure. Stream Process Manager (SPM) is responsible for activation and deactivation of operators, migration of operators in case of high load or failure, routing of data streams by selecting the provider of a subsequent operator, doing backups of internal state snapshots, and invocation of failure handling processes.
4.3. HDB-Layer Tasks for DSM Operator Activation: Whenever operator execution is needed to be activated in the HDB layer of a suitable provider, an activation message is sent. This activation messages contains process ID and operator ID. SPM requests the corresponding query definition and operator backup information from the replication manager. If this operator is executed the first time ever, operator backup information is empty and no internal state initialization is needed, otherwise the new operator is initialized with the operator backup. Stream Routing: SPM controls the stream routing locally in peer-to-peer fashion. For each outgoing data stream, suitable service providers for subsequent operators need to be selected. Therefore, SPM retrieves the corresponding subscription and join information from the local replication manager. If a subsequent operator has only a single input, this decision can be made locally depending on distributed load information and an activation message is sent to the chosen provider. In case of a subsequent operator with multiple inputs (a join), only one preceding provider, the designated provider, is responsible for making the routing decision. The designated provider is selected by the routing flag of the corresponding TU. The routing decision is propagated to other input providers by updating the join repository via the local replication manager. Providers, where the routing flag in the corresponding TU is not set, have to wait for an update event on their local replicas. As long as no de-
cision is made, transmission is stopped and data elements are stored in output queues. Internal State Backup: SPM is also responsible for triggering internal state snapshots provided by OM, and backup of these snapshots. The backup destination is registered in the operator backup repository via the replication manager. An internal state contains internal state information, sequence numbers of the last consumed stream elements, sequence numbers of the last produced stream elements, and provider addresses of outgoing streams. Recovery of failed operators can be realized by moving internal states and all input data elements after corresponding input sequence numbers to new service providers, which are able to produce correct outgoing data elements after the corresponding output sequence numbers. An important constraint for snapshots of internal states in queries where subsequent operators run on the same provider is to guarantee that snapshots of internally connected operators fit together such that no data elements are missing or duplicated. Assume two subsequent operators. Then, the last output sequence number of the internal state snapshot of the first operator has to be the last input sequence number of the internal state snapshot of the second operator. If the internal state snapshot of the second operator is later in time, data elements would be missing after recovery. Otherwise, if the internal state snapshot of the second operator is to early, data elements would be duplicated after recovery. This constraint is important because internal connections bypass ST, therefore tuples on the fly over these connections are not saved in output queues. Adaptive interval duration between to internal state snapshots is considered as topic for further research. Output Queue Trimming: After a backup of an internal state snapshot, input elements before snapshot time are no longer needed for recovery or migration of this operator. OM calls ST to inform the corresponding sender that this provider no longer depends on these elements. If ST of the corresponding sender receives such acknowledgements from all consuming providers, the referenced data elements in its output queue are subject for discarding. Migration, Failure and Recovery: In case of high load, SPM can offload operators by moving operator execution to another suitable provider. In case of a node failure, upstream SPM detect it (via ST), and wait for recovery. If there is no recovery within a given time, the designated input SPM has to choose a suitable new service provider. Non-designated input SPM also detect the failure, but need to wait for the update of join information from the replication manager. The new service provider receives an activation message and loads the internal state for operator initialization from the backup, found via operator backup repository. Using additional routing information provided by the internal state backup, the old output provider can be served
again. In case no alternative provider is found, SPM can invoke user defined processes, both alternative stream processes or processes, for failure handling.
DSM Example Revisited: For illustration, we describe how Fred’s monitoring applications is using these tasks. Fred’s physician has designed the stream process (according to Fig. 2) and has also adjusted Fred’s operators to his individual parameters (e.g., thresholds for the critical detection). After applying the necessary body sensors to Fred, the stream process is invoked, by sending activation messages for the sensor operators to Fred’s PDA. Beginning from there, all subsequent operators (on Fred’s PC and caregiver’s PC) are activated in peer-to-peer style. For now, all operators are up and continuously processing data streams. In case of critical conditions (e.g., Fred has a heart attack), his physician has designed appropriate processes (e.g., calling the emergency service) which are invoked by the local HDB layer if necessary. Snapshots of internal operator states are saved regularly on an operator backup device for recovery (e.g., Fred’s PC for operators running on Fred’s PDA and vice versa). If Fred’s PDA fails (e.g. due to empty batteries), operators running on Fred’s PDA are migrated to Fred’s PC and are initialized with the recently saved internal states. Now, Fred’s PC is receiving data directly from the wireless body sensors, because it hosts the acquisition operators. Fred is required to stay near his PC, due to limited transmission range of the body sensors. When Fred is out of range, the HDB layer on Fred’s PC is able to invoke processes to deal with this situation (e.g., inform Fred to come back in range or stop the streaming process gracefully). Because of our peer-to-peer approach processes may be executed off-line without central control. As shown in this section, our proposed hyperdatabase system will incorporate DSM into process management in peer-to-peer style. Our solution offers great flexibility for healthcare applications. Medical users are able to define individual stream processes for monitoring different patients. Additionally, processes for failure handling, result delivery, critical event handling are supported. Our system is extendable to new applications or diseases by adding new operator elements (e.g. an additional glucose sensor if Fred will suffer from diabetes in the future). Also reliability at the systems level is provided. Whenever a node fails, this is handled by node recovery or, if not possible, by operator migration. In case operator migration is also not available, user defined failure processes (both streaming and traditional) may be invoked. In contrast, processes can affect stream processes. For example, a process called ”Fred leaves the house” can stop DSM gracefully. In general, this combination permits more complex applications, where process and data stream management work seamlessly together.
5. Related Work There are many existing projects regarding DSM. Early work in this field, e.g., [15, 17], suggests novel operations on streams similar to operations in a DBMS. Projects like NiagaraCQ [6], STREAM [3], and COUGAR [19] have addressed various aspects of DSM, like approximate results, query languages, query optimization, and sensor networks. In what follows, we briefly introduce main DSM projects, focusing on distribution and reliability aspects. Aurora [4] allows for user defined query processing by placing and connecting operators in a query plan. Queries are based on a set of well-defined operators. QoS definitions specify performance requirements. Aurora is a single node architecture, where a centralized scheduler determines which operator to run. Extensions like Aurora* and Medusa [7] also address DSM in distributed environments. Aurora* is a connection of multiple Aurora nodes to one query network. If a node has high load, it can offload operators to other nodes to reduce its load and improve the overall QoS. Upstream providers are saving output queues to allow for operator migration, which allows for high availability. TelegraphCQ [5] is a DSM project with special focus on adaptive query processing. Fjords allow for inter-module communication between an extensible set of operators enabling static and streaming data sources. Eddies describe adaptive query processing. Sets of operators are connected to the Eddy, and Eddy is choosing individually for each tuple where to go. Flux [16] provides load balancing and fault tolerance by providing adaptive partitioning of operator execution over multiple network nodes. This is realized by placing Flux between producer consumer pairs. In contrast to our work, load balancing is controlled centrally. PeerCQ [8] offers a decentralized peer-to-peer approach supporting continual queries running in a network of peers. The work is focused on capability-sensitive service partitioning, which assigns a monolithic continual query to a appropriate peer, rather than distributed query execution. DFuse [10] is a framework for distributed data fusion. DFuse also allows for distributed query processing by connecting operators. DFuse offers a data fusion API, which supports the development of data fusion applications. DFuse puts main emphasis on distributed power-aware placement of operators across nodes in wireless ad hoc sensor network, where routing has also to be considered. Compared to other projects in this field, our infrastructure offers two unique characteristics. Firstly, dynamic peerto-peer process execution where local execution is possible without centralized control. Secondly, the combination of DSM and transactional process management enables sophisticated failure handling. Not only recovery of providers or operator migration due to redundancy of providers is
possible, our infrastructure also offers customized failure handling, defined by the application designer (e.g., definitions of alternative stream processes or invocation of transactional processes for failure handling). In addition, our infrastructure is designed for arbitrary complex operators, as they can be found in healthcare applications, e.g., specialized processing of ECG data with side effects (e.g., store critical events in the patient record, or call emergency service). Therefore, it is only in special cases possible to partition an output stream across multiple operators or to apply query optimization techniques as described in various other DSM projects.
6. Conclusion The proliferation of ubiquitous computing and the huge amount of existing information systems is leading towards a world where sophisticated information management based on web service standards is becoming a crucial requirement. DSM describes information management for streaming data which is realized by pipelined operators. Process management offers a platform for processes which are a set of logically linked activities. DSM and process management are very similar in many respects and can benefit from each other. Aspects like distributed execution, distribution of meta information, load balancing, fault tolerance are examples of issues in both. These two paradigms are also closely related at the application level. Applications often need continuous monitoring of exceptional conditions or tendencies (e.g., whether Fred’s blood pressure curve is increasing), applicable processes have to be invoked to face this trend (e.g., tell Fred to rest and calm because of his high blood pressure). Therefore, our stating point is the observation that an information management infrastructure shall provide both process and data stream management. Security and privacy are other very important aspects but outside the scope of this paper. In this paper, we have proposed an infrastructure that jointly supports data stream and process management, based on the hyperdatabase vision and our running HDB prototype OSIRIS. Our solution puts main emphasis on providing flexibility and reliability for monitoring applications in healthcare. In contrast to other projects in this field, our concept allows for distributed process execution and DSM working seamlessly together without the need of central components, arbitrary complex operators with side effects, and sophisticated failure handling. The implementation of the extended OSIRIS infrastructure is still under development and first experimental results will follow in future work. Future research will also address the problem of adaptability controlled by QoS definitions and intermitted connectivity of mobile devices.
References [1] American Heart Association. 2002 Heart and Stroke Statistical Update, 2001. [2] H. Asada et al. Mobile Monitoring with Wearable Photoplethymographic Biosensors. IEEE EMB Magazine, 22(3):28–40, 2003. [3] B. Babcock, S. Babu, M. Datar, R. Motwani, and J. Widom. Models and Issues in Data Stream Systems. In Proc. of PODS Conf., pages 1–16, Madison, WI, USA, 2002. [4] D. Carney et al. Monitoring Streams - A New Class of Data Management Applications. In Proc. of VLDB Conf., pages 215–226, Hong Kong, China, 2002. [5] S. Chandrasekaran et al. TelegraphCQ: Continuous Dataflow Processing for an Uncertain World. In Proc. of CIDR Conf., Asilomar, CA, USA, 2003. [6] J. Chen, D. DeWitt, F. Tian, and Y. Wang. NiagaraCQ: A Scalable Continuous Query System for Internet Databases. In Proc. of SIGMOD Conf., pages 379–390, Dallas, TX, USA, 2000. [7] M. Cherniack et al. Scalable Distributed Stream Processing. In Proc. of CIDR Conf., Asilomar, CA, USA, 2003. [8] B. Gedik and L. Liu. PeerCQ:A Decentralized and SelfConfiguring Peer-to-Peer Information Monitoring System. In Proc. of Distributed Computing Systems Conf., pages 490–499, Providence, RI, USA, 2003. [9] The Georgia Tech Wearable Motherboard: The Intelligent Garment for the 21st Century. http://www. smartshirt.gatech.edu. [10] R. Kumar et al. DFuse: a framework for distributed data fusion. In Proc. of SensSys Conf., pages 114–125, Los Angeles, CA, USA, 2003. [11] H.-J. Schek et al. Hyperdatabases. In Proc. of WISE Conf., pages 28–40, Hong Kong, China, 2000. [12] H.-J. Schek, H. Schuldt, C. Schuler, and R. Weber. Infrastructure for Information Spaces. In Proc. of ADBIS Conf., pages 22–36, Bratislava, Slovakia, 2002. [13] H. Schuldt, G. Alonso, C. Beeri, and H.-J. Schek. Atomicity and Isolation for Transactional Processes. ACM Transactions on Database Systems, 27(1):63–116, 2002. [14] C. Schuler, R. Weber, H. Schuldt, and H.-J. Schek. Peer– to–Peer Process Execution with OSIRIS. In Proc. of ICSOC Conf., pages 483–498, Trento, Italy, 2003. [15] P. Seshadri, M. Livny, and R. Ramakrishnan. The Design and Implementation of a Sequence Database System. In Proc. of VLDB Conf., pages 99–110, Mumbai, India, 1996. [16] M. Shah, J. Hellerstein, S. Chandrasekaran, and M. Franklin. Flux: An adaptive partitioning operator for continuous query systems. In Proc. of ICDE Conf., Bangalore, India, 2003. [17] M. Sullivan and A. Heybey. Tribeca: A System for Managing Large Databases of Network Traffic. In Proc. of USENIX Conf., New Orleans, LA, USA, 1998. [18] R. Weber, C. Schuler, P. Neukomm, H. Schuldt, and H.-J. Schek. Web Service Composition with OGrape and OSIRIS. In Proc. of VLDB Conf., Berlin, Germany, 2003. [19] Y. Yao and J. Gehrke. Query Processing for Sensor Networks. In Proc. of CIDR Conf., Asilomar, CA, USA, 2003.
