republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a ... for such a generic evolvement: (i) the growing availability of cheap .... tainty due to the dependency on several shipping companies, on traffic and ...... example, the logistics domain) there are existing, dedicated opti- mization ...
Towards Proactive Event-Driven Computing Yagil Engel
Opher Etzion IBM - Haifa Research Lab Haifa, Israel
{yagile,opher}@il.ibm.com
ABSTRACT Event driven architecture is a paradigm shift from traditional computing architectures which employ synchronous, request-response interactions. In this paper we introduce a conceptual architecture for what can be considered the next phase of that evolution: proactive event-driven computing. Proactivity refers to the ability to mitigate or eliminate undesired future events, or to identify and take advantage of future opportunities, by applying prediction and automated decision making technologies. We investigate an extension of the event processing conceptual model and architecture to rt proactive event-driven applications, and propose the main building blocks of a novel architecture. We first describe several extensions to the existing event processing functionality that is required to support proactivity; next, we extend the event processing agent model to include two more type of agents: predictive agents that may derive future uncertain events based on prediction models, and proactive agents that compute the best proactive action that should be taken. Those building blocks are demonstrated through a comprehensive scenario that deals with proactive decision making, ensuring timely delivery of critical material for a production plant.
Figure 1: Components of Event-Driven Architecture.
Categories and Subject Descriptors H.1.0 [Information Systems]: Models and Principles:General
General Terms Design
Keywords Proactive, Event Processing
1.
INTRODUCTION
Event driven architectures and conceptual models that support them have evolved in the last several years, departing from the traditional computing architectures which employ synchronous, requestresponse interactions between client and servers. This is a paradigm shift in two senses: first, event driven architectures support applications that are reactive in nature, in which processing is triggered in
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. DEBS’11, July 11–15, 2011, New York, New York, USA. Copyright 2011 ACM 978-1-4503-0423-8/11/07 ...$10.00.
125
response to events, contrary to traditional responsive applications, in which processing is done in response to an explicit request. Second, event driven architecture adhere to the decoupling principle, in which there are event producers, event consumers and event processing agents that are mutually independent. Figure 1 shows an illustration of such architecture [7]. We introduce a conceptual architecture for what can be considered as the next phase of the evolution: proactive event-driven computing. Proactivity refers to the ability to mitigate or eliminate undesired future events, or to identify and take advantage of future opportunities, by applying prediction and automated decision making technologies. Consider the three examples in Figure 2. A responsive application is the one where a student is querying the Web in order to get material needed for writing a school assignment; many consumer and enterprise applications are of the same type (e.g. getting quote from insurance company, querying a data warehouse for a summary of sales in a certain segment). A reactive application detects a traffic jam, either by a single camera observation (raw event), or by summing up the number of vehicles that enter a road segment within a certain time interval (a complex event), and reacts by changing a street light schedule. The third is an example of a proactive application: a technician is delayed at a customer’s house, and because there is also a traffic jam, he is expected to miss the next scheduled customer he should visit; while this has not happened yet, we wish to eliminate this anticipated event, by rescheduling the planning of all the technician team. Proactive applications have been developed in an ad-hoc manner for several years; in particular in the IT infrastructure and management domain. Some examples include proactive security systems [5], proactive routing in mobile ad-hoc wireless networks [14], proactive network management with failure handling [8], proactive SLA negotiation in service oriented systems [17], and proactive caching [13]. Observing the evolution from responsive to reactive computing, we note that reactive applications have been developed for many years in ad-hoc hard-coded sporadic fashions; however, the major breakthrough that turned event-driven application perva-
match demands. The actions available to the power company correspond to increasing/decreasing production at various costs; for instance, the company can start up diesel generators which are approximately 10 times more expensive than the coal generators. In addition, the company can launch emergency fix to many or all of its coal generators; coal generators normally lose some of their production capacity over time, and are fixed according to a regular maintenance schedule. Finally, power companies can also affect consumption through particular pricing mechanisms with industrial consumers; the contract with these consumers defines the timing in which a price increase can be announced. The example above represents a typical enterprise operational control problem of resource management and allocation in face of uncertainty about resource availability and the demand for the enterprise’s services. Two kinds of actions are typically available to the system: actions that change resource-allocation policy, and actions that affect the availability of resources for a certain period of time. In many cases events affecting production and demand can be forecasted in advance; for example, consumption is affected by weather conditions. Production is also affected by particular conditions: for example, severe weather conditions may harm coal generators and power lines; sun and wind affect renewable power production. These weather conditions can in turn be predicted rather accurately using various measurements. The role of the proactive application in this scenario would be: (i) detect those early measurements and predictive factors using standard event processing means (e.g., detect an event pattern that usually precedes a generator failure), (ii) generate event prediction, and (iii) alter the policy given the specific prediction (e.g., emergency fix of all generators).
Figure 2: Evolution of computing paradigms. sive and part of the main-stream computing was the development of models and tools to express and execute reactive systems in an easy way. A similar breakthrough is required in order to enable pervasive use of proactive computing, instead of ad-hoc, multiple models “gluing” employed today. Several factors in today’s computing infrastructure open the door for such a generic evolvement: (i) the growing availability of cheap and pervasive sensor technology, (ii) the spreading of broadband connectivity, pretty much anywhere, and (iii) the developments in predictive analytics technology. The latter highlights a different angle to this process. Analytics has evolved from being merely descriptive (understanding of historical data), to being predictive (providing forecasts of future behavior). The next step is prescriptive analytics, a term which stands for the use of data to prescribe the best course of action to realize the best outcome [24]. We can view the proactive idea as the event-driven variation of prescriptive analytics; reactive computing, coupled with predictive analytics, yields the ability to react to events before they occur, which is the essence of proactive event-driven computing. After motivating the proactive event-driven paradigm using examples below, we introduce the conceptual model and architecture of proactive event-driven applications. In Section 2 we discuss the concepts and facilities of event driven architecture and event processing conceptual model, and provide essential background on the decision making models we employ; in Section 3 we overview our approach for proactive architecture, in Sections 4, 5, and 6 we drill down to explain the new architecture constructs: predictive agents and proactive agents, and in Section 7 we show a complete scenario modeled with the conceptual architecture we presented.
1.1
E XAMPLE 2. A chemical and petroleum (C&P) company operates various heavy machinery. The overall objective is to minimize maintenance costs while preventing damage to machines and ensuring maintenance is performed at convenient times. In recent years it becomes evident that following manufacturers’ maintenance schedules is overly conservative; these schedules are designed for a worst case scenario, whereas in practice preventive maintenance should depend on the actual condition of the machine, which in turn depends on its usage and operation environment. The alternative is a proactive maintenance approach: track the machine condition using various indicators, and if a change in a machine condition is predicted, adapt the maintenance policy. The role of the proactive application in this scenario is similar in nature to the previous example: (i) detect events that can predict a change in machinery condition, (ii) forecast the specific change of condition, and (iii) change maintenance policy accordingly. E XAMPLE 3. Logistics scenarios of long distance shipping usually require complex planning, and involves high degree of uncertainty due to the dependency on several shipping companies, on traffic and weather conditions, customs procedures, insurance, and others. These factors often require a change of plans during the scenario execution. Currently, many freight providers use various software solutions for plan optimization; however, when the need for plan change arises, the online nature of the decision often leads to highly sub-optimal results. A proactive system can greatly enhance the online decision making in this domain; we illustrate that with a modeling example in Section 7.
Example Scenarios
There are numerous examples for the applicability of proactive event-driven computing, ranging from instantaneous operational control, through financial markets, and to heavy industries. We begin with the following example scenarios, provided by IBM customers and business partners, in order to give a flavor of the type of problems we address. E XAMPLE 1 ([3]). A power company controls both the production of electric power, as well as its allocation to different geographic areas. This is a typical resource management setting: given a set of resources (power production in specific plants) and demands (coming from different geographic areas), the overall operational control problem is to manage and allocate resources to
2.
BACKGROUND AND RELATED WORK
We now provide background in several areas: conceptual modeling of event processing (EP), uncertainty in event processing, and models for optimal decision making over time.
126
2.1
The conceptual model of event processing
2.3
The conceptual model of proactive event-driven applications is built as an incremental part over the existing work in event processing architectures, thus we briefly survey the concepts we are using. At the heart of this model is the notion of event processing network (EPN); looking back at Figure 1, there is a layer of intermediary event processing that stands between the event producers and consumers. This layer is not monolithic, but rather consists of various types of processing elements called event processing agents (EPA), each performing single type of processing. The notion of event processing network has been defined by Luckham [15] and refined by Etzion and Niblett [7] as a collection of event processing agents, producers, and consumers connected by a collection of channels. A channel is a processing element that receives events from one or more sources and routes them to one or more sinks. Event processing agents are further refined into types, as shown in Figure 3.
Our vision for proactive computing relies heavily on the solution of dynamic and uncertain decision problems. We provide background on the model we consider the most appropriate in this setting; this background is essential for understanding Sections 3.2 and 6.1. A Markov Decision Process (MDP) is a quadruple hS, R, A, T i, where S is the state space describing the world, and R : S → < is the reward function, indicating a numeric value of being one time step in a given state. A is a set of actions, and T : S × A × S → [0, 1] is a transition function with T (s, α, s0 ) capturing the 0 probability of achieving Pstate s by applying action α in state s; for each (s, α) ∈ S × A, s0 ∈S T (s, a, s0 ) = 1. MDP models a sequence of decision points over time. A solution to an MDP is a policy π : S → A, indicating the optimal action to perform at each state. During runtime, it is expected that at each time point the system observes its current state s ∈ S and executes π(s). The criterion for optimality is measured by a value function vπ : S →
