A Knowledge Integration Framework for Complex Network ... - CiteSeerX

Final Submission of IMDS 4938

A Knowledge Integration Framework for Complex Network Management Xiangyang Li and Charu Chandra Department of Industrial and Manufacturing Systems Engineering, University of Michigan – Dearborn 4901 Evergreen Road Dearborn, Michigan 48128, USA Email: [email protected] and [email protected]

A Knowledge Integration Framework for Complex Network Management Xiangyang Li and Charu Chandra Research Paper Autobiographical Note Xiangyang Li is assistant professor and Charu Chandra is associate professor, both at the Department of Industrial and Manufacturing Systems Engineering, University of Michigan – Dearborn,

4901

Evergreen

Road,

Michigan

48128,

USA,

with

email

addresses

[email protected] and [email protected]. Purpose Large supply and computer networks contain heterogeneous information and correlation among their components, and are distributed across a large geographical region. This paper investigates and develops a generic knowledge integration framework that can handle the challenges posed in complex network management. It also examines this framework in various applications of essential management tasks in different infrastructures. Methodology/Approach Efficient information and knowledge integration technologies are key to capably handling complex networks. An adaptive fusion framework is proposed that takes advantage of dependency modelling, active configuration planning and scheduling, and quality assurance of knowledge integration. We use cases of supply network risk management and computer network attack correlation to elaborate the problem and describe various applications of this generic framework.

1

Findings Information and knowledge integration becomes increasingly important, enabled by technologies to collect and process data dynamically, and faces enormous challenges in handling escalating complexity. Representing these systems into an appropriate network model and integrating the knowledge in the model for decision-making, directed by information and complexity measures provides a promising approach. The preliminary results based on a Bayesian network model support the proposed framework. Originality Firstly, we discuss and define the challenges and requirements faced by knowledge integration in complex networks. Secondly, we propose a knowledge integration framework that systematically models various network structures and adaptively integrates knowledge, based on dependency modelling and information theory. Lastly, we use a conceptual Bayesian model to elaborate the application to supply chain risk management and computer network attack correlation of this promising framework. Keywords information integration; complex systems; supply chain; risk management; computer network security Introduction Evolving into complex networks, critical national infrastructures play essential roles in supporting the modern society and the globalized economy. These important facilities include computer networks, supply chain/network, power grid, financial networks, ad-hoc wireless networks, sensor networks, disaster surveillance and response systems, transportation infrastructures, healthcare systems, intelligence systems, social networks, among others. Great

2

demand exists to strengthen the quality and reliability of these infrastructures critical to national interests, in terms of various performance metrics. With new theories, latest computing technologies, and ever-growing computational capacities, novel solutions to describe such networks, understand the problems, and optimize their performance are constantly emerging. Recent developments in wired and wireless technology have greatly expedited the evolution of large networks and enhanced the information and knowledge processing in these systems (Varshney et al., 2000; Hughes and Love, 2004). New sensor and storage technologies such as RFID tagging technique (Brewer et al., 1999) and ad-hoc wireless networks largely reduce the information monitoring and collection time, and allow the central management module to gain live information updated continuously about virtually every component (Jin et al., 2004). Computer systems and infrastructures themselves have become increasingly complex in the form of large scale, multi-application, heterogeneous platforms, and special requirements on service type and quality. A computer network infrastructure as a whole has to manage and defend itself actively and confidently against malicious attacks, in a timely and accurate manner, just like a military force in a dangerous battlefield. In another example, imagine a large supply chain network that consists of thousands of business nodes. No matter whether they are located at headquarter, center, branch, department, or even individual employee and facility, these nodes are linked flexibly. Ideally they can at any instant exchange with each other, business knowledge about inventory, capacity, quality, prediction, failure, etc., and are impacted by management strategies of such knowledge (Hsu, 2005; Yu et al., 2001) Such a system can extend across geographical boundaries and run uninterruptedly, synchronized by dawn and sunset across continents.

3

These complex networks generate, contain and process business knowledge continuously. Business and network knowledge is captured in features and variables that represent the state of various nodes in this network, such as an error message generated at one network node, or repair orders at a customer service branch. It is not enough to apply deterministic solutions such as rule-based analysis of logging data in computer intrusion detection (Axelsson, 2001; Debar et al, 1999), or in traditional supply chain studies the mathematical models from operations research (Graves et al., 1998) and game theory (Shubik, 2002). Emerging network structures of modern networked enterprises raise challenges in terms of complexity and uncertainty. The information may not be as accurate as required by the above models, as well as that the limitation of supporting technology such as the communication bandwidth and computation cost of a wireless network or RFID nodes may not allow accurate and complete information generated and disseminated as requested, such as the bullwhip effect in supply chains (Lee et al., 1997; Metters 1997). All these networks are complex temporal-spatial systems where the synchronization of data over time (time series data) and the integration of distributed data (heterogeneous data sources and data structures) can be an enormous challenge. More significantly, these networks are often complex adaptive systems that existing models commonly are not able to handle very well. We have to build up a model that can comprehensively and accurately capture the knowledge in this complex system, such as in the exploratory study by (Ye et al., 2000; Choi et al., 2000). Decision-making based on heterogeneous and distributed information is one type of task of knowledge integration or information fusion. We must be able to efficiently integrate the knowledge represented in this model, which often has a network structure. Therefore, the objective of this paper is to investigate and develop a generic knowledge integration framework that can handle the challenges posed in emerging complex networks. We

4

also try to examine such a framework in various applications that are essential management tasks of different infrastructures. Our study makes several important contributions. Firstly, we discuss and define the challenges and requirements faced by knowledge integration in complex networks. Secondly, we propose a knowledge integration framework that systematically models various network structures and adaptively integrates knowledge. This adaptive framework is based on dependency modelling and information theory to support active configuration planning and scheduling, and quality assurance of knowledge integration. Lastly, we use a conceptual Bayesian model to elaborate the application to supply chain risk management and computer network attack correlation of this promising framework. Knowledge Integration in Complex Networks Knowledge integration and management face enormous challenges in complex enterprise networks. For a suitable knowledge integration strategy, we require a systematic approach that can handle the complexity and uncertainty in such large systems. This approach must have the ability to model and manage the reconfigurability of knowledge collection and integration structure in these systems. Finally, it must be able to accomplish quality assurance in achieving information integration for efficient decision-making. In this section we use two important management tasks in supply chain network and computer network to elaborate these challenges. Supply Chain Networks and Risk Management With the rapid expansion and growth of modern enterprise and market, traditional supply chain systems have evolved into large and complex systems with broader geographical reach and product variety. “Supply Network” has come to replace the supply chain in representing the network-based structure of the emerging supply-consumer paradigm. A modern supply network (SN) spans beyond the single layer of supplier. It can include many layers of suppliers,

5

production plants, distribution centers, and retailers across countries and continents, interlinked by information and logistics flows of hundreds or even thousands of products. These SN members are flexibly connected to each other according to different levels of responsibility and information sharing (Léger et al., 2006). To improve the performance of such a complex system and respond to emerging risks, information technologies such as enterprise resource planning software and large data centers are being deployed. A large supply network is a typical temporal-spatial system as shown in Figure 1. Risk management assesses the impact of risk and configures countermeasures to minimize loss due to the occurrence of risk. Risks are very common in manufacturing and logistics enterprises. Familiar risk events are natural disasters, such as earthquakes, hurricane and fires, and artificial causes, such as labour strike, transportation delay and terrorist attack. In the setting of an automotive enterprise a fire in a component plant can cause the engine and transmission plant to halt operations. The propagation of this event can finally prevent the entire SN from delivering a certain model of a car. Such risks even become routine in modern enterprises, especially with global operations. We still clearly remember the serious setback to world economy due to events such as Iraqi war, SARS breakout, and the financial crisis in Asia. In such events, we may observe the impact on local enterprises from a small town in the USA to a big city in China. These events can cause interruption to supply chain operations, soaring increase in production cost, and damage to stock share values, as discussed by Hendricks and Singhal (2003). Modern supply chain management will never be fulfilled in a risk-free environment.

Take in Figure 1

6

An essential task for risk management is to quantify the impact of events associated with risk and provide countermeasure in reactive or proactive mode in order to reduce their negative impact. This can be compared to the classic optimization studies that apply operations research methods. For example, we can find out the bottleneck from the impact analysis with the aid of Critical Path Analysis (CPA) or other algorithms and with the help of planning tools such as the CPFR (Fliedner, 2003). However, uncertainty and complexity may prevent us from deriving an efficient solving strategy since such a large and complex system induces inevitably a NP-hard problem. We cannot even guarantee a convergence resolution in this system. Here intelligent dissemination and integration of information is the key to success. However, the huge amount of knowledge is too much to be considered as a whole in response to a specific incident. Therefore, we have to dynamically determine the information boundary, and integrate the complete and relevant knowledge in a timely manner. Each demand and supply node in Figure 1 is a member of the automotive SN enterprise. Interaction between nodes encompasses the entire life cycle of the material or process or information or financial flow in the SN; hence the location of each incoming demand node must be configured at the conceptual design stage of the network as in supply chain management. The impact of design/configuration specifications in the SN (and any subsequent modifications to it) must be reflected at the 1) physical system level in the SN entities, 2) logical system level in the strategies and policies adopted in the problem-solving models, and 3) virtual system level in determining the configurations (reconfigurations) to meet the dynamic requirements in response to the change in environment where the SN resides. Supply chain management of an enterprise system deals with complex interactions among supply chain members and decision-making problems at the above-defined levels, viz., physical,

7

logical, and virtual levels of the enterprise system. Establishing a supply chain configuration (or reconfiguration) of existing supply chain is one of the principal supply chain management decisions. It has profound impact on other subsequent managerial decisions. The configuration defines the operating basis of SN, the structure and parameter as in SN configuration. Other operational decisions are made using the elaborated configuration as input. Configuration decisions ought to be subjected to particularly comprehensive evaluation, requiring utilization of integrated models and methodologies. The problem of configuration (reconfiguration) as researched so far has limitations in terms of considering primarily only the logical system level, partly the virtual system level, and physical system level only by implication to the two higher levels (Chandra et al., 2005; Chandra and Grabis, 2001, 2004; Suh, 1995). The proposed research described in this paper extends the current research to include all three-system levels in an integrated manner. Here, the supply chain configuration is utilized as an approach to “network enterprise” creation and reuse that considers enterprises as assemblies of reusable components (units) defined on a common domain knowledge model, such as a “product - process - resources” (PPR) model. The objective is to generate customized solutions based on standard components, such as templates, baselines, and models. The network enterprise configuration is represented by the following relationship: “Configuring product (sensor components) → configuring business process (process structure, operation types) → configuring resource (structure of system, equipment).” The implementation of this approach is based on the shareable information environment that supports the PPR-model used for integration and co-ordination of user’s activity (Giachetti et al., 1997; Hirsch, 1995). The PPR-model is based on the concept of ontology-oriented constraint networks (Royer, 1992).

8

Multi-ontology classes of entities, the logic of attributes and the constraint satisfaction problem model represent the SN structure. Computer Network Attack Correlation Management of computer networks, which support modern enterprises far beyond just supply chains, is one crucial task of national infrastructure protection. However, information warfare is full of uncertainty and complexity in several enabling technology areas to support robust communication with assured information security. One limitation is the intrusion detection and response technologies even in the more secure military systems. Intrusion detection and response research has come a long way since the seminal rule-based pattern-matching model by Denning (1987). A variety of intrusion detection techniques, exemplified by the classification of anomaly detection and misuse detection, have emerged such as in (Li and Ye, 2003, 2006; Ye et al., 2001). In distributed environments, network attack correlation (NAC) detects various types of attack and diagnoses about root cause and attacking trace, in order to direct dynamic configuration and efficient investment. There are plenty of distributed intrusion detection systems (DIDSs) that either claim to handle distributed attacks or are distributed over a computer network, as in (Staniford-Chen et al., 1998; Porras and Neumann, 1997; Kulkarni et al., 2001; Zhang et al., 2003). Existing techniques have significant limitations in terms of: 1) isolated studies in developing ad hoc, domains and/or environments specific data collection monitors and analysis engines. Deploying, configuring, and managing a large number of heterogeneous intrusion detection systems (IDSs) is still an expensive and error-prone activity; 2) lack of certainty and confidence in decision-making. They cannot tell us how serious is the overall picture of the system; and 3) incapability of dealing with complexity in large computer systems. They cannot provide a coordinated consistent view of the entire network to protect. Existing techniques cannot 9

efficiently capture the operational implication of network attacks such as root causes and attack paths over time. Over decades of evolvement, computer attacks take a range of forms of physical exploitation, malicious code, virus, worms, coordinated service requests, or cognitive and social engineering. A specific type of attack to consider is the distributed denial of service (DDoS) (Mirkovic and Reiher, 2004). A set of computers coordinate to send out large volume of malicious network traffic to the victim machine in order to compromise the availability of the normal service on this computer. Attacks of this kind are very difficult to detect and diagnose by any individual IDS. Information warfare has to rely on efficient knowledge fusion strategies across complex computing infrastructures, which are committed to active information collection, integration and analysis. Figure 2 illustrates a typical DIDS structure that is illustrated for later discussion on our proposed framework. A knowledge fusion process follows the “report-control” cycle occurring between the local IDS and the regional administrator, which is the central manager for this subnetwork. The local ID systems report their findings regarding intrusive or alertness levels about the location they reside at to the administrator. The feedback from the administrator controls further collection and detection tasks of the local ID systems until the suspicious hypothesis of this region is either rejected or confirmed.

Take in Figure 2

10

Dependency Model and Information and Utility Theory Two essential knowledge bases of dependency model and information/utility theory are employed in the proposed framework. Dependency comes from information flow, material flow, transportation/logistics, etc. among various network components. Information theory provides the metrics in defining the qualities of configuration and fusion in complex systems. Utility calculation produces criteria for comparing solutions in order to optimize the knowledge integration process. Dependency Network Models Dependency or causal network models use nodes (vertexes) to represent system variables and links (edges) to represent relationships and dependency among these variables (Glymour and Cooper, 1999). The weight associated with the dependence link represents the probability, confidence, impact, knowledge from business logic, etc. Dependency network models have many specific types, such as the probabilistic dependency model where probability is added to deal with uncertainty. It is also possible to extend dependency models into more general forms such as the Generalized Semi-Markov Processes models (Shedler, 1993), which can generate simulation analysis. In this paper, we use Bayesian networks as the tool in our approach to seek analytical solutions. Bayesian networks (BNs) are probabilistic graphical models representing joint probabilities of a set of random variables and their conditional independence relations (Jensen, 1996; Pearl, 1998). The nodes characterize the hypothesis/goal variables, other state variables, and evidence/observation variables in the physical system, while the arcs linking these nodes represent the causal dependency among these variables, represented in prior or conditional probability. The hypothesis nodes could be the network components that we are interested in or

11

that are hard to directly estimate. Evidence nodes are those components that yield information about its states. The state in each node can be discrete, indicating separate states of that component, or continuous, i.e. the value of that variable. Static Bayesian Networks (SBNs) work with evidence and belief from a single time instance. Dynamic Bayesian networks (DBNs) are made up of interconnected time slices of SBNs. The relationships between two neighbouring time slices are modelled by a Markov model, i.e., random variables at time t are affected by the corresponding random variables at time t-1 only, as well as by other variables at time t. DBNs represent a generalization of the conventional systems, such as Kalman filtering and Hidden Markov Models for modeling dynamic events.

Take in Figure 3

Bayesian networks have several advantages for modeling and inferring in dynamic and complex systems. Firstly, BNs provide a hierarchical framework to systematically represent information from different modalities and at different abstraction levels and systematically account for their uncertainties. Furthermore, with the dependencies coded in the graphical dependency model, Bayesian networks can handle situations where some data entries are missing. Secondly, the system’s dynamically changing state and the surrounding situations call for a framework that not only captures the beliefs of current events, but also predicts the evolution of future scenarios. DBNs provide a very powerful tool with a coherent and unified probabilistic framework for evidential information representation, integration, and inference over time.

12

Information and Utility Theory Information theory is developed for concepts relevant to information processing and transformation (Cover and Thomas, 1991; Mackay, 2003). In this study, we are interested in several technologies that try to quantify the uncertainty and complexity in large-scale systems reflected in decision-making. Of these, we will use the information entropy (Shannon, 1948), the definition of complexity (Zurek, 1990), and the utility theory (Mackay, 2003). Entropy and complexity are quantitative measures that characterize the level of certainty, consistency or simplicity. Based on subjective evaluation of different solutions and actions, utility-theoretic theory can provide the quantitative measures of tradeoff between benefits and costs to select alternative decisions. Such decisions can be about, where to search for changes in diagnosis or how good the alerts are in detection. In entropy, complexity or utility calculation, probabilistic theory enables handling the uncertainty and the temporal relationship in participating units. As in the statistical process control technique used in quality control and improvement (Farnum, 1994), we need to continuously monitor certain quality and disturbance scores in the profiling procedures over time and space, i.e. locally and globally. A Generic Adaptive Knowledge Integration Framework We consider the generic tasks essential to the adaptive knowledge fusion in risk assessment or network attack correlation to be: 1) correlation of individual evidence outcomes to provide the indication of root cause location and epidemic path, and to identify the reconfiguration alternatives, and 2) quality assurance to decide when to engage and stop information integration, what information and where to collect from (e.g. the portion of network, group of sensory components, and type of evidence), and how good the quality of the correlation outcome is.

13

In the proposed approach, we use a probabilistic dependency model to represent information flow among network nodes and reliability of evidence at individual nodes; we calculate utility scores within this dependency model to characterize the capability of sensory components, and to adapt the knowledge integration in a dynamic feedback control scheme; we exploit information entropy to determine the uncertainty/complexity of integration outcome and the change in network dynamics. Dependency Modeling As shown in Figure 4, a Bayesian network with hierarchical inter-correlation structure models the dependency and uncertainty in a distributed enterprise network, including but not limited to logistics topology, information/material flow, and quality and reliability of information collection in sensor components. Thus, the state at each node represents the associated risks or the different attacks. The belief or probability of each state shows the level of certainty. (1) In Figure 4(a) the links between two node (N and N’) layers and their associated probabilities describe the inter-connection among the physical nodes, with focus on the interdependency within the information flow among these nodes. The top and the second layers represent the input and output information queues at the node. The dependency between NA’ and NA corresponds to information generated at node A. The dependency of node B on A is defined as I(AB)/I(A), where I(A) and I(AB) are the traffic throughputs through A, and through B from A respectively. The dynamic behavior in short-term and long-term historical data is used in determining the structure and parameters. In Figure 4(b) the links between the SN node layers and their associated probabilities describe the inter-connection among the physical SN constructs, with focus on the interdependency within the information or material flow among these nodes. Here, we only show two layers although there can be many more layers.

14

(2) In Figure 4(a) the links between the node layer and the information (I) layer, and the associated probabilities describe the deployment of IDS components and their information collection capacity. Configuration of IDS components regarding the information boundary is part of the configuration decision. For example, the sampling frequency of an IDS component can change, and a suspicious node may be worth a close look. We could also remove or add IDS components. Thus the dependency structure between these two layers can dynamically change. In Figure 4(b) similarly the links between the SN node layer and the S layer, and the associated probabilities describe the deployment of information collection components, and their information collection capability. The figure shows that layer B is a layer of which the information can be directly observed. But, it could be any other observable SN layer nodes. The dependency structure involving these two layers can dynamically change. For example, the sampling frequency of a sensor component can change, and a high-risk node may be worth a close look. We could also remove or add sensor components. (3) In Figure 4(a) the links between the information layer and the evidence (E) layer and the associated probabilities describe the reliability and quality of IDS components and their intercorrelation due to the employed detection functions or other factors. Even if we put an IDS where an attack occurs, this IDS may not be able to detect this attack. IDSs using the same algorithm may be good at detecting certain type of attack while not good at other types. This layer is optional in Figure 4(b). (4) The P layer in Figure 4(b) is used to describe the potential risk (prior knowledge) associated with each SN construct. Thus the prior probabilities of different states at Pj can represent the probabilities of different risks. These risks can be natural disasters and artificial emergencies. The conditional probability between the P layer and the SN layer enables the

15

incorporation of such prior knowledge into knowledge integration. When we use DBN models, the P layer can be replaced by the transitional probabilities between time slices. This layer is optional in Figure 4(a) since the priors can be directly given to the network node layer (N’).

Take in Figure 4

In Bayesian network representation, a set of random variables, V = {V1, …VM}, represent the network nodes in the above model, and the prior and conditional probabilities represent the dependency among the states of these nodes. In risk management of supply chains, the states of network nodes represent different events, such as fire or transportation delay. In network attack correlation, the states of concern to us are the normal state and various attacks. In this paper, we simplify the representation by using the basic hypothesis about these nodes, i.e. binary states being normal (or state ‘0’) or abnormal (or state ‘1’). In the probabilistic dependency model, the knowledge integration includes two inference tasks, the belief inference and the most probable explanation (MPE), given the observed reports (evidence E=e) from a set of individual sensors. The belief inference is to assess the posterior probability (belief) for different possible states of network nodes (variables), i.e. pposterior(Vi=0 or 1) = p(Vi=0 or 1|E=e). The MPE task is to find out an assignment of states for all the nodes that best explains the observed evidence (reports of individual sensors). Therefore, the beliefs of node states tell us the current picture of the systems, or in other words, the integration result. Inference in the causal models, supported by causal relationships represented into its structure, can be used in confirmation and alleviation of different hypotheses at all network nodes. Thus, the evidences of alarms, warning and other information from different sensors may

16

reduce the certainty present at single evidence during propagating to those network nodes. The posterior probability about these hypotheses of the hypothesis nodes is the composite score generated by knowledge integration, a more reliable indicator about threat levels. Those nodes with high threat levels reveal possible critical risk locations and attacking paths A computer network is a dynamic system with the information flow pattern changing over time constantly, just like the material flow pattern changes constantly in a supply network. The dependency model is updated constantly to reflect the latest situation, and consequently the control scheme should be able to change the configuration in terms of sensor placements and parameters in working cycles. Utility, Configuration and Planning We consider the knowledge integration task as a control system involving similar “report-control feedback” repetitions. Decisions should be made dynamically in the overall working stages of deployment, detection and diagnosis. As shown in Algorithm 1, we seek an active and dynamic working strategy based on belief update and utility calculation. The utility of sensor components is calculated and updated over time within this probabilistic model, in terms of the possibility of clarifying a hypothesis and the associated cost, reliability, etc. Thus, we are able to compare alternative configurations, i.e., where to invest and install sensors, and which sensors to engage and fine-tune. Planning and scheduling can determine more complicated aspects including the volume, level and boundary of information to choose and collect, in terms of dimensions, locations, details, time granularity, etc. Algorithm 1: Basic knowledge fusion procedures I.

Configure the sensors;

II. Sensors collect information and generate reports to the administrator;

17

III. Administrator fuses the reports generating a composite score about the hypothesis; IV.

If the composite score provides insufficient support to either reject or confirm the hypothesis, change sensor configuration as necessary, and go to II;

V.

Administrator reports the composite score as output.

We elaborate the control scheme by a very important decision, i.e. which sensor to turn on and when. First, we define the benefit of each sensor in terms of mutual information of this sensor to the hypothesis variable, i.e. those network nodes in the model. According to Shannon’s entropy theory (1948), mutual information measures the dependency between two random variables as: I ( N j ; S i ) = ∑ ∑ p ( S i = e ki , N j = hlj ) log( ki

lj

p ( S i = e ki , N j = hlj ) p ( S i = e ki ) P ( N j = hlj )

)

where the ith sensor provides as evidence the kth state eki, and the jth node has the lth state hlj. The higher this mutual information score is, the better this sensor. When there are several (m) network (hypothesis) nodes in consideration and several (n) sensors to activate at one time, the above equation is extended to the following: I ( N ; S ) = ∑ I ( N j ; S1 ,...S n ) / m j

= ∑∑ ...∑∑ p( S1 = ek1 ,..., S n = ekn , N j = hlj ) log( j

k1

kn

lj

p ( S1 = ek1 ,..., S n = ekn , N j = hlj ) p( S1 = ek1 ,..., S n = ekn ) P ( N j = hlj )

)/m

The extended formula rewards a higher mutual information value to the sensor that is tightly correlated to all hypothesis network nodes. During the inference in this Bayesian model, the change of the positions of sensor and the parameters in terms of the conditional probabilities between random variables will change the mutual information values consequently. Thus, a

18

control mechanism can be induced in the adjustment of network reconfigurations, and this is the key topic of our study. Acquiring information incurs cost that negates the benefit from activating a sensor. The cost may be due to reconfiguration and information collection, the computation time for data processing, and the hardware execution time. We consider the cost C of selecting a set of n sensors, where the costs for different sensors are assumed to be incorporated with the same importance, using the following formula: n

C ( S ) = ∑ Ci i =1

m

∑C j =1

j

where Ci or Cj is the cost to acquire the information from sensor i or j, and m is the total number of sensors. Combining the mutual information and information acquisition cost, we form the expected utility index for a set of sensors, where α is the balance coefficient between the two terms. EU ( S ) = α I ( N ; S ) − (1 − α ) C ( S )

Examining the utilities for all combination of existing sensor allows dynamically selecting a subset of sensors of the highest utility in order to adaptively assess the underlying situation. The main challenge is the inference cost for this NP-hard problem with the evolvement of network scope to incorporate more and more system constructs. In addition to applying approximation inference algorithms such as sampling or Monte Carlo methods, parametric approximation methods, and bounded cutset conditioning methods, we can also get help with multi-agent structures that decouple original nodes into relatively independent subsets with the help of soft evidential update (Kim et al., 2004). It is obvious that a lot of independence (not just the

19

conditional independence in Bayesian networks) exists in a computer network or a supply chain system, e.g. sub-networks or single plants. An Integrated Quality Model As in the statistical process control technique used in quality control and improvement, we need to continuously monitor certain quality scores in the fusion procedures. The challenge lies in the fact that the situation of the system at any time instance involves the states of a large number of variables (nodes) associated with different belief values. We need to define quality measures over the states (beliefs) summarized across all these variables in this complex system, consisting of a large number of variables representing the system status. The first requirement of this quality model is to characterize the confidence of the fusion outcome. Confidence represents the certainty level of the fusion outcome. Basically the less uncertain and complex, the more confident are we. For this sake, a global score can be defined on the complexity of current state beliefs. Of the candidates including the information entropy by Shannon and the Kolmogorov complexity, we use the information entropy form here: CF ( N ) = 1 − H ( N ) = 1 − ∑ H ( N j ) / s = 1 + ∑∑ ( p ( N j = hlj ) log p ( N j = hlj )) / m j

j

lj

This confidence score becomes bigger when these nodes have less uncertainty, i.e. one state of each variable has a high belief (probability) while other states of this node have very low beliefs. In other words, we are sure about the true state of this node, e.g. a delay at one supply chain node or an attack happening at one network node. Besides confidence, we need to further distinguish different situations. For example, two nodes N1 and N2 may take a state of either 0 or 1 each. However the four different situations of such a simple network yield the same confidence score. We are interested in characterizing the dynamics of these nodes in terms of the changing states over time, rather than the stable states at 20

one time instance. We want to capture the change point when the system undergoes external disturbance. The disturbance can be signalled by the evolvement of complexity in the system. For example, the state transition of N1 and N2 from ‘00’ to ‘11’ demands more attention to increase the computation power or information collection. Thus, we calculate the relative entropy between two time instances of fusion: RE (t | t − 1) = D (t | t − 1) =

1 mT

∑∑ p t ( N j = hlj ) log( j

lj

p t ( N j = hlj ) p t −1 ( N j = hlj )

)

where pt and pt-1 are the state beliefs at current and last time instance respectively and T is the time interval. This relative entropy is nonnegative and equals zero, only when there is no change in complexity between the two time instances. Based on the definition of above quality scores, we can set different thresholds to control the adaptive knowledge fusion process, e.g. a minimum threshold on the confidence of fusion outcome to be satisfied, and a maximum threshold for the relative entropy between two consecutive knowledge fusion repetitions that cannot be exceeded. Evaluation Using Conceptual Models For evaluation, we focused on the working mechanism rather than a high fidelity model. We used a network structure that has four nodes to deploy sensors as shown in Figure 5. The same structure can model applications in both supply chain risk management and network attack correlation, similar to Figure 4. Four sensors can be deployed shown by the dependency model. Choosing

which

sensor

to

collect

and

detect

evidence

corresponds

to

the

configuration/reconfiguration in operations. The implementation was in MATLAB with the BNT toolkit (Murphy, 2001). The Bayesian inference algorithm was the junction tree engine. Take in Figure 5

21

We ignored the sensor cost to simplify the utility calculation for sensors. We considered only discrete states for the nodes with binary nodes. The prior probabilities for states of the first node layer are evenly distributed at the beginning (0.5, 0.5). We randomly generated different sets of conditional probability between the two node layers to model dynamically changeable information/material flow scenarios in real-world. The conditional probabilities between the node layer and the sensor layer represent 0.99 for the abnormal state detection accuracy, and 0.95 for the normal state, as in Table I. There were 10 consecutive fusion repetitions in the adaptive fusion. Every sensor yielded state 1 as evidence if it chosen, i.e. abnormal. Then the state beliefs for the four nodes at the first layer were updated and recorded.

Take in Table I

We considered the following tasks: 1) adaptive knowledge fusion that actively chooses the best sensor (configuration); 2) impact of sensor accuracy and reliability on (1); and 3) calculation of CF and RE measurements. The sensor with reliability issue has lower detection rate (80%) and higher false alarm rates (20%), changing consequently the conditional probability. Table II shows the configuration/reconfiguration in fusion, i.e. the dynamically selected sensor sequence for different network flow scenarios (conditional probability sets). The sensor with the highest utility score is selected and instantiated to state 1. Different network flow settings and changes in the node state beliefs generate different selection sequences. This sequence of sensor engagement shows the adaptive nature of this fusion framework.

Take in Table II 22

Figure 6 plots the knowledge integration performance of adaptive fusion and inference with a fixed sensor location for the same network flow scenario. Other scenarios show similar features in their plots. In Figure 6(a), the states for most network nodes become clear very soon along with the adaptive knowledge fusion repetitions. For example, the states of three of the four nodes have very little uncertainty after several repetitions. That is, the belief for the intrusive state is either close to 1 or 0. CF increases during the progress of this adaptive fusion while RE decreases in each fusion cycle quickly. Figure 6(b) shows the performance of fixed configuration that puts the sensor at node D for flow scenario 1, which is the best among the four possible fixed node locations according to the calculation of initial utility. We easily observe that compared to the adaptive knowledge fusion, the CF and RE converge more slowly in the fusion cycles. From this observation and more experimental results on different network scenarios, we believe that this adaptive fusion strategy can yield better performance.

Take in Figure 6

Lastly, we examine the impact of the reliability of sensor on this adaptive fusion process. Figure 7 shows the beliefs curves when all sensors have lower hit rate and higher false alarm rate, for the same network flow scenario as in Figure 6. It shows clear difference in state belief, CF and RE curves caused by the different sensor selection sequence due to the new parameters. By examining CF and RE here and other experimental results, we can conclude that generally sensors with high reliability can achieve higher confidence and lower relative entropy within the same fusion time period, meaning more efficient fusion.

23

Take in Figure 7

In Figure 8, we give the example of the performance for another situation of sensor reliability for network flow scenario 1, where only sensor at node location A has lower hit rate and higher false alarm rate. From the configuration sequence in Table II for the same network flow setting without reliability issue, sensor A is selected at the first three time slices with the highest utility. Figure 8 shows that the curves become different, indicating that the sensor selection sequence changes. In fact, sensor D is selected in the first three cycles, and sensor A is never selected during the ten cycles. This clearly shows again the adaptive nature of the framework.

Take in Figure 8

Discussion and Conclusions Imagine that we have a network graph that contains an accurate model about an enterprise system. This model is dynamically learned and constantly updated to reflect the most current status. Many important applications, including enterprise situation surveillance, risk assessment and migration, computer attack prevention, logistics monitoring and control, etc., exist in such complex systems and can get great help from such a model. Evidence about part of the enterprise, in terms of reports from physical, software or human sensors that can indicate emerging or potential problems, by error messages, going-down facilities, delays, machine starvation alarms, etc., will be entered into this model. We suspect that something is going on wrong in our system. But we have to ask ourselves first that given these evidences what the big

24

picture really is about this system, and where the critical positions of the enterprise system are, before we try to generate a solution about how to invest capital and manpower to migrate the problems in the face of an uncertain and complex operational environment. We have to do this in an active and timely manner where we are able to handle incomplete and uncertain information systematically. This is exactly what an adaptive knowledge integration scenario achieves. In this paper, we are motivated by the challenge raised in complex enterprise network management to seek a system paradigm shift to novel knowledge integration methodologies for the above applications. Compared to existing efforts, this research makes the important contributions to 1) apply a system and process-oriented approach to engineer the essential management tasks in a distributed and complex systems environment, following the principles from supply chain/network management discipline; 2) develop an adaptive framework that seamlessly integrates a dependency model, a dynamic feedback control mechanism and a quality model based on information and utility theory, and 3) support an active knowledge fusion strategy with a predictable quality assurance that seeks the optimal configuration of intrusion detection resources. This research serves as an exploratory study into the system modelling and knowledge management in response to complex network management paradigms that efficiently apply latest networking and connective technologies. In our view, representing these systems into an appropriate network model and integrating the knowledge in the model for decision-making, directed by information and complexity measures provides a promising approach. References Axelsson, S. (2001), “Intrusion detection systems: a survey and taxonomy”, Report, Dept. of Computer Engineering, Chalmers University of Technology, Goteborg, Sweden.

25

Brewer, A., Sloan, N., Landers, T. (1999), “Intelligent tracking in manufacturing”, Journal of Intelligent Manufacturing, Vol. 10, No. 2-4, pp. 245-250. Chandra, C., Everson, M., Grabis, J. (2005), “Evaluation of enterprise-level benefits of manufacturing flexibility”, OMEGA the International Journal of Operations Management, Vol.33, No 1, pp. 17-31. Chandra, C., Grabis, J. (2004), “Categorization of mass customization research”, in Chandra, C., and Kamrani, A. (eds.) Mass Customization: Supply Chain Approach, New York: Kluwer Academic, pp. 21-59. Chandra, C., Grabis, J. (2001), “Reconfiguration of multi-stage production systems to support product customization using generic simulation models”, Proceedings of the 6th International Conference of Industrial Engineering Theory, Applications and Practice, Nov. 18-20, San-Francisco, California. Choi, T., Dooley, K., Ye, N. (2000), “Control versus emergence in scaleable enterprise systems”, NSF Awardees Conference 2000. Cover, T. M., Thomas, J. (1991), Elements of Information Theory, Wiley. Debar, H., Dacier, M., Wespi, A. (1999), “Towards a taxonomy of intrusion-detection systems”, Computer Networks, Vol. 31, pp. 805-822. Denning, D. (1987), “An intrusion-detection model”, IEEE Trans. on Software Engineering, Vol. 13, No. 2, pp. 222-232. Farnum, N. (1994), Modern Statistical Quality Control and Improvement, Belmont, CA: Wadsworth, 1994. Fliedner, G. (2003), “CPFR: an emerging supply chain tool”, Industrial Management & Data Systems, Vol. 103, No. 1, pp. 14-21. 26

Giachetti, R. E., Young, R. E., Roggatz, A., Eversheim, W., Perrone, G. (1997), “A methodology for the reduction of imprecision in the engineering process”, European Journal of Operational Research, Vol. 100, pp. 277–292. Glymour, C., Cooper, G. (eds.) (1999), Computation, Causation & Discovery, Menlo Park, CA: AAAI Press/The MIT Press. Graves, S. C., Kletter, D. B., Hetzel, W. B. (1998), “A dynamic model for requirements planning with application to supply chain optimization”, Operations Research, Vol. 46, No. 3, pp. 3549. Hendricks, von K. B., Singhal, V. R. (2003), “The effect of supply chain glitches on shareholder wealth”, Journal of Operations Management, Vol. 21, pp. 501-522. Hirsch, B. (1995), “Information system concept for the management of distributed production”, Computers in Industry, Vol. 26, pp. 229–241. Hsu, L-L. (2005), “SCM system effects on performance for interaction between suppliers and buyers”, Industrial Management & Data Systems, Vol. 105, No. 7, pp. 857-875. Hughes, V., Love, P.E.D. (2004), “Toward cyber-centric management of policing: back to the future with information and communication technology”, Industrial Management & Data Systems, Vol. 104, No. 7, pp. 604-612. Jensen, F. (1996), An Introduction to Bayesian Networks, London: UCL Press. Jin, C., Li, M., Wang, D., Li, L. (2004), “A logistic prototype based on GIS and GPS”, Proceedings of the 2004 International Conference on Communications, Circuits and Systems, Vol. 2, pp. 1485-1490. Kim, Y. G., Valtorta, M., Vomlel, J. (2004), “A prototype system for soft evidential update”, Applied Intelligence, Vol. 21. 27

Koudal, P., Lee, H. L., Whang, S., Peleg, B., Rajwat, P. (2004), “OnStar: Connecting to customers through telematics”, Case Study Report, Stanford University. Kulkarni, A.B., Bush, S.F., Evans, S.C. (2001), “Detecting distributed denial of service attacks using Kolmogorov complexity metrics”, Report 2001CRD176, GE R&D Center, Dec. Lee, H.L., Padmanabhan, V., Whang, S. (1997), “Information distortion in a supply chain: The bullwhip effect”, Management Science, Vol. 43, No. 4, pp. 546–558. Léger, P.M., Cassivi, L., Hadaya, P., Caya, O. (2006), “Safeguarding mechanisms in a supply chain network”, Industrial Management & Data Systems, Vol. 106, No. 6, pp. 759-777. Li, X., Ye, N. (2003), “Decision tree classifiers for computer intrusion detection”, Journal of Parallel and Distributed Computing Practices, Vol. 4, No. 2. Li, X., Ye, N. (2006), “A supervised clustering and classification algorithm for mining data with mixed variables”, IEEE Transactions on Systems, Man, and Cybernetics-Part A, Vol. 36, No. 2. Mackay, D. (2003), Information Theory, Inference, and Learning Algorithms, Cambridge, UK: Cambridge University Press. Metters, R., (1997), “Quantifying the bullwhip effect in supply chains”, Journal of Operations Management, Vol. 15, No. 2, pp. 89–100. Mirkovic, J., Reiher, P. (2004 ), “A taxonomy of DDoS attack and DDoS defense mechanisms”, ACM SIGCOMM Comput. Commun. Rev. Vol. 34, No. 2, pp. 39-53. Murphy, K. (2001), “The Bayes Net Toolbox for Matlab”, Computing Science and Statistics, Vol. 33.

28

Pearl, J. (1998), Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, San Mateo, CA: Morgan Kaufmann Publishers. Porras, P.A., Neumann, P.G. (1997), “EMERALD: Event monitoring enabling responses to anomalous live disturbances”, Proceedings of the Nineteenth National Computer Security Conference, Baltimore, Maryland, October, pp. 353-365. Royer, J. (1992), “A new set interpretation of the inheritance relation and its checking”, ACM SIGPLAN OOPS Messenger, Vol. 3, No. 3, pp. 22-40. Shannon, C. E. (1948), “A mathematical theory of communication”, Bell System Technical Journal, Vol. 27, pp. 379-423,623-656. Shedler, G. S. (1993), Regenerative Stochastic Simulation, San Diego, CA: Academic Press. Shubik, M. (2002), “Game theory and operations research: Some musings 50 years later”, Operations Research, Vol.50, pp. 192-196. Staniford-Chen, S., Tung, B., Schnackenberg, D., (1998), “The Common Intrusion Detection Framework (CIDF)”, Information Survivability Workshop, Orlando FL, October. Suh, N. P. (1995), “Design and operation of large systems”, Journal of Manufacturing Systems. Vol 14, No. 3, pp. 203–213. Varshney, U., Vetter, R. J., Kalakota, R. (2000), “Mobile E-commerce: A new frontier”, IEEE Computer, Vol.33, No. 10, pp. 32-38. Ye, N., Choi, T., Dooley, K. - Cochran, J. (2000), Modeling and simulation of SN enterprise, INFORMS2000.

29

Ye, N., Li, X., Chen, Q., Emran, S. M., Xu, M. (2001), “Probabilistic techniques for intrusion detection based on computer audit data”, IEEE Transactions on Systems, Man, and Cybernetics-Part A, Vol. 31, No. 4. Yu, Z., Yan, H., Cheng, T.C.E. (2001), “Benefits of information sharing with supply chain partnerships”, Industrial Management & Data Systems, Vol. 101, No. 3/4, pp. 114-119. Zhang, Y., Lee, W., and Huang, Y., (2003), “Intrusion detection techniques for mobile wireless networks”, ACM/Kluwer Wireless Networks Journal (ACM WINET), Vol. 9, No. 5, September. Zurek, W.H. (ed.) (1990) Complexity, Entropy and the Physics of Information, Proceedings of the 1988 Workshop on the Complexity, Entropy and the Physics of Information, New York: Addison-Wesley.

30

Figure 1 A temporal-spatial illustration of an automotive enterprise system Supplier

Supplier

Supplier

Component plant

Component plant

Component plant Assembly plant

Assembly plant

Assembly plant Distribution center

Distribution center

Distribution center Dealer

Dealer

Dealer

Customer

Customer

Customer

t-2

t-1

t

31

time

Figure 2 An illustration of a distributed intrusion detection system, where local IDSs collaborate with a regional IDS administrator IDS IDS administrator Report Control

32

Figure 3 A DBN consisting of time slices, where H represents a collection of hypothesis nodes, S state nodes, Es observation nodes, and t represents time

E1

0

t-1

t

H0

Ht-1

Ht

S0

St-1

St

E2

E3

E1

E2

33

E3

E1

E2

E3

Figure 4 The conceptual dependency model of network nodes and sensor nodes, for (a) computer network attack correlation and (b) supply chain risk management. Node output

N1’

…

Ni’

…

Nk’

P Layer

Nn’ P(Pj)

P1

P(N|N) Node input

N1

…

Ni

…

Nk

Nn

…

…

Ii

…

Ik

Sensor P(Si|Nk)

…

… …

Si

Bi

Ai …

Layer B Node B1

Pj

…

…

…

Sk

Pk

…

Ak

Bk …

P(E|I) IDD evidence

P(Ni|Nk)

A1

…

P(I|N) Real info.

Layer A Node

…

E i’

…

Ek’

Pn

(a)

…

…

…

…

An’

Bm

(b)

34

…

Figure 5 The evaluation model of a simple network for (a) supply chain risk management or (b) network attack correlation SA A1

A1

B1

C1

D1

A2

B2

C2

D2

IDA

IDB

IDC

IDD

A2 SB

B1

B2 SC

C1

C2

D1

D2

SD

(a)

(b)

35

Figure 6 The change of beliefs of network nodes, CF, and RE in the knowledge fusion cycles, where only the belief for the intrusive state is plotted for each node.

(a) Network flow scenario 1 for adaptive fusion

(b) Network flow scenario 1 where the IDS is fixed at node D

36

Figure 7 The performance for sensor having lower hit rate and higher false alarm rate in network scenario 1, where only the belief for the “abnormal” state is plotted for each node

37

Figure 8 Plot to show the performance for network flow scenario 1 with sensor A having lower hit rate and higher false alarm rate.

38

Table I Conditional probability for sensors and nodes’ states Nodei Sensori Normal (0) Intrusive (1)

Normal (0) 0.95 0.01

Intrusive (1) 0.05 0.99

39

Table II Sensor selection sequence in the adaptive knowledge fusion Time slice Scenario 1 Scenario 2 Scenario 3 1 A D A 2 A D A 3 A A A 4 C A A 5 C A A 6 C B A 7 C A D 8 C B D 9 C A D 10 B B D

40