Optimization Problems Involving Collections of Dependent Objects

Download PDF
2 downloads 0 Views 830KB Size Report
Comment
Associated Universities under DOE contract number DE-AC05-00OR22750. .... bridge is known to be stable, there may be a cost in fuel to cross. ... some probability of an attack on a convoy transporting the goods, what is the correct order to ...
Optimization Problems Involving Collections of Dependent Objects David L. Roberts∗ and Charles L. Isbell Jr. School of Interactive Computing College of Computing Georgia Institute of Technology Atlanta, GA USA {robertsd,isbell}@cc.gatech.edu

Michael L. Littman Department of Computer Science Rutgers University Piscataway, NJ USA [email protected]

Abstract We describe a class of problems motivated by numerous real-world applications where there is a collection of objects that have both a cost and a value, but where some of those objects depend upon other objects to obtain their full value. Applications include finding an optimal order for transferring files under threat of system failure, ordering sequences of actions by a heterogeneous team of agents or robots, picking an optimal set of products to store in a warehouse, selecting courses to take at a university, or picking what products to cut from production. We formalize the problem of representing objects and their dependence relationships as a directed acyclic graph (DAG). We define simple formulae for calculating the utility of both sets and sequences of graph vertices. We motivate, using real-world examples, a taxonomy of problems associated with the model we present. We also prove that two variants of problems associated with our formalism are NP-hard, and present an efficient algorithm for solving a restricted version of a third problem.

1 Introduction Imagine that a user is composing a paper, p.tex, in LATEX. The paper includes a number of postscript figures, f1 .eps, f2 .eps, . . ., fi .eps, . . ., fn .eps. The p.tex file as well as the fi .eps files are all necessary to generate the corresponding pdf file. We can say that p.tex is dependent on the set {. . . , f i .eps, . . .}; that is, with respect to the problem of generating a final pdf file, p.tex has no value if any of {. . . , f i .eps, . . .} is unavailable.1 Note that in this example the postscript figures do not necessarily depend upon p.tex for their own value. They can be edited and displayed properly without p.tex, or may be used as figures in other contexts. In this paper, we introduce a mechanism for representing and reasoning about these kinds of relationships. Our main inspiration arises from our efforts to build systems where agents are adaptively coordinating the storage and transfer of related sets of files across many devices [Roberts et al., 2006]. In particular, we have been concerned with either selecting some subset of files to transfer between devices, selecting ∗ Part of this research was performed while on appointment as a U.S. Department of Homeland Security (DHS) Fellow under the DHS Scholarship and Fellowship Program, a program administered by the Oak Ridge Institute for Science and Education (ORISE) for DHS through an interagency agreement with the U.S. Department of Energy (DOE). ORISE is managed by Oak Ridge Associated Universities under DOE contract number DE-AC05-00OR22750. All opinions expressed in this paper are the author’s and do not necessarily reflect the policies and views of DHS, DOE, or ORISE. 1 There may be some value in having only p.tex. For example, the user may simply want to perform a spell check. In one context the file alone is sufficient; however, in the context that we described above, it is not. The problem of determining context for file usage is the subject of ongoing research.

1

.eps 1 (3, 2)

.eps ... 2 (4, 2)

.eps n (2, 3)

.tex (6, 2)

Figure 1: Sample graph representing dependencies between a collection of data files. Each file is a vertex labeled with its value and cost, respectively. In this example, the .eps i files enable the .tex files. some subset of files to delete or identifying an order in which to transfer files that will result in the best performance under the possibility of network failures. Although we are motivated by file-transfer problems, our concern here is broader. We are interested in formally defining a class of Dependent Value Optimization (DVO) problems, of which our file-transfer problems happen to be instances. DVO problems also include, but are not limited to, communication across multiple agents, warehouse inventory selection, and coordination in multirobot systems, as well as other logistics problems. In our formulation, each object in a system is represented by a vertex in an acyclic graph. 2 We encode dependence using directed edges. The graph corresponding to our p.tex example is shown in Figure 1. Each vertex is labeled with two positive integers: value and cost. In general, if there is an edge from a vertex v i to a vertex vj , we say vertex i enables vertex j. The enabling relationship is not transitive: dependency affects the value of the child vertex in the graph only.3 If a vertex is enabled, it realizes its value. If the vertex is not enabled, it realizes no value. The enabling relationship forms the basis for DVO problems. The main contributions of this work are: • A Taxonomy of DVO problems. We formally define terminology useful for discussing and analyzing DVO problems and their solutions, grounding a taxonomy of DVO problems in a discussion of some of the many applications that are well-described by the framework. • Proofs of Hardness. We provide proofs that two of three variants of the DVO problems we describe are NP-hard. The status of the third is still open. • Algorithms. We provide an exact polynomial-time algorithm for finding a solution to the third DVO problem when the graph is a polytree. This paper is organized roughly along the lines of the contributions. We describe a set of applications that can be couched in our framework before formally defining DVO problems and introducing the DVO taxonomy. We then prove that two of the DVO problems are NP-hard, and show that a special case of one of the DVO problems is solvable in polynomial time using dynamic programming. We then place DVO 2

We choose to model dependencies as an acyclic graph in order to make the mathematics more manageable. We believe that, in most cases, cyclic dependencies are actually a matter of context (i.e. in one context object 1 depends on object2 and in another context it is the other way around). Although we do not allow cycles in our graph, the formalism can handle cyclic dependencies through the use of “hyper-vertices” that represent all of the objects in a cyclic dependence relationship. 3 There are situations where transitive dependencies are appropriate. For example, a flashlight enables the use of screw driver in a dark area but without batteries this can’t happen. In that case, the batteries also enable the screw driver. It is trivial to include transitive closure edges in the graph to achieve the desired semantics.

2

(1, 3)

sat

(6, 2) com

a

w

s

(3, 3)

(1, 2)

(3, 3)

Figure 2: Sample graph representing dependencies between a satellite, a communication robot, and a collection of acquisition robots. problems in the larger framework of graphical representations of similar problems and describe several avenues for further research.

2 Applications and Motivation There are a variety of applications that are well-modeled by the DVO formalism. Here, we will briefly describe a number of those applications to motivate and ground the types of problems common to the DVO formalism. Consider a collection of music files and a program responsible for filling a digital music player that cannot store the entire collection. Further, suppose this music manager maintains a model of the listening habits of its user. The manager may know that the user likes to listen to certain genres of music together, or whole albums at a time. These relationships among music files can be modeled as the enabling relationship. The goal of the transfer program then becomes finding the set of music files to fill the music player that will maximize listening enjoyment subject to the enabling constraints. Additionally, imagine that this particular user regularly is rushed in the morning and rarely leaves her music player connected long enough to transfer all of the selected music files. Then, the transfer manager needs to consider a second important problem of determining the order to transfer those files that will result in the maximum expected utility given that the transfer of files may be interrupted. For the purposes of this paper, we assume access to measures of user preference. In general, these preferences can be estimated based on statistical models of user behavior. See earlier work on this topic [Isbell et al., 2004] for more information. As another example, consider a multirobot system. Robots can be represented as vertices and dependencies between their functions represented as directed edges. Specifically, imagine three robots that can acquire air, soil, or water samples from a contaminated area and a fourth robot that can relay data between the satellite and the other robots. Acquiring data is useful only if it can be relayed to the satellite: the communication robot enables each of the acquisition robots. Communicating data is useful on its own but is more useful if it can be communicated immediately via the satellite: the satellite enables the communication robot. (This DVO graph is shown in Figure 2.) Now, suppose each robot must cross a bridge that is thought to be unstable and may collapse at any point. This problem is analogous to the music-transfer problem in that the robots must find an optimal order in which to cross. On the other hand, even if the bridge is known to be stable, there may be a cost in fuel to cross. Then, it is necessary for the robots to select the subset of themselves that should cross that will maximize their capability on the other side while minimizing their cost. This problem is slightly different from the transfer manager’s first problem. Whereas the transfer manager is filling a specific size container, the robots are trying to select the set that maximizes utility independent of size. We can also imagine transport logistics for the military benefiting from our formalism. Suppose there is a supply warehouse that contains various goods that need to be transported to forward bases and troops on 3

the front lines. The DVO framework is especially relevant when you start to consider dependencies between goods like flashlights and batteries or water purification tablets and canteens. Specifically, the flashlights are effectively useless without batteries. This situation can be modeled by the dependencies in our graph structure. It is natural to ask a few types of questions about the goods in the warehouse. Specifically, given some probability of an attack on a convoy transporting the goods, what is the correct order to transfer them to maximize the usefulness of the goods that get through to the front lines? Second, given limited capacity of the transport vehicles, what is the correct subset of goods to transfer to the front lines that maximizes the use of available capacity. Third, it would also be natural to ask if there are some goods that should not be warehoused (for example, are there some goods that are not worth transporting to the front line). Similarly, we might want to model a company that manufactures products that depend on each other to function. For example, a company may manufacture remote-control cars as well as the remotes that operate the cars. Given costs and profits as well as dependencies for each product, the company can identify which, if any, of its products should be cut from production to maximize net profit by solving a specific DVO problem. Students can model courses and prerequisites using DVO graphs based on their specific preferences. Specifically, the value of courses could correspond to relevance or enjoyment and cost could be a measure of workload. Then, students can determine the best set of courses to take that will fill all credit hour requirements. On the other end of the spectrum, professors can use DVO graphs to help build the syllabus for a class. Specifically, some topics are of interest to them and others are necessary to enable the teaching of the topics they find interesting. Costs may model hours spent in class, or preparation time. Again, we can use this formalism to construct a syllabus that will maximize value or utility of a class. We could also use the DVO formalism to construct a presentation sequence given a distribution over the likelihood of losing a certain number of class hours due to unplanned travel, sickness, inclement weather, or errors in measuring the amount of time needed to explain material. For the final application we discuss, suppose there is a set of tests one can run on containers entering a port. Each test provides different data about a container. Further, suppose there are some non-binding precedence constraints between the tests. For example, consider a test that measures the temperature of the air inside the container and another test that measures the concentration of a heat-sensitive volatile chemical in the air inside the container. Knowing the temperature is less useful if the chemical concentration is not known. Personnel inspecting containers can use the answers to any of the three DVO problems we discuss to identify an optimal subset of tests or optimal ordering of those tests.

3 Dependent Value Optimization Problems In this section, we will first define the worth of a vertex and then broaden that definition by describing different types of relationships between objects that we may encounter in real-world applications.

3.1

What is the worth of a vertex?

There are several measures of worth for a vertex that we must consider. Each of these measures is based on the two positive integers: value and cost. These vertex attributes form the basic measures of worth: value itself (Val (vi )) and utility (Util (vi ) = Val (vi ) − Cost(vi )). Perhaps even more important measures of worth are enabled value and enabled utility. They capture the notions that a vertex may not always realize

4

.eps 1 (3, 2)

.eps 2

(4, 2)

(8, 3)

.eps 3

(2, 3)

(6, 2)

.pdf AND

(3, 2)

.tex

.ppt (15, 6)

.mp3 1

.mp3 ... 2 (4, 2)

.mp3 n (2, 3)

(6, 2) .mpu

OR

THRESHOLD

(a) AND/OR

(b) Threshold

Figure 3: Examples of three types of dependence constraints. In this scenario, we see evidence of two types of constraints on the relationship between files in 3(a) and a third type in 3(b). its value. Let G be a graph and S be a set of vertices in G. Then: ( Val (vi ) if Par (vi , G) ⊆ S EV (vi |S, G) = 0 otherwise ( Util (vi ) if Par (vi , G) ⊆ S EU (vi |S, G) = −Cost(vi ) otherwise

(1) (2)

where Par (vi , G) is the set of vertices in G that have a directed edge to vi (its parents) and S is a set of previously selected (or “available”) vertices.4 There is also cumulative enabled value (CEV): X CEV (S, G) = EV (s|S \ {s}, G). (3) s∈S

CEV represents the sum total of the enabled value of the set of vertices. The notation CEU (S, G) is defined similarly using EU . As we shall see later, there are cases where we may be concerned with a sequence of graph vertices and an associated probability distribution that indicates the probability a particular point in the sequence will be reached. In this case, the measure of worth is expected cumulative enabled value: → − |S |

X → − (pi · CEV ([s1 , · · · , si ], G)), ECEV ( S , P, G) =

(4)

i=1

→ − where S = hs1 , s2 , . . .i is a sequence (rather than a set) of vertices and P is the probability distribution over the sequence with pi equal to the probability of the sequence being terminated after the first i elements.

3.2

Representing dependencies

There are several possible dependence relationships that we can consider. Recall the LATEX example described in Section 1 and depicted in Figure 1. In that example, the set of eps files enabled the tex file. It is necessary that all figures are available for compilation to succeed. This requirement, as well as the constraint expressed in Equations 1 & 2, is an AND constraint on the preconditions for the tex file to be enabled. 4

For notational convenience we may drop G as an argument to functions if there is no ambiguity.

5

.eps 1 (3, 2)

.eps 2 (4, 2) [1]

[1]

.eps 3

.tex (6, 2)

(2, 3) [1]

[1]

(8, 3)

.pdf

[4]

(15, 6) .ppt VOTED (4)

Figure 4: A third example of constraints on eps, tex, and ppt files. In this scenario, there is a voted threshold constraint. In Figure 3(a), we consider multiple tasks associated with the same set of files. The first task is similar to before—to make a pdf. The second task is to use the contents of the paper to build a (ppt) presentation. Let us assume that the focus of the presentation is on the analysis contained in the text of the paper and not on the contents of the figures. In this case, we are interested in two files: the pdf and the ppt. To succeed in the create-pdf task, all of the eps files and the tex source are required. The same AND constraint mentioned before appears here; however, when it comes time to build the presentation, either the latex source or the compiled pdf will enable the ppt file. This example represents a second type of constraint—an OR constraint. In Figure 3(b), we see evidence of a third type of constraint. The vertex at the bottom of the graph represents a playlist of n mp3 files that, when played consecutively, last four hours. The playlist on its own can be used without many of the actual mp3 files it references (players typically just skip songs in the list that are missing) but actually has little or no value in that case. Suppose the playlist is used to select music to transfer to a portable player for a three-hour train ride. If there are enough of the n mp3 files in the playlist available to fill up the three hours, then it is useful and realizes its value. If there are not enough mp3 files to fill up the three hours, then it is useless. This third type of constraint on the relationships between entities is a threshold constraint. The voted threshold is a variant of the threshold constraint. Rather than a simple threshold where each vertex contributes the same amount to reach the threshold, different vertices can contribute different amounts. Thus, there can be one vertex that satisfies the constraint or possibly a set of vertices that can satisfy the constraint in its place. In Figure 4, we recast the eps, tex, ppt example to use a voted constraint. Each edge is labeled with its “contribution” in brackets and the threshold is listed below the ppt vertex. This type of constraint is similar to that of a threshold activation function in a McCulloch-Pitts neuron [McCulloch and Pitts, 1943] used in Artificial Neural Networks. In general, as in the example from Figure 3(a), it need not be the case that every vertex in the graph has the same type of constraint. For the purposes of this paper, we restrict our discussion to graphs for which every vertex uses an AND constraint.

3.3

Three DVO Problems

As indicated by the applications described above, when we begin to think about the world in terms of objects with value and cost and the enabling relationship, there are a few natural problems that arise. Specifically, we find the need to solve three basic problems based on these graphs: 1. Given a graph and a probability distribution over sequence lengths, find a total ordering of the vertices such that the expected utility of sequence prefixes (according to the order and weighted by the probability distribution) is maximized. We call this problem the DVO Sequence Problem. It models the robot bridge crossing or the music-transfer problem when catastrophic failure is possible. 2. Given a graph and a constant k, find a subset of the vertices with cost at most k for which the value is 6

maximized. We call this problem the DVO Constrained Subset Problem. It models the fill-the-digitalmusic-player problem when transfer is guaranteed. 3. Given a graph, find a subset of the vertices such that the utility of the set is maximized. We call this problem the DVO Intrinsic Subset Problem. It models the robot bridge-crossing problem when there is no danger of bridge collapse but there is a toll to cross. These graph problems are the abstraction of many problems in manufacturing, warehousing, multiagent systems, transport logistics, and other areas. Unfortunately, as we show below, at least two of these problems are NP-hard. Nonetheless, we have developed heuristic algorithms and/or exact solutions to various restricted DVO problems. In addition to our proofs of hardness, we present one of these algorithms, a solution to the DVO Intrinsic Subset Problem when the graph structure is restricted to a forest of polytrees.

4 Hardness of the DVO Problems Here, we prove that both the DVO Sequence problem and the DVO Constrained Subset problem are NPhard.

4.1

The DVO Sequence Problem

First, we show that the DVO sequence problem is NP-hard by showing that a special case of the DVO sequence problem is NP-hard. Theorem 4.1. The DVO Sequence Problem is NP-hard. Proof. We show that a polynomial-time solution to the DVO Sequence Problem would provide a polynomialtime solution to the k-clique problem, well known to be NP-hard [Garey and Johnson, 1979]. This reduction shows that the DVO Sequence Problem is also NP-hard. An instance of k-clique consists of a graph G0 = (V 0 , E 0 ) and an integer k > 1. The solution is “yes” if there exists a subgraph of G0 consisting of k nodes and k(k − 1)/2 edges—all pairs of selected nodes are connected. Otherwise, the solution is “no.” We can take an instance of k-clique and create an instance of the DVO Sequence Problem such that a solution to the DVO Sequence Problem can be used to solve the k-clique instance. An instance of the DVO Sequence Problem consists of a graph G = (V, E) and a probability distribution over sequence lengths. First, we create a node in V for each node in V 0 and one for each edge in E 0 . So, V = {i|i ∈ V 0 } ∪ {(i, j)|(i, j) ∈ E 0 }. For all i ∈ V , we define Cost(i) = 2 and Val (i) = 1, while for all (i, j) ∈ V , we define Cost((i, j)) = 1 and Val ((i, j)) = 2. We then create edges in E as follows. For each (i, j) ∈ E 0 , add an edge to E from i to (i, j) and from j to (i, j). That is, nodes i ∈ V have no parents and one child for each neighbor, and nodes (i, j) ∈ V have two parents and no children. Formally, E = {(i, (i, j))|(i, j) ∈ E 0 } ∪ {(j, (i, j))|(i, j) ∈ E 0 }. We define the probability distribution that assigns likelihood to sequence prefixes so that it assigns probability 1 to a prefix of length k(k − 1)/2 + k = k(k + 1)/2 and zero to all other lengths. Thus, at most k(k + 1)/2 nodes can be usefully selected for the solution. Now, we ask, what is the solution to the DVO Sequence Problem for this instance? It will consist of some number, say x, of nodes that were originally nodes in G 0 and some number, say y, of nodes that were originally edges in G0 . Here, x + y ≤ k(k + 1)/2, as no larger set of nodes will be assigned non-zero probability. 7

What is the utility of the selected subset? There will be a utility of −1 for each node selected (x). There will be a utility of −1 for each edge selected for which its endpoints in G 0 have not been selected. As such, there is no utility benefit to selecting an edge whose endpoints have not been selected, as such nodes have negative utility and do not enable any other nodes. We can assume, therefore, that all y edges that have been selected also have their endpoints selected and, therefore, provide a utility of 1. Thus, the utility of the subset will be y − x. If the selected subset is a k-clique, then x = k and y = k(k − 1)/2 for a utility of y − x = k(k − 3)/2. We argue that no other subset of size k(k + 1)/2 has so large a utility. First, note that y ≤ x(x − 1)/2 since we can’t have more edges than pairwise combinations of nodes. So, the utility is bounded by x(x − 3)/2 and any x < k will have smaller utility than that of a k-clique. In addition, note that y ≤ k(k − 1)/2 − x, so x > k also results in a reduction of utility compared to that of a k-clique. Since the selection of a k-clique leads to the unique maximal utility, an optimal solution to the DVO Sequence Problem will have utility of k(k − 3)/2 if and only if the original graph has a k-clique. Since determining this fact is known to be NP-hard, solving the DVO Sequence Problem must also be NP-hard.

4.2

The DVO Constrained Subset Problem

Here, we show the DVO Constrained Subset Problem is NP-hard. Theorem 4.2. The DVO Constrained Subset Problem is NP-hard. Proof. We show that a polynomial-time solution to the DVO Constrained Subset Problem would provide a polynomial-time solution to the Maximum Edge Subgraph problem, known to be NP-hard [Kann, 2003]. This reduction, similar to that in the previous section, shows that the DVO Sequence Problem is also NPhard. An instance of Maximum Edge Subgraph consists of a graph G 0 = (V 0 , E 0 ), a weight function w : c0 ⊆ V 0 such that E 0 → N , and integers k 0 > 1 and t0 > 1. The solution is “yes” if there exists a subset V 0 0 c | = k and |V X w((i, j)) ≥ t0 . c0 ×V c0 ) (i,j)∈E 0 ∩(V

Otherwise, the solution is “no”. We can take an instance of Maximum Edge Subgraph and create an instance of the DVO Constrained Subset Problem such that a solution to the DVO Constrained Subset Problem can be used to solve the Maximum Edge Subgraph instance. An instance of the DVO Constrained Subset Problem consists of a graph G = (V, E) and two integers k > 1 and t > 1. First, we let k = k 0 and t = t0 . Next, we create a vertex in V for each vertex in V 0 and one for each edge in E 0 . So, V = {i|i ∈ V 0 } ∪ {(i, j)|(i, j) ∈ E 0 }. For all i ∈ V , we define Cost(i) = 1 and Val (i) = 0, while for all (i, j) ∈ V , we define Cost((i, j)) = 0 and Val ((i, j)) = w((i, j)). We then create edges in E as follows. For each (i, j) ∈ E 0 , add a directed edge to E from i to (i, j) and from j to (i, j). That is, nodes i ∈ V have no parents and one child for each neighbor, and nodes (i, j) ∈ V have two parents and no children. Note that vertices i and j enable vertex (i, j). Formally, E = {(i, (i, j))|(i, j) ∈ E 0 } ∪ {(j, (i, j))|(i, j) ∈ E 0 }. Now, we ask, what is the solution to the DVO Constrained Subset Problem for this instance? Since all vertices (i, j) ∈ V have no cost and non-negative value, they can all be included in the solution (although they may not be enabled). Next, since all vertices i ∈ V have cost 1, the solution can include up to k of them. The specific k vertices that are used will be chosen to enable the most value in the (i, j) vertices. 8

What is the value of the selected subset? For each of the vertices i ∈ V that has been selected, the value is 0. Thus, the total value for the selected set will be ( X V al(v) if v = (i, j) is enabled CEV (V, G(V, E)) = 0 otherwise. v∈V

Next, we show that the solution to this instance of the DVO Constrained Subset Problem is a soluc0 = {i ∈ V |i has been included}. Note, tion to the Maximum Edge Subgraph instance as well. Let V c0 | = k 0 . Next, suppose since we included exactly k of those vertices and since k = k 0 , we have |V CEV (V, G(V, E)) ≥ t and the answer to the DVO Constrained Subset instance is “yes”. Then, we have ( X V al(v) if v = (i, j) is enabled t ≤ CEV (V, G(V, E)) = 0 otherwise v∈V ( X w((i, j)) if v = (i, j) is enabled = 0 otherwise v∈V ( c0 and j ∈ V c0 X w((i, j)) if i ∈ V = 0 otherwise v∈V X = w((i, j)) c0 ×V c0 ) (i,j)∈E 0 ∩(V

and therefore the answer to the Maximum Edge Subset instance is also “yes”. Now, assume by way of contradiction that the solution to the DVO Constrained Subset instance is “no”, but the solution to the Maximum Edge Subgraph instance is “yes”. Then, without loss of generality, we c0 , but p is not included in the solution to the DVO Constrained Subset have some p ∈ V 0 for which p ∈ V instance. Since we included exactly k = k 0 vertices v ∈ V in the solution to the DVO Constrained Subset problem, we know there must be some other q ∈ V 0 that is included in the DVO Constrained Subset solution c0 . Additionally, we know there must exist some other vertex r both in V c0 and in the solution to the but q ∈ /V 0 5 c0 \ {p} be the set of vertices in the DVO Constrained Subset instance for which (p, r) ∈ E . Let W = V both the solution to the DVO Constrained Subset instance and the Maximum Edge Subset instance. Then, X X V al((i, j)) < t = t0 ≤ V al((i, j)) (i,j)∈E 0 ∩((W ∪{q})×(W ∪{q}))

(i,j)∈E 0 ∩((W ∪{p})×(W ∪{p}))

and therefore w(q, r) < w(p, r)—a contradiction since p would have contributed more to the CEV of the DVO Constrained Subset instance. Thus, the DVO Constrained Subset Problem is NP-Hard.

4.3

The DVO Intrinsic Subset Problem

At the time of this writing, it remains an open question whether the DVO Intrinsic Subset Problem is NPhard;6 however, we can show that there exists an efficient dynamic-programming algorithm for solving the 5 We can assume the existence of r since p must have an edge to a vertex already in the solution or it would not contribute any weight to the solution thus lowering the overall value. 6 Space does not allow a detailed discussion; however, we do know the Intrinsic Subset Problem too is NP-hard if we allow OR constraints. Briefly: we start with an instance of the subset cover problem and construct an or-based Intrinsic Subset Problem by using a bipartite graph where the parents represent the subsets and the children represent the objects.

9

b

a

I: (2, 4) N: (2, 5) X: (2, 6)

(4, 3)

(2, 6)

I: (5, 6) X: (1, 2)

c

d

b

a

(4, 3)

(5, 2)

c

d

(4, 1)

(a)

(9, 1)

(b)

Figure 5: Sample DVO Graphs where the impact that different value–cost pairs for vertex c have on the optimal subset of vertices will be discussed. Intrinsic Subset Problem when the graph structure is restricted to forests of polytrees, a case that we expect to encounter often in the file-transfer domain. In this section, we present and motivate that algorithm by first discussing some subtleties of intrinsic subsets using a simple example. 4.3.1 The Subtleties of Intrinsic Subsets Because vertices can have negative utility, it may be the case that there are some vertices in the graph that contribute negative utility even in the best case. Unfortunately, identifying these vertices is not as simple as finding all vertices with negative utility and removing them from consideration. To illustrate the complexities of the Intrinsic Subset Problem, consider Figure 5. Note how in both Figure 5(a) and Figure 5(b) vertex c has been labeled with multiple pairs of values and costs. In both graphs, the vertex is labeled with either “I” or “X” and in one case an “N” (intended to represent Include, eXclude, and Not sure) to show how different values and costs of a vertex can effect its desirability as well as the desirability of other vertices in the graph. First, consider Figure 5(a), in which V = {a, b, c, d}. Note that a, b and d all have positive utility while c always has negative utility. Consider case “I” where EU (c|V ) = −2. Both a and b should be included: they have positive utility that is always enabled. As a result, the negative utility of c is outweighed by the positive utility of d, and the optimal set of vertices is all of V . Now, consider case “X”, where EU (c|V ) = −4. The negative utility of c outweighs the positive utility of d and therefore they should both be removed (once c is removed, d is no longer enabled and EU (d|V \ {c}) = −1). Lastly, in the “N” case, the negative enabled utility is equal in magnitude to the positive utility of d. In this case, two different-sized subsets obtain the same utility, so the choice could depend upon an application-specific bias toward smaller or larger vertex sets. For example, when dealing with files, one could make the argument that it is better to keep more files than fewer. On the other hand, one could argue that keeping more files increases the chance for a failure to break the dependency of an enabled vertex.7 Now, consider Figure 5(b). Note that in both cases “I” and “X”, EU (c|V ) = −1, that the value and cost of vertices a and d have changed from Figure 5(a), and finally that in both cases the set of vertices to keep includes b, c, and d. The effect of changing the value and cost from “I” to “X” actually changes the desirability of vertex a—not vertex c. Consider the case where vertex c has the value and cost associated with “I”. Without a, keeping c requires paying the cost of 6 for c. In that case, it is worth paying 4 for including a to get the benefit of c’s value; however, when c has value and cost associated with “X”, it is no longer the case and a should not be included. 7

For the remainder of the paper, we assume the bias is toward larger sets rather than smaller ones.

10

4.3.2 An Algorithm on Polytrees Our algorithm for solving this problem uses a dynamic-programming approach to optimization. We are given an instance of the DVO Intrinsic Subset Problem, which consists of a graph G = (V, E). Nodes in G can have multiple parents and children, but, we assume that the graph is a polytree. Ignoring the direction of the edges in a polytree results in a graph that is a tree—it is singly connected. In a polytree, removing any node from the graph leaves it disconnected into one or more separate polytrees. The observation that removing nodes from polytrees produces smaller polytrees leads to a well-defined decomposition of the graph. If i0 ∈ V is the root of the full graph, the direct neighbors of i0 are each the root of a graph that comes from removing i0 and its incoming edges, thus leaving one disconnected subgraph for each of i0 ’s neighbors. Continuing recursively, every node i ∈ V is the root of its own polytree. It might be helpful to imagine this decomposition as the tree obtained by doing a breadth-first search on an undirected version of the graph G where i0 is at the top and the “children” of this new tree are written below their parents. We define three related functions and show how their utilities can be computed efficiently and used to solve the DVO Intrinsic Subset Problem for polytrees. • U I (i): The maximum utility achievable by including i in the polytree rooted at i (i not enabled). • U X (i): The maximum utility achievable by excluding i from the polytree rooted at i. • U E (i): The maximum utility achievable by including i in the polytree rooted at i (i enabled). If we have the utility of these functions for every node in the polytree, in particular the root i 0 , we know the utility of the optimal solution to the DVO Intrinsic Subset Problem on the original polytree. Computing the subset itself involves recursively checking each node to see whether its maximum utility comes from its inclusion or exclusion—the procedure is standard in dynamic programs. To begin, let us compute the maximum utility achievable by including i in the polytree rooted at i, where i is not enabled. Because i is included, its cost must be included. For each parent node (in the polytree), h, the polytree rooted at h can either include or exclude h—whichever leads to the highest utility. For each child node, j, of i, the polytree rooted at j can either include or exclude j—whatever leads to the highest utility—because i is included and therefore does not un-enable an otherwise enabled child. Mathematically, U I (i) = −Cost(i) X + max(U I (h), U X (h), U E (h)) h∈Par (i)

+

X

max(U I (j), U X (j), U E (j)).

j∈Ch(i)

Here, Ch(i) is the set of children of node i and Par (i) is the set of parents of node i. Next, let us compute the maximum utility achievable by including i in the polytree rooted at i, where i is enabled. Because i is included, its cost must be included. Because it is enabled, its value should also be included. Furthermore, for each parent node (in the polytree), h, the polytree rooted at h must include h, because it is necessary for enabling i. The nodes themselves can either be enabled or not—whichever leads to the highest utility. For each child node, j, of i, the polytree rooted at j can either include or exclude j—whatever leads to the highest utility—because i is included and therefore does not un-enable an enabled

11

(6, 4) U I(1) = −1 UX(1) = 2 UE(1) = 5

1

(2, 1) U I(2) = −1 UX(2) = 0 UE(2) = 1

2

5

7

3

(1, 3) U I (3) = 0 UX(3) = 2 UE(3) = 1

(1, 6) U I (5) = 1 UX(5) = 0 UE(5) = 2

6

4

(4, 3) U I(4) = −3 UX(4) = 0 UE(4) = 1

(2, 4) U I(6) = −3 UX(6) = 1 U E(6) = −1

(8, 5)

(6, 2) U I(7) = −2 UX(7) = 0 UE(7) = 4

8

U I(8) = −5 UX(8) = 0 UE(8) = 3

Figure 6: Example of the DVO Intrinsic Subset Algorithm. The vertices are labeled with U I , U X , and U E as assigned by the algorithm with vertex 1 as the root of the polytree. child. Mathematically, U E (i) =Val (i) − Cost(i) X + max(U I (h), U E (h)) h∈Par (i)

+

X

max(U I (j), U X (j), U E (j)).

j∈Ch(i)

Finally, let us compute the maximum utility achievable by excluding i in the polytree rooted at i. Because i is excluded, its cost and value are irrelevant. For each parent node (in the polytree), h, the polytree rooted at h can either include or exclude h—whichever leads to the highest utility. For each child node, j, of i, the polytree rooted at j can either include or exclude j—whichever leads to the highest utility—but, j cannot be enabled because its parent i is not included. Mathematically, X U X (i) = max(U I (h), U X (h), U E (h)) h∈Par (i)

+

X

max(U I (h), U X (h)).

j∈Ch(i)

An efficient computation can proceed by dynamic programming, which essentially orders the computation so each function value is computed only once and stored for later use. The computation of the values at each node of the graph requires a constant number of operations per adjacent edge, given a standard data structure such as adjacency lists. Once these values have been computed, an optimal decision for each vertex can be made by visiting each vertex according to the graph decomposition (i 0 first) of the polytree and including or excluding each vertex based on its utility values and the decision for its parents. This process is performed in the true topological ordering of the polytree. The decision to include or exclude a vertex is made by comparing the greater of U X with either U I or U E depending on the decision made for any parent vertices. Thus, the asymptotic running time is O(|V | + |E|), which is linear in the size of the initial graph. Figure 6 shows the value of all three functions for the vertices of a sample polytree. The resulting solution to the DVO Intrinsic Subset Problem for this graph is {1, 2, 3, 4, 5, 7, 8} (all nodes but 6). The correctness of this approach follows from the additive definition of cumulative enabled utility and the nontransitivity of the enabling relationship. 12

5 Related Work A complete and exhaustive comparison of the vast literature regarding constraint based methods and/or optimization is well beyond the scope of this paper. Instead, this section is intended to serve as a brief overview of the various methods that may appear to be similar to ours, such as traditional constraint satisfaction methods of Artificial Intelligence [Feldman, 1988]. Our encoding of DVO problems bears a striking resemblance to AND/OR graphs, often used to optimize manufacturing flows or to model problem-solving processes [Nilsson, 1980]. The basic idea behind AND/OR graphs is to label each vertex as an “and” vertex or an “or” vertex to indicate its constraints on its children. The solution to an AND/OR graph instance is the subgraph induced by a subset of vertices for which every child of an “and” vertex and one child for every “or” vertex is included. An optimal solution to an AND/OR graph is subject to a cost function either defined in terms of the maximum cost of an edge between included vertices or the sum of edge costs. In either case, these costs are not defined by vertex weights like in DVO problems and are based on a graph traversal rather than a set of vertices. This subtle, but important, difference enables our DVO formalism to represent situations that AND/OR graphs cannot (and of course vice versa). Specifically, AND/OR graphs are commonly used, for example, in warehousing problems to try to find the best route to take starting and ending at a loading dock while stopping at the location of each good that needs to be loaded. While DVO graphs (currently) cannot be used in this situation, they can be used to select the order to load goods given an unknown capacity or unknown time of departure. Similarly, they can be used to select the proper set of goods to load onto a particular truck. While neither representation is strictly more powerful than the other, the DVO representation enables a new class of problems to be analyzed. Approaches to solving AND/OR graph problems include exact solutions under theoretically unbounded resources as well as minimum time-cost solutions [Desmedt and Wang, 2002] using max flows through networks. Early work showed that best-first graph path search algorithms generalize reasonably well to AND/OR graphs [Chakrabarti et al., 1988]. Similarly, heuristic search methods have been shown to provide an optimal sum-cost solution to path finding given certain constraints on the AND/OR graph [Mahanti and Bagchi, 1985]. Later work applied AND/OR graphs to multi-processor scheduling: edges in the graph represent precedence constraints between processes. Schedules can be computed to minimize a wide array of costs, including runtime [Kwok and Ahmad, 1998] and power [Zhu et al., 2004]. Similarly, depending on how the value and cost labels are assigned to vertices in a DVO graph, various cost metrics can be minimized or value metrics can be maximized. There are additional characteristics of AND/OR graph problems that set them apart from DVOs. For example, the constraints in AND/OR graphs are binding. In other words, if in an instance of an AND/OR graph there is a directed edge between two vertices vi → vj and vj precedes vi in a solution to the instance, then the solution is either invalid or vj must appear again in the sequence after vi . In the DVO case, the solution is still valid, however, the benefit for having vertex v j is not realized until vi is reached in the sequence. There has been recent interest in software multi-agent coordination that is related to our DVO problems. For example, there has been work on analyzing multi-agent environments from a social perspective, such as marriage and dating [Das and Kamenica, 2005] (mostly inspired by the stable marriage problem [Gusfield and Irving, 1989]). This work seeks to model and understand how agents can coordinate their behavior under different assumptions about communication. Multi-agent behavior can be viewed as two separate problems [Hunsberger, 2005]: communicating intent in a context-free environment and communicating goal pursuit once intent has been established. Our work addresses the former problem. We believe it is best framed by the three optimization problems we present, although others [Faltings and Nguyen, 2005, Pearce et al., 2004] view it as a (possibly distributed) constraint satisfaction problem. Still others have begun to examine the relationships between the “structure” 13

of agent environments and optimal behavior [Dignum and Dignum, 2005], an approach also closely related to our work.

6 Concluding Thoughts In this paper, we have introduced a novel use of DAGs to represent a set of real-world situations where there are groups of objects that depend on each other. We have argued for the general applicability of this representation by motivating it with a number of examples, and identifying a taxonomy of three DVO problems. We have also shown that two of the three variants are NP-hard, while the status of the third remains unknown. Although we have restricted ourself to presenting only one algorithm in this paper, we have efficient algorithms for other variants of DVO problems as well. Specifically, we have an optimal solution based upon dynamic programming for the DVO Sequence Problem under a class of failure conditions that assume that the likelihood of failure is monotonically non-decreasing with time, and when the graph is restricted to a forest. We also hope to work toward an efficient approximation algorithm that will allow arbitrary probability distributions to be associated with different sequences of vertices. To approximate a solution to the DVO Constrained Subset problem, we have identified an algorithm that uses simple heuristics similar to those used to deal with traditional knapsack problems. Although a detailed discussion of the algorithm is beyond the scope of this article, it has proven to perform well and efficiently in practice. In general, we intend to focus future work on expanding the applicability of our algorithms to more general constraints (such as those presented in Section 3.3) as well as more general DAG structures other than trees and polytrees. In particular, we hope to identify an efficient algorithm for the general-case DVO Intrinsic Subset Problem or prove that it is NP-hard. Finally, we are focusing our efforts on developing application-specific mechanisms for extracting the structure of DVO graphs under different contexts through observation. For example, it would be useful in the file-transfer problem to learn both the value of a file and its dependences in a variety of contexts automatically. This advance will allow machines or computers to make complex decisions regarding collections of objects or files in the presence of arbitrary dependences with little or no human intervention.

References P. P. Chakrabarti, S. Ghose, and S. C. DeSarkar. Best first search in and/or graphs. In Proceedings of the 1988 ACM sixteenth annual conference on Computer Science, pages 256–261. ACM Press, 1988. Sanmay Das and Emir Kamenica. Two-sided bandits and the dating market. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-05), pages 947–952, 2005. Yvo Desmedt and Yongge Wang. Maximum flows and critical vertices in and/or graphs. In Computing and Combinatorics: 8th Annual International Conference, pages 238–248. Springer-Verlag GmbH, 2002. Virginia Dignum and Frank Dignum. Task and social coordination in agent organizations. In Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems, pages 1183– 1184, 2005. Boi Faltings and Quang Huy Nguyen. Multi-agent coordination using local search. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-05), pages 953–958, 2005.

14

Roy Feldman. Design of a dependency-directed compiler for constraint propagation. In IEA/AIE ’88: Proceedings of the 1st international conference on Industrial and engineering applications of artificial intelligence and expert systems, pages 141–146, New York, NY, USA, 1988. ACM Press. Michael R. Garey and David S. Johnson. Computers and Intractability: A Guide to the Theory of NPCompleteness. W.H. Freeman, 1979. Dan Gusfield and Robert W. Irving. The Stable Marriage Problem: Structure and Algorithms. MIT Press, 1989. Luke Hunsberger. A framework for specifying group decision-making mechanisms. In Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems, pages 1191–1192, 2005. C. Isbell, O. Omojokun, and J. Pierce. From devices to tasks: Automatic task prediction for personalized appliance control. In Personal and Ubiquitous Computing, volume 3, pages 146–153, 2004. Viggo Kann. Maximum edge subgraph. In A compendium of NP optimization problems. 2003. http://www.ensta.fr/ diam//ro/online/viggo wwwcompendium/node46.html. Yi-Kwong Kwok and Ishfaq Ahmad. Benchmarking the task graph scheduling algorithms. In Proceedings of the First Merged International Conference and Symposium on Parallel and Distributed Processing, pages 531–537. IEEE Press, 1998. A. Mahanti and A. Bagchi. And/or graph heuristic search methods. Journal of the ACM, 32:1:28–51, 1985. Warren McCulloch and Walter Pitts. A logical calculus of ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5:115–133, 1943. N. J. Nilsson. Principles of artificial intelligence. Morgan Kaufmann, 1980. Jonathan P. Pearce, Rajiv T. Maheswaran, and Milind Tambe. Dcop games for multi-agent coordination. In CP 2004 Workshop on Distributed Constraint Reasoning (DCR-04), 2004. David L Roberts, Sooraj Bhat, Charles L Isbell, Brian F Cooper, and Jeffrey S Pierce. A decision-theoretic approach to file consistency in constrained peer-to-peer device networks. In Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems, Hakodate, Japan, 2006. Dakai Zhu, Daniel Moss´e, and Rami Melhem. Power-aware scheduling for and/or graphs in real-time systems. In IEEE Transactions of Parallel and Distributed Systems, pages 849–864. IEEE Computer Society, 2004.

15