Learning Goal-Decomposition Rules using Exercises - Semantic Scholar

Learning Goal-Decomposition Rules using Exercises Chandra Reddy & Prasad Tadepalli

Department of Computer Science Oregon State University, Corvallis, OR-97331. USA

freddyc,[email protected]

Abstract Exercises are problems ordered in increasing order of diculty. Teaching problemsolving through exercises is a widely used pedagogic technique. A computational reason for this is that the knowledge gained by solving simple problems is useful in eciently solving more dicult problems. We adopt this approach of learning from exercises to acquire search-control knowledge in the form of goal-decomposition rules (d-rules). Drules are rst order, and are learned using a new \generalize-and-test" algorithm which is based on inductive logic programming techniques. We demonstrate the feasibility of the approach by applying it in two planning domains.

1 Introduction Planning is computationally an intractable task. Usually, control knowledge is used to make planning manageable; it helps a planner in constraining its search for a solution. The research work in learning control knowledge falls between two extremes: supervised speedup learning and unsupervised speedup learning. In supervised speedup learning, the system is provided with examples or solved instances by a teacher (DeJong & Mooney, 1986; Mitchell, Keller, & Kedar-Cabelli, 1986; Shavlik, 1990). In unsupervised speedup learning, the system is given only problems without their solutions (Laird, Rosenbloom, & Newell, 1986; Minton, 1988). In this paper, we explore a middle course of learning from exercises| useful subproblems of natural problems that are ordered in increasing order of diculty.

Teaching problem-solving through exercises is a widely used pedagogic technique. A human teacher selects certain problems and orders them according to their level of diculty to form a sequence of exercises. A human student starts by solving simple problems rst; then attempts harder problems by applying the knowledge gained from solving the earlier problems; and then attempts still harder problems, and so on. This process continues until the student learns the concepts satisfactorily. Machine learning of problem solving using exercises, apart from following this pedagogic tradition, oers a compromise between supervised speedup learning and unsupervised speedup learning. Supervised speedup learning, although more ecient than the latter, places the burden of providing the solutions to the training problems on the teacher| usually a human. Unsupervised speedup learning, in contrast, expects the learner to solve the training problems, while unburdening the teacher. However, this is computationally hard for the learner for it lacks control knowledge and, hence, its only recourse is brute-force search. In the exercises approach, the teacher has the task of providing an exercise set|a sequence of problems ordered by diculty. The learner has to solve the exercise problems using the bootstrapping method akin to the above-described method followed by a human student. The exercises approach for speedup learning has been used for learning control rules (Natarajan, 1989). In this work, we use exercises for learning goaldecomposition rules (d-rules). The learning of control rules in (Natarajan, 1989) is essentially propositional, whereas learning d-rules is rst-order or relational. Moreover, d-rules can be recursive, which complicates solving of exercise problems: to solve a particular goal, the learner should already know something about solving that very goal. Previously, learnability of d-rules in supervised setting|using solved examples|is demon-

strated in (Reddy, Tadepalli, & Roncagliolo, 1996). In this work, we instead use exercises to learn d-rules. Our approach, implemented as a system called XLearn, follows two main steps: For each exercise, (1) an exercise-solver solves the exercise, and outputs the solution (the plan) and the subgoals used; and (2) a rst-order inductive learner forms an example drule using the initial state as the d-rule condition and the subgoals as the d-rule subgoals, and then uses a \generalize-and-test" algorithm to include this example d-rule in the previous hypothesis d-rules. In the generalization phase, the generalize-and-test algorithm nds the least general generalization ( ) of the example d-rule with previously formed similar hypothesis d-rules. In the test phase, to ensure appropriate generalization, the algorithm generates examples of the hypothesized d-rule and tests whether the examples are correct. The rest of the paper is organized as follows. Section 2 provides a short introduction to d-rules. Section 3 discusses the learning approach and the related algorithms. Section 4 discusses the results of experiments. Section 5 concludes with a discussion of related issues and future work. lgg

2 D-rule Representation Macro operators and control rules are two prominent methods of representing control knowledge (Minton, 1988). D-rules is another method for this purpose. Representation. A d-rule is a 3-tuple h i that decomposes a goal into a sequence of subgoals , provided the condition holds. The goal component is a single literal. The condition component and the subgoals component are conjunctions of literals. To represent subgoal order, , in addition, has an implicit left-to-right ordering. All literals in a drule are rst-order function-free positive literals. A requirement on the subgoals component, , is that each variable in it occurs also either in or in . A d-rule, in STRIPS-world domain, looks like this: g; c; sg

g

sg

c

g

c

sg

sg

sg

g

g: c:

c

(in-room robot ?rg) ((in-room robot ?rst) (left-of ?rfr ?rg) (left-of ?rst ?rg) (door ?d ?rfr ?rg) ) (in-room robot ?rfr) (in-room robot ?rg)

sg: h

i This d-rule can be read as follows: to reach the room ?rg, starting from ?rst, go to the room ?rfr that is immediately to the left of ?rg, and then enter the ?rg room. This rule is recursive in the sense that, to reach ?rfr, the planner may have to make use of the same rule. A d-rule may also decompose a goal into prim-

itive operations which can be readily executed. For example, going from the ?rfr room to the neighboring ?rg room, the robot needs to open the connecting door and enter, which are primitive operations. D-rules are a specialization of hierarchical transition networks of (Erol, Hendler, & Nau, 1994), the only dierence being that the subgoals are totally ordered in d-rules. D-rule representation compares favorably against macro operators and control rules, in terms of computational complexity of planning when there is a loosely coupled set of goals and subgoals. D-rule planner. The d-rule-based planner for a domain takes as input a goal, a state from which the goal needs to be achieved, a set of d-rules, and the domain theory (comprising primitive-operator de nitions, and background theory). The planner nds the d-rules that correspond to the goal, and picks a d-rule whose condition is satis ed in the current state. The chosen d-rule suggests a subgoal sequence. The suggested subgoals in the sequence are achieved one by one. Each subgoal in turn becomes a goal, and has its own d-rules. So, the planner achieves goals recursively, until the recursion bottoms out when a d-rule suggests only primitive operators as subgoals. Primitive operators are operators of the domain which are readily executable without recourse to a d-rule. The planner outputs the plan (the operator sequence) and the nal state, in which the goal should be satis ed. Since d-rules require their subgoal components to have only the variables in the condition and goal components, when the planner is trying to achieve a particular goal and the condition of a d-rule has been satis ed, the subgoals will all be ground. Moreover, the subgoals are ordered. This makes planning ecient. However, in general, there may be multiple d-rules for a goal, and their conditions may not be disjoint. This presents a choice for the planner. There are four design options here: (1) require that d-rules be disjoint; (2) if not, require that any d-rule choice for a goal achieve the goal; (3) allow backtracking; or (4) have control rules that know how to choose among dierent d-rules when there is a choice. Allowing backtracking makes the planner inecient. We assume that the set of drules the planner gets is \good"|i.e., any applicable d-rule achieves its goal. With good d-rules, planning is linear in the number of d-rule applications, and, hence, in the solution or plan length. The d-rule planner can be incomplete if the d-rules are not perfect; but it is always ecient.

3 Learning from Exercises Learning from exercises requires that the teacher give a sequence of problems ordered according to their level of diculty: from least dicult problems which come rst, to most dicult problems which come last. This is similar to the way exercises are presented at the end of a lesson in a textbook on mathematics. In the examples framework, on the other hand, the examples the learner gets are randomly chosen from a natural distribution. However, in the exercises framework, the burden of solving the input problems is on the learner, unlike in the examples framework where that burden is on the teacher. We have not yet de ned what a problem's level of diculty means. In (Natarajan, 1989), the solution length of a problem is chosen as the level of diculty|i.e., the problems with higher solution length are more dif cult. This, however, does not work for d-rules. A d-rule, depending on the problem it is applied to, may produce varying solution lengths. Therefore, exercises are ordered, instead, according to a bottom-up examination of the goal-subgoal hierarchy of the domains. That is, an exercise related to a goal in the hierarchy is less dicult than an exercise related to goals that are higher in the hierarchy. This, however, does not work when there is recursion among goals, for there is no strict hierarchy then. In the case of recursion, exercises for base cases are considered less dicult than the exercises for more general recursive cases. X-Learn

dRuleSet fg hstate; goali Next-Exercise() /*next problem*/ while hstate;goali is not FAIL do f depthLimit MIN-DEPTH done FALSE while depthLimit