On Designing CO$T: A New Approach and Programming Environment for Distributed Problem Solving Based on Evolutionary Computation and Anytime Algorithms Eugene Eberbach
Andrew Eberbach
Computer and Information Science Dept. University of Massachusetts North Dartmouth, MA 024747-2300 Email:
[email protected]
School of Computer Science University of Waterloo Waterloo, ON N2L 3G1 Email:
[email protected]
Abstract— In this paper we present a unified view of AI inspired by ideas from Evolutionary Computation as design of bounded rational agents. The approach specifies optimal programs rather than optimal actions, and is based on process algebras and anytime algorithms. The search method described in this paper is so general than many other search algorithms, including evolutionary search methods, become its special case. In this paper, we present a practical design of the programming language and environment targetting real-time complex domains. As AI systems move into more complex domains, all problems become real-time, because the agent will never have long enough time to solve the decision problem exactly.
I. I NTRODUCTION : P ROBLEM S OLVING AND E VOLUTIONARY C OMPUTATION Turing Machines and algorithms are two fundamental concepts of computer science and problem solving. Turing Machines describe the limits of algorithmic problem solving, and laid the foundation of current computer science in the 1960s. It turns out that undecidable problems cannot be solved by TMs and intractable problems are solvable, but require too many steps. For undecidable problems effective recipes do not exist - problems are called nonalgorithmic or nonrecursive. On the other hand, for intractable problems algorithms exist, but running them on a deterministic Turing Machine, requires an exponential amount of time (the number of elementary moves of the TM) as a function of the TM input. Evolutionary Computation (EC) is used primarily for the solution of intractable problems and it can be understood as a probabilistic beam search, looking for the solution with a globally optimal value of the fitness function. The fitness function represents the quality of the solution, and is obtained by the process of interaction of the solution with its environment. To do it, EC uses evolutionary algorithms with problem specific representation, variation and selection operators, fitness measure, initialization and termination condition. Populations of solutions and (possibly) evolutionary algorithms are changed in each generation, and the probabilistic search for a solution is, in a general, an infinite process.
When the best solutions are preserved from generation to generation (the so-called elitist strategy), evolutionary computation is guaranteed to converge to its goal in the infinity [13]; otherwise, there is no guarantee that the goal will be reached. Usually, evolutionary computation is terminated after a finite number of generations, producing approximate solutions. The halting of genetic algorithms is enforced by the necessity to fit them to the Turing Machine model of computation, with its finiteness restrictions. EC is related both to random-restart hill climbing and simulated annealing, i.e., the approaches which do not keep the entire search tree, but unlike to hill climbing approach, it allows to overcome partially the problem to be trapped in local optima. EC consists of four main subareas Genetic Algorithms (GA), Genetic Programming (GP), Evolution Strategies (ES), and Evolutionary Programming (EP). Some researchers consider GP to be a special case of GA. However, even if this is correct, GP outgrew GA into its own separate area and became at least equally distinct from other EC subareas in a similar way as GA, ES and EP differ among themselves. Thus GP became de facto a separate subarea of EC. It is interesting that the one of the main concepts of nonalgorithmic (by its nature) Evolutionary Computation, is called an algorithm (more precisely, an evolutionary algorithm). This describes how strong and dominant role Turing Machines and algorithms play in computer science. Formally, an evolutionary algorithm is not an algorithm at all, because it does not terminate (for elitist strategies, it guarantees to terminate in an infinite number of steps). To fit it to the old “algorithmic” approach - an artificial (also called pragmatic) stop criterion has had to be added (see e.g., Michalewicz’s definition [10]). The evolutionary algorithm, to remain algorithmic, has to be stopped after a finite number of generations or when no visible progress is observable. Definition 1: A generic Evolutionary Algorithm can be described in the form of the functional equation working in a
simple iterative loop in discrete time t [10]: x[t + 1] = s(v(x[t])), where x - is a population under a representation (e.g., fixed binary strings for GAs, Finite State Machines for EP, parse trees for GP, vectors of reals for ES), s - is a selection operator (e.g., truncation, proportional, tournament), v - is a variation operator (e.g., variants of mutation and crossover), x[0] - is an initialization of population, f (x[t]) - is a fitness function - its optimum describes the halting criterion for evolution of x. The above equation is applicable to main subareas of EC: GA, EP, ES, GP. Sometimes only the order of variation and selection are reversed, i.e., selection is applied first, and variation second. Variation and selection depend on the fitness function. The termination condition is to find a population with a global maximum fitness of one of individuals in the population. Usually, this is not possible to achieve (or verify), and EC terminates after a predefined number of iterations, or when no more improvements are observed through several generations. It is possible to think and implement more complex variants of evolutionary algorithms, like for instance Reynold’s cultural algorithm, or Michalewicz’s Genocop algorithm, or Ant Colony System, or Particle Swarm Optimization [10]. Evolutionary algorithms evolve population of solutions x, but they may be the subject of self-adaptation (like in ES) as well. In general, every evolutionary algorithm EA can be encoded as an instance of TM M operating on population x, i.e., pairs (M, x) consisting of a Turing machine M encoding of EA (of course, for every algorithm, this is possible) and its input x being the specific input of the Universal Turing Machine UTM. Then a Universal Evolutionary Algorithm represents instances of Turing Machines from a Universal Turing Machine representing all possible evolutionary algorithms. In other words, the Universal Evolutionary Algorithm consists of all pairs (M, x), where M is TM encoding of evolutionary algorithm, and x is its input population. A formal model of EC - an Evolutionary Turing Machine and a Universal Evolutionary Turing Machine have been introduced in [3], [5]. Definition 2: By a Universal Evolutionary Turing Machine UETM we mean a series of (possibly infinite) instances of Universal Turing Machines (M [t], x[t]): (M [t], x[t])t=0,1,2,... = [(M [0], x[0]), (M [1], x[1]), ...], where • each M [t] represents (encodes) an evolutionary algorithm with population x[t], and evolved in generations t = 0, 1, 2, ...; (in the most common case M consists of the composition of variation v followed by selection s), • the outcome of generation t = i, i = 0, 1, 2, ... is the pair (M [i + 1], x[i + 1]), and
the goal (or halting) state of U ET M is represented by the optimum of fitness performance measure f (x[t]). Note that because the fitness function can be the subject of evolution as well (it is a part of TM encoding M ), evolution is an infinite process. Note also, that by definition, evolution is a self-adaptive process, i.e., an evolutionary algorithm M [t] is changeable as well (currently true only for ES, but nothing stops us to think about adaptive evolutionary algorithms, i.e., to allow evolution to evolve itself). However, in most cases (at least now), an evolutionary algorithm M is fixed and does not change in time. We are convinced that computer computations are not restricted to algorithms only. An algorithm should consist of a finite number of steps, each having well defined and implementable meaning. We can suspect that Evolutionary Computation being in general infinite, violates a classical definition of algorithms, i.e., may lead to non-algorithmic computations. Definition 3: By superTuring computation we mean any computation that cannot be carried out by a Turing Machine as well as any (algorithmic) computation carried out by a Turing Machine. In [6], [7], [15], several superTuring models have been discussed and overviewed, including Turing’s o-machines, cmachines and u-machines, cellular automata, discrete and analog neural networks, Interaction Machines, Persistent Turing Machines, Site and Internet Machines, the π-calculus, and the $-calculus. An Evolutionary Turing Machine, proposed as a formal model of Evolutionary Computation, is another superTuring model. SuperTuring models derive their higher than the TM expressiveness using three principles: interaction, evolution, or infinity. In the interaction principle the model becomes open and the agent interacts with either a more expressive component or with an infinite many components. In the evolution principle, the model can evolve to a more expressive one using nonrecursive variation operators. In the infinity principle, models can use unbounded resources: time, memory, the number of computational elements, an unbounded initial configuration, an infinite alphabet, etc. The details can be found in [5], [7], [15]. Evolutionary Computation applies both the evolution and infinity principles to gain its higher than TM expressiveness, i.e., possibly operating on infinite populations, or using an infinite number of generations, or using variation operators generating nonrecursive solutions. The potential for higher expressiveness of EC is associated with the representation used to encode individual solutions. For example, it is difficult to expect higher expressiveness from GAs, operating on fixed length binary strings (there the search space is finite). However, cellular programming evolves cellular automata, EANNs evolve neural networks (consisting of potentially infinite number of cells, or neurons, respectively), GP evolves potentially unbounded parse trees, and ES evolves vectors of reals i.e., all operating on potentially infinite objects, which by the infinity principle can lead to higher expressiveness than TM. •
In practical applications, they are trimmed to finite solutions, because is is difficult to implement an applicable way to evolve infinite objects, and additionally, we simulate them on the conventional finite von Neumann computers. In this paper, we describe one of the superTuring models, the $-calculus of which Evolutionary Computation is its special case. In section 2, we describe the syntax of the $-calculus, and its $-operator resembling EC fitness. In section 3, its search engine for problem solving is presented. It performs a very general types of search, called the kΩ-optimization which allows to simulate many other search algorithms (including an evolutionary algorithm). In this paper, we omit theoretical rather (at least at this moment) investigations on undecidable problems, and we describe design and implementation of the programming languages targetting intractable problems. Two such languages are under implementation: a generalpurpose problem solving CO$T language, and specialized CCL language designed for distributed mobile robotics. In section 5, preliminary results and conclusions are presented. II. T HE $-C ALCULUS : A P ROCESS A LGEBRA FOR D ISTRIBUTED P ROBLEM S OLVING BASED ON A NYTIME A LGORITHMS The $-calculus is a mathematical model of processes capturing both the final outcome of problem solving as well as the interactive incremental way how the problems are solved. The $-calculus is a process algebra of Bounded Rational Agents for Interactive Problem Solving, targetting intractable and undecidable problems. It has been introduced in the late of 1990s [1], [2], [6], [7], [15]. The $-calculus (pronounced cost calculus) is a formalization of resource-bounded computation (called also anytime algorithms, proposed by Dean, Horvitz, Zilberstein and Russell in late 1980s and begin of 1990s [9]). Anytime algorithms are guaranteed to produce better results if more resources (e.g., time, memory) become available. The standard representative of process algebras, the π-calculus [11] is believed to be the most mature approach for concurrent systems. The $-calculus rests upon the primitive notion of cost in a similar way as the π-calculus was built around a central concept of interaction. Cost and interaction concepts are interrelated in the sense that cost captures the quality of an agent interaction with its environment. The unique feature of the $-calculus is that it provides a support for problem solving by incrementally searching for solutions and using cost to direct its search. The basic $-calculus search method used for problem solving is called kΩ-optimization. It is a very general search method, allowing to simulate many other search algorithms, including A*, Minimax, dynamic programming, tabu search, or evolutionary algorithms. The $-calculus allows in a natural way to express evolution. Its cost performance measure allows to model fitness function, and an infinite search for global optima. Additionally, $calculus allows both to optimize the quality of solutions and to minimize search (problem solving) cost.
A. Syntax In $-calculus everything is a cost expression: agents, environment, communication, interaction links, inference engines, modified structures, data, code, and meta-code. $-expressions can be simple or composite. Simple (contract) $-expressions α are considered to be executed in one atomic indivisible step. Composite (interruptible) $-expressions P consist of distinguished components (simple or composite ones) and can be interrupted. Definition 4: The $-calculus The set P of $-calculus process expressions consists of simple $-expressions α and composite $-expressions P , and is defined by the following syntax: α
::= |
($i∈I Pi ) (→i∈I a Pi )
cost send Pi with evaluation through channel a | (←i∈I a Xi ) receive Xi from channel a | (0i∈I Pi ) suppress evaluation of Pi , double suppression causes evaluation of Pi exactly once | (ai∈I Pi ) defined call of simple $-expression a with parameters Pi negation of defined call of simple | (ai∈I Pi ) $-expression a P ::= ( ◦ i∈I Pi ) sequential composition | ( k i∈I Pi ) parallel composition ∪ cost choice | ( ∪ i∈I Pi ) + P ) adversary choice | (∪ i∈I i | (ti∈I ( ◦ αi Pi )) general choice | (fi∈I Pi ) defined process call f with parameters Pi , and its associated definition (:= (fi∈I Xi ) P ) with body P The indexing set I is possibly countably infinite. In the case when I is empty, we write empty parallel composition, general and cost choices as ⊥ (blocking), and empty sequential composition as ε (invisible transparent action, which is used to mask, make invisible parts of $-expressions). Adaptation (evolution/upgrade) is an essential part of $-calculus, and all $-calculus operators are infinite (an indexing set I is unbounded). $-calculus agents interact through send-receive pair as the essential primitives of the model. Sequential composition is used when $-expressions are evaluated in a textual order. Parallel composition is used when expressions run in parallel and it picks a subset of non-blocked elements at random. Cost choice is used to select the cheapest alternative according to a cost metric. Adversary choice is used to select the most expensive alternative according to a cost metric. General choice picks one non-blocked element at random. General choice is different from cost and adversary choices. It uses guards satisfiability. Cost and adversary choices are based on cost functions. Call and definition encapsulate expressions in a more complex form (like procedure or function definitions in programming languages). In particular,
they specify recursive or iterative repetition of $-expressions. Simple cost expressions execute in one atomic step. Cost functions are used for optimization and adaptation. The user can use built-in standard cost function, or is free to define his/her own cost metrics. Send and receive perform handshaking message-passing communication, and inferencing. Additionally, a user is free to define her/his own simple $expressions, which may or may not be negated. B. The Performance $ Measure Operator Cost (utility) functions represent a uniform criterion of search in problem solving. They can be defined in many ways. Costs have their roots in the von Neumann/Morgenstern’s utility theory [14]. The main (but rather technical) difference compared to the utility theory is that we perform uniformly in the style of control theory rather minimization of costs than maximization. In the utility theory and the decisiontheoretic metareasoning [14] from gains are subtracted costs, in $-calculus the opposite is true: from costs are subtracted gains. Costs/utilities may have many possible interpretations and formulas. For this reason no single definition is enforced by the calculus, Nevertheless, typical costs are functions representing time, money, fitness, error, probability, possibility, belief, complexity, distance, energy, entropy, resources, etc. Costs representing probabilities of cost expressions lead to a probabilistic $-calculus, and the costs interpreted as the fuzzy set characteristic function - lead to a fuzzy-logic $-calculus. Performance of multiple agents or the influence of several factors can be combined in one cost for multiobjective optimization, or they can be kept separate as a vector of component costs. In multimodal optimization we look for multiple optima for a given cost function. Costs may be positive, and/or negative (payoff, gains). Any domain where a linear order is defined is feasible. However, most commonly real numbers are used. Normalized costs are defined on unit interval [0, 1]. Having both positive and negative costs allows all optimization problems to be expressed uniformly in $-calculus as cost minimization. Instead to change the sign of the cost function, for maximization problems if there exists an upper bound on the objective function it is always possible to convert it to a minimization problem by subtracting from the upper bound the objective function, and using the result as a current cost function. If we are interested to have only nonnegative costs, it is possible to shift all costs to nonnegative values by adding the absolute value of the lower bound of the cost function (pending that it exists). III. S EMANTICS : S EARCHING FOR THE B EST S OLUTIONS WITH M INIMAL C OSTS A. Completeness, Optimality and Total Optimality of Search An ideal rational (intelligent) agent is one that always takes the action that is expected to optimize its performance measure, given the percept sequence it has seen so far [14]. However, it requires an unrealistic assumption that an agent
has unlimited resources (e.g., infinite speed of computation, or infinite memory). Unfortunately, most of the real problems requiring intelligence are computably undecidable or intractable (either unsolvable or exponentially complex), thus perfect optimization required for intelligence is out of the question. A limited (bounded) rational agent is one that always takes the action that is expected to optimize its performance measure, given the percept sequence it has seen so far and the limited resources one has [14]. A bounded rational agent solves constraint optimization problems, taking into account both the quality of a solution and the costs of resources used to obtain it. Bounded optimality fits intuitive idea of intelligence and provides a bridge between theory and practice. Intelligence of an agent can be judged on the basis of cleverness of its search strategy/algorithm for the best (least costly) solution. Search algorithms are evaluated on the basis of their • Completeness: is the strategy guaranteed to find a solution when there is one? • Optimality: does the strategy find the highest quality solution when there are several different solutions? • Search Cost: how much resources (e.g., how long does it take and how much memory) are used to find a solution? • Total Optimality: does the strategy find the highest quality solution with optimal search cost? Search can be cooperative, competitive, or random, and it may involve single or multiple agents. Search can be offline (the complete solution is computed and then executed by an agent without perception), or online (an agent interleaves computation and actions). B. The kΩ-Search as a Generalization of Evolutionary Search The basic $-calculus search method used for problem solving is called kΩ-optimization. It is a very general search method, allowing to simulate many other search algorithms, including A*, Minimax, dynamic programming, tabu search, or evolutionary algorithms [10], [14]. The problem solving works iteratively: through select, examine and execute phases. In select phase the tree of possible solutions is generated up to k steps ahead, and agent identifies its alphabet of interest for optimization Ω. This means that the tree of solutions may be incomplete in width and depth (to deal with complexity). However, incomplete (missing) parts of the tree are modeled by silent $-expressions ε, and their cost estimated (i.e., not all information is lost). The above means that kΩ-optimization may be if some conditions are satisfied to be complete and optimal. In the examine phase the trees of possible solutions are pruned minimizing cost of solutions, and in the execute phase up to n instructions are executed. Moreover, because $ operator may capture not only the cost of solutions, but the cost of resources used to find a solution, we obtain a powerful tool to avoid methods that are too costly, i.e., $-calculus directly minimizes search cost. This basic feature, inherited from anytime algorithms, is needed to tackle directly hard optimization problems, and allows to solve total optimization
problems (the best quality solutions with minimal search costs). The kΩ-search can be used both for single and multiple cooperative or competitive agents working online (n 6= 0) or offline (n = 0). Given: •
•
•
•
•
•
•
A - an alphabet of simple $-expressions of the universe consisting of an enumerable Snumber of agents Ai (including an environment), A = i Ai , Ai ∩Aj = ∅, i 6= j. The agent population size will be denoted by p = 1, 2, ..., ∞. Ai - an alphabet of simple $-expressions of the i-th agent, Pi [0] ∈ P - an initial $-expression, and optional (explicit) goals Qi ⊆ P, an implicit goal is min($ Pi [0]). Ωi ⊆ A - a scope of deliberation/interests of the i-th agent, i.e., a subset of the universe’s simple $-expressions chosen for optimization. All elements of A−Ωi represent an environment or irrelevant part of an agent(s), and will become invisible (replaced by ε), thus either ignored or unreachable for a given agent (makes optimization local spatially). Expressions over A − Ωi will be treated as observationally congruent (cost of ε becomes neutral, e.g., typically equal 0). All expressions over Ωi − Ai will be treated as strongly congruent - they will be replaced by ε and although invisible, their cost will be estimated using best available knowledge of an agent (may take arbitrary values from the cost function domain). $i - a cost function performance measure (selected from the library or user defined). Typically a user provides cost of simple $-expressions or an agent can learn such costs (e.g., by reinforcement learning). The user selects or defines also how the costs of composite $-expressions will be computed. Typically, for example $( ◦ P Q) = $(P )+ ∪ + $(Q), $( ∪ P Q) = min($(P ), $(Q)), $( ∪ P Q) = max($(P ), $(Q)), $(t P Q) = p∗$(P )+(1−p)∗$(Q), where p is the probability to choose P , and so on. The cost of the search tree is the sum of its components: costs of nodes and arcs. This allows to express both the quality of programs (rewards) and search cost. bi = 1, 2, ..., ∞ - a branching factor of the search tree, i.e., the maximum number of generated children for a parent node. For example, hill climbing has bi = 1, for binary tree bi = 2, and bi = ∞ is a shorthand to mean to generate all children (possibly infinite many). ki = 0, 1, 2, ..., ∞ - represents the depth of deliberation, i.e., the number of steps in the derivation tree selected for optimization in the examine phase (decreasing ki prevents combinatorial explosion, but can make optimization local in time). ki = ∞ is a shorthand to mean to the end to reach a goal (may not require infinite number of steps). ki = 0 means omitting optimization (i.e., the empty deliberation) leading to reactive behaviors. Steps consist of multisets of simple $-expressions, i.e., a parallel execution of one or more simple $-expressions constitutes one elementary step. ni = 0, 1, 2, ..., ∞ - the number of steps selected for
execution in the execute phase. For ni > ki steps larger than ki will be executed without optimization in reactive manner. For ni = 0 execution will be postponed until the goal will be reached. Find: $-expression with min($ Pi [0]) (default goal). The kΩ-Search The kΩ search is itself a complex cost expression consisting of init, loop, sel, exam, exec, goal, $, and constructing solutions from predefined and user defined simple and complex $expressions. Each i-th agent performs the following kΩ-search procedure: (:= kΩsearch ( ◦ (init Pi [0]) /* initialize Pi [0] */ (loop Pi )) /* basic cycle: select, ) examine, execute */ where loop is the select-examine-execute cycle performing the kΩ-optimization until the goal is satisfied. At that point, the agent re-initializes and works on a new goal in the style of the never ending reactive program: (:= (loop Pi ) /* loop recursive definition */ /* goal not satisfied, default (t ( ◦ (goal Pi ) goal min($ Pi [0]) */ (sel Pi ) /* select: build search tree k step deep, b wide */ (exam Pi ) /* examine: prune search tree ∪ and in in cost ∪ + choices */ adversary ∪ (exec Pi ) /* execute: run optimal Pi n steps */ (loop Pi )) /* return back to loop */ ( ◦ (goal Pi ) /* goal satisfied kΩsearch) re-initialize search */ ) More details of the kΩ-search components are presented below 1) Initialization (init Pi [0]): Select search parameters ki , bi , ni , Ωi . Generate an initial $-expression Pi [0]) over alphabet Ai ∪ Ωi either in the style of evolutionary computation (i.e., randomly), or as an imperative program. It can be a single $expression or the population (vector) of $-expressions. Define simple and complex $-expressions used in construction of solutions, define or select from the library the cost function $i , define costs of simple $-expressions, and define optionally the goal of computation (goal Pi ). Initialization resulting in the empty cost expression Pi [0] = ⊥, or for ki = ni = 0 will halt the kΩ-search (until interrupt, which will allow for re-initialization). For the depth of deliberation ki = 0, the kΩ-search will work in the style of imperative programs (reactive agents), executing up to ni consecutive steps in each loop iteration. For ni = 0 search will be offline, otherwise for ni 6= 0 - online. 2) Goal (goal Pi ): The default goal is min(Pi [0]). Because the default
goal may require an infinite number of iterations, it can be supplemented or overwritten by another stopping criterion like the maximum number of select-examineexecute cycles, the lack of the improvement, etc. 3) Select Phase (sel Pi ): Skip Select and Examine Phase if ki = 0 and go directly to Examine Phase. Generate a search tree with the branching factor bi and ki steps deep (at most) over alphabet Ai ∪ Ωi starting from current state Pi . In the construction of the derivation tree replace (perhaps by exchanging messages with other agents) $-expressions hidden by ε in previous select phase to the real values and learn their costs. In the expansion replace all simple $-expressions which do not belong to Ωi by ε (i.e. hide them - make them invisible - they will be treated as observationally congruent with neutral cost (e.g., equal 0). Replace also $-expressions which are at boundary depth larger than ki , or belonging to Ωi − Ai by ε, and estimate their costs, i.e. they will be treated as strongly congruent. 4) Examine Phase (exam Pi ): For strong congruence, i.e., actions belonging to Ωi , use estimates or precise values of costs for hidden (by ε) $-expressions (these are expressions belonging to other agents or outside the depth or width of the search). For observation congruence, i.e., actions performed by the agent but outside of its interest Ωi , use ($ ε) = 0. Prune the search tree by selecting paths with minimal cost in cost choices and with maximal cost in adversary choices. Ties are broken randomly. 5) Execute Phase (exec Pi ): • If ni = 0 (offline search) and goal has not been reached in the Select/Examine Phase, pick up the most promising leaf node of the tree (with minimal cost) for expansion, i.e., make it a current node and return to Select Phase. • If ni = 0 (offline search) and goal has been reached in the Select/Examine Phase, execute optimal Pi up to the goal node. • If ni 6= 0 (online search), execute optimal Pi up to at most ni steps. If execution interrupted, and ni 6= 0, decrease ni by the value proportional to the remaining steps or if ni = 0 decrease ki . If execution not interrupted increase ni pending 0 < ni ≤ ki . If ni = ki increase ki . If cost of search (cost of sel and exam) too large decrease ki and/or bi , otherwise increase it. Modify other parameters of kΩ-search, including costs of simple $-expressions, Ωi if they have changed. The ideal (but unrealistic) situation would be if the agent worked with k = b = ∞, i.e., to consider a complete search tree. However, in the complex domains this is not possible, and k and b have to be small. To compensate for that, the estimates of missing parts (ε) are used. Also non-algorithmic solutions of undecidable problems (not the subject of this
paper), require either truly infinite search trees (with an infinite width and/or depth) or with non-implementable (oracle-type) subtrees/branches. C. The Power and Versatility of kΩ-search: Simulating Other Search Algorithms Although, it is clear from the theory of computation that on average all optimization search algorithms perform the same (see e.g., the No Free Luch Theorem [16], and no best search algorithm exists, search methods can be indefinitely improved [5], and they can be more or less generic. Because in most cases it is not clear which search algorithm will be the best, the kΩ-search ability to switch from one method to another is a very powerful feature indeed. Without the claim that the kΩ-search allows to cover all possible search methods, we will show that it allows to simulate most typical search algorithms (perhaps, for the price of efficiency compared to specialized search algorithms). In all search methods presented, if we set costs of goal, sel, examine to 0, we will look for the best solutions only (where costs of solutions represent cost of execution of $-expressions, $(exec) 6= 0), otherwise for nonzero costs of goal, sel, examine the kΩ-search will look for total optima (the best quality solution with minimal search cost). For example, the kΩ-search can simulate many other search algorithms (using the same code but changing its control parameters and some function definitions): • Breadth-First-Search: single agent, i.e., p = 1, k = 1, b = ∞, n = 0, costs are either irrelevant, or, alternatively, the same for all $-expressions. • Depth-First-Search: p = 1, k = ∞, b = 1, n = 0, costs are irrelevant, or, alternatively, the same for all $expressions. • A*: single agent, i.e., p = 1, k = 1 (expand search tree one step ahead), n = 0 (offline search - postpone execution until goal is found), b = ∞ (all children are expanded for a selected node), Ω = A. • Minimax, Expectiminimax, Alpha-Beta Search: two agents p = 2, i ∈ {1, 2} executing its own i-th algorithm, ki = bi = ∞ (expand tree up to the end), bi = ∞ for minimax and expectiminimax (consider all children) and bi ≤ ∞ for alpha-beta, ni = 1 (execute only one step), Ω1 = Ω2 = A1 ∪ A2 . • Hill Climbing: p = 1, k = 1, b = 1, n = 0, only child node survives and parent is forgotten. • Simulated Annealing: p = 1, k = 1 (look one step ahead only), b = 1 (generate one child only replacing parent if child is better or if worse - with the Boltzmann probability distribution), n = 0. • Evolutionary Algorithm: p = 1, k = 1 (apply crossover and mutation to modify parent population once only, after selection only child survives and parent node is forgotten), b = 1 (only one population is considered, if population takes the form of binary string vectors we have a genetic algorithm, for vector of reals - evolution strategy, for vectors of transition and output relation matrices
•
•
•
•
•
of finite state machines - evolutionary programming, for vectors of parse trees (with their own depth and width) genetic programming), n = 0. Tabu Search: p = 1, k = 1, b ≤ ∞ (all children having cost ∞ will form a tabu list, i.e., they never will be selected for execution). Dynamic Programming: p = 1, k = ∞ (for value iteration k = const), b = ∞, n = 0. The cost of a state (a node of a search tree) is equal to the immediate reward/cost for that state plus the discounted cost of the state sequences that might follow it. Swarm Intelligence: p = 1, k = 1, b = 1, n = 0, for initialization we consider vector of populations, each representing particle swarm; each individual has abstract state/position modified with abstract speed. Each individual updates its state relativey to its previous best and neighborhood best state. All these updates are done synchronously - all particles share the same clock/generations. In reality, more natural would be to model individuals as multiple agents (p ≥ 1) exchanging information with neighbors by send/receive primitives and working asynchronously with different speed. Ant Colony System: p ≥ 1 - multiple agents/“ants”, k = 1, b = 1, n = 0, multiple “ants” work with their own clock and exchange information by message passing in the form of abstract pheromone. Neural Networks: p = 1 (muliple neurons, but typically only weights are updated by a single program in consecutive iterations, i.e., usually topology of the network is fixed), k = 1 (update weights once in each loop), b = 1 (do not remember previous weights/parent node), n = 0 (postpone work until learning is done).
IV. O N D ESIGN AND I MPLEMENTATION OF CO$T AND CCL: A N EW P ROGRAMMING L ANGUAGE AND E NVIRONMENT FOR D ISTRIBUTED P ROBLEM S OLVING Our new experimental programming languages CO$T (a generic Java implementation by the second author) and CCL (a specialized C implementation [4], [8]) are both derived from the $-calculus process algebra, and are designed to provide the support for distributed problem solving based on ideas of anytime algorithms and evolutionary computation. Our approach is not tied to one specific programming paradigm. Although it has some similarities with objectoriented, procedural, functional and logic paradigms - it is rather independent from one specific paradigm. In fact, it can be considered as a new “cost” programming paradigm, where each statement of the language has its associated cost, and instructions with the smallest cost are those selected for execution (i.e., the execution is “cost-driven”). Note that in spite of current popularity of the object-oriented approach, a universally best programming paradigm does not exist, and the history of computer science languages confirms that. Thus “cost” programs can be designed in styles of other programming paradigms: the encapsulation of code for agents resembles objects, the “bytecode” of $-expressions with infix
notation resembles Lisp functions or genetic programming, the inference search engine performing kΩ-search resembles Prolog, and initial cost expressions are written as a procedural/imperative code. For portability reasons, the new language will be implemented, similar to Java or Parle from the ESPRIT Fifth Generation SPAN project [12], in two steps: as a compiler and an interpreter. The source code through an operator GUI, providing a user-friendly interface, will be compiled to the intermediate “byte-like” code of $-expressions, and downloaded to the interpreter. The interpreter’s Virtual Machine will dynamically search for the best executable code and execute it employing the kΩ-optimization. Interpreters can run on shared memory architectures (using multithreading) or on distributed computers/robots/agents. A benefit to implementing the first step as a compiler is the efficiency of compilation and user-friendliness of its GUI (hiding from the user nested $-expressions that resemble to some degree the output generated by genetic programming). On the other hand, using the intepreter as a second step, makes it more flexible for online search algorithms for real-time dynamic environments. Additionally, adding the intermediate level, makes our design scalable and portable. A source program will consist of one or more agent definitions. Each agent will consist of • An initial state (for init) in the form of an arbitrary $expression (can be user defined or automatically generated by init). • The cost function $ whose minimum defines the default goal, and optionally supplemented by another termination condition (for goal). • A set of atomic and composite $-expressions to construct the kΩ-search tree, parameters for optimization k, b, n, Ω, and costs of atomic $-expressions (if not provided can be learned by reinforcement learning for the price of extra computations) (for init, sel, exam, exec). • A user can use predefined or overwrite or modify (i.e., the system is open) the modules: init, goal, sel, exam, exec, and then a user will provide their new updates. Truly to write an interpreter is to implement the init,goal,sel, exam and exec functions. Note that having an interpreter (and not a compiler) performing optimization is not a limitation. If we want to optimize the compiler’s output (e.g., the optimal partition of tasks among agents), this can be reformulated as a subproblem and reprocessed through the interpreter. The interpreter’s output can be fed back to the compiler producing an optimal partitioning. Note that in our case, we do not plan to use existing middleware standards like COM, DCOM or CORBA. COM and DCOM are the Microsoft proprietary “standards”, and our design supposed to be portable and not tied to one specific OS, i.e., should be open in the style of GNU/Linux. CORBA on the other hand, was designed to glue heterogeneous software written perhaps in different languages and using legacy software. In our case, the “hardware” can be heterogeneous, but the source files will be written in the same CO$T/CCL code using
GUI. Also the “bytecode” will be the same. Thus our approach is closer in the spirit to a one language environment of Java with RMI, or the MPI library standard for C/C+/Fortran. Two experimentary systems are under design and implementation: • CO$T: a general purpose multi-agent programming environment and language, as a generic unified tool for AI and computer science, and • CCL: a specialized Common Control Language for mission planning and control of multiple Autonomous Underwater Vehicles (AUVs), i.e., a generalization of Evolutionary Robotics with online control algorithms [4], [8]. To implement compiler and intepreter phases: scanning, parsing, semantic analysis, intermediate/target code generation and optimization, we use standard compiler generators tools. The kΩ-search will be an essential part of the interpreter’s optimizer phase. The output of the compiler will be in the form of $-expression “bytecode”, and the output of the intepreter will be downloaded and executed in the select-examine-execute loop controlled by the interpreter. V. A PPLICATION A REAS , P RELIMINARY R ESULTS AND C ONCLUSIONS The uniqueness of the presented approach is twofold: • To target the real-world intractable optimization problems (by solution of total optimization problems, two languages are under implementation: CO$T and CCL [4], [8]), and • To provide support for nonalgorithmic solutions of undecidable problems (a more theoretical work on foundations of problem solving in computer science, AI, and Evolutionary Computation; and based on superTuring models of computation, i.e., the $-calculus and Evolutionary Turing Machines). The $-calculus is applicable to robotics, software agents, neural nets, and evolutionary computation. Potentially it could be used for design of cost languages, cellular evolvable cost-driven hardware, DNA-based computing and molecular biology, electronic commerce, and quantum computing. The $-calculus provides a natural way to express evolution. Its cost performance measure allows to model fitness function, and an infinite search for global optima. Additionally, $calculus allows both to optimize the quality of solutions and to minimize search (problem solving) cost. The behavior of robots, and its interaction with an environment and other robots can be expressed in a uniform form by the $-calculus-based preliminary Generic Behavior MessagePassing Language (GBML) [1], and CO$T and CCL languages [4], [8]. For implementation of CO$T and CCL, we considered C, C++, Java and Lisp. Perhaps, the fastest way to implement a quick and dirty interpreter would be to use Lisp, however for many reasons we decided to bypass that step. Work on CO$T is initiated by the second author at University of Waterloo using Java for implementation. In case of Java,
GUI for the compiler is (almost) for granted. To implement a compiler and intepreter, JavaCC scanner and parser generator (with LL(k) context-free grammar) are used. CCL, on the other hand, is under implementation at Naval Undersea Warfare Center and University of Massachusetts in C using the flex scanner generator and bison parser generator (with a LALR grammmar). Currently, a simplified GUI and a simplified interpreter (without the kΩ-optimization) has been developed to demonstrate the concept of the portability using a solar AUV, a simulator, and an ER1 land robot [4], [8]. ACKNOWLEDGMENTS Research of the first authour was partially supported by grants from Office of Naval Research ONR N66604-03-M4659 and ONR N000140310421. The contributions by Chris Duarte, Christine Buzzell and Gery Martel to the CCL language design and implementation are appreciated. R EFERENCES [1] Eberbach E., Brooks R., Phoha S., Flexible Optimization and Evolution of Underwater Autonomous Agents, in: (eds. N.Zhong, A.Skowron, S.Ohsuga) New Directions in Rough Sets, Data Mining, and GranularSoft Computing, LNAI 1711, Springer-Verlag, 1999, 519-527. [2] Eberbach E., $-Calculus Bounded Rationality = Process Algebra + Anytime Algorithms, in: (ed.J.C.Misra) Applicable Mathematics: Its Perspectives and Challenges, Narosa Publishing House, New Delhi, Mumbai, Calcutta, 2001, 213-220. [3] Eberbach E., On Expressiveness of Evolutionary Computation: Is EC Algorithmic?, Proc. 2002 World Congress on Computational Intelligence WCCI’2002, Honolulu, HI, 2002, 564-569. [4] Eberbach E., Duarte Ch., Buzzell Ch., Martel G., A Portable Language for Control of Multiple Autonomous Vehicles and Distributed Problem Solving, Proc. of the 2nd Intern. Conf. on Computational Intelligence, Robotics and Autonomous Systems CIRAS’03, Singapore, Dec. 15-18, 2003. [5] Eberbach E., On the Connection between the No Free Lunch Theorem and the Trivial Property for Recursively Enumerable Languages, Proc. 2003 Congress on Evolutionary Computation CEC’2003, vol.4, Canberra, Australia, 2003, 2611-2618. [6] Eberbach E., Wegner P., Beyond Turing Machines, The Bulletin of the European Association for Theoretical Computer Science (EATCS Bulletin), 81, Oct. 2003, 279-304. [7] Eberbach E., Goldin D., Wegner P., Turing’s Ideas and Models of Computation, in: (ed. Ch.Teuscher) Alan Turing: Life and Legacy of a Great Thinker, Springer-Verlag, 2004, 159-194. [8] Duarte Ch., Martel G., Eberbach E., Buzzell Ch., A Common Control Language for Dynamic Tasking of Multiple Autonomous Vehicles, Proc. of the 13th Intern. Symp. on Unmanned Untethered Submersible Technology UUST’03, Durham, NH, August 24-27, 2003. [9] Horvitz E., Zilberstein S. (eds.), Computational Tradeoffs under Bounded Resources, Artificial Intelligence 126, 2001, 1-196. [10] Michalewicz Z., Fogel D.B., How to Solve It: Modern Heuristics, Springer-Verlag, 2000. [11] Milner R., Parrow J., Walker D., A Calculus of Mobile Processes, I & II, Information and Computation 100, 1992, 1-77. [12] Refenes A.N., Eberbach E., McCabe S.C., Treleaven P.C., PARLE: A Parallel Target Language for Integrating Symbolic and Numeric Processing, LNCS 366, Springer-Verlag, (1989), vol.II, 181-198. [13] Rudolph G., Convergence Analysis of Canonical Genetic Algorithms, IEEE Trans. on Neural Networks: Special Issue on EC, vol.5:1, 1994, 96-101. [14] Russell S., Norvig P., Artficial Intelligence: A Modern Approach, 2nd edition, Prentice Hall, 2003. [15] Wegner P., Eberbach E., New Models of Computation, The Computer Journal, 47(1), The British Computer Society, Oxford University Press, 2004, 4-9. [16] Wolpert D.H., Macready W.G., No Free Lunch Theorems for Optimization, IEEE Trans. on Evolutionary Computation, vol. 1, no.1, 1997, 67-82.
