Session 18 Automatic Programming A HEURISTIC APPROACH TO ...

Session 18

Automatic Programming A HEURISTIC APPROACH TO PROGRAM VERIFICATION Shmuel M. Katz and Zohar Manna Department of Applied Mathematics The Weizmann I n s t i t u t e of Science Rehovot, I s r a e l

Abstract program), and output v a r i a b l e s z (which are assigned

We present various h e u r i s t i c techniques f o r use in proving the correctness of computer programs. The techniques are designed to o b t a i n a u t o m a t i c a l l y the " i n d u c t i v e a s s e r t i o n s " attached to the loops of the program which p r e v i o u s l y required human "understanding" of the program's performance. We d i s t i n g u i s h between two general approaches: one in which we o b t a i n the i n d u c t i v e a s s e r t i o n by analyzing predicates which are known to be t r u e at the entrances and e x i t s of the loop (top-down approach), and another in which we generate the i n d u c t i v e a s s e r t i o n d i r e c t l y from the statements of the loop (bottom-up approach). I.

values only at the end of the e x e c u t i o n ) .

we are given " i n p u t p r e d i c a t e " $ ( x ) , which puts r e ­ s t r i c t i o n s on the input v a r i a b l e s , and "output p r e d i ­ c a t e " < K x , z ) , which i n d i c a t e s the desired r e l a t i o n be­ tween the input and output v a r i a b l e s .

tach an appropriate i n d u c t i v e a s s e r t i o n Q i to each cutpoint

Introduction

which are known to be true at the entrances and e x i t s

f o r showing p a r t i a l correctness o f i t e r a t i v e ( f l o w ­

of the l o o p , and

c h a r t ) programs, t h a t i s , i t shows t h a t i f the program

(b)

terminates, a given i n p u t - o u t p u t r e l a t i o n i s s a t i s f i e d .

bottom-up approach in which we generate the i n ­

ductive a s s e r t i o n of a loop d i r e c t l y from the s t a t e ­

The method involves c u t t i n g each loop of the program,

ments of the loop.

a t t a c h i n g to each c u t p o i n t an " i n d u c t i v e a s s e r t i o n "

For " t o y " examples, having only a s i n g l e l o o p , it

(which I s a p r e d i c a t e i n f i r s t - o r d e r predicate c a l c u l ­

is g e n e r a l l y c l e a r t h a t the top-down approach is the

u s ) , and c o n s t r u c t i n g v e r i f i c a t i o n conditions f o r each

n a t u r a l method to use.

path from one a s s e r t i o n to another (or back to i t s e l f ) .

However, t h i s is d e f i n i t e l y not

the case f o r r e a l ( n o n - t r i v i a l ) programs w i t h more com­

The program i s p a r t i a l l y c o r r e c t i f a l l the v e r i f i c a ­

plex loop s t r u c t u r e .

Elements of these t e c h ­

In t h i s case some bottom-up t e c h ­

niques were found i n d i s p e n s i b l e .

niques have been shown amenable to mechanization.

Most commonly we have

found it necessary to combine the two techniques, w i t h

King [1969], f o r example, has a c t u a l l y w r i t t e n a ' v e r i ­

the bottom-up methods dominant.

f i e r ' program which, given the proper i n d u c t i v e asser­

Preliminary attempts to a t t a c k the problem of f i n d ­

t i o n s f o r programs w r i t t e n i n a s i m p l i f i e d A l g o l - l i k e

i n g assertions have been made by Floyd [ p r i v a t e commun­

Thus, i t i s

i c a t i o n ] , and Cooper [1971].

f a i r l y c l e a r t h a t the p a r t s of t h i s method which in-

Heuristic rules basically

s i m i l a r to some of our top-down r u l e s have been discov­

volve generating v e r i f i c a t i o n conditions from i n d u c t i v e

ered Independently by Wegbreit [1973].

assertions and then proving or d i s p r o v i n g t h e i r v a l i ­ d i t y is a d i f f i c u l t but programmable problem.

top-down approach in which we o b t a i n the I n d u c t ­

i v e a s s e r t i o n i n s i d e a loop by analyzing the predicates

Floyd [1967] has provided a proof method

language, can prove p a r t i a l correctness.

i.

We d i s t i n g u i s h between two general approaches: (a)

is c o r r e c t has been noted repeatedly in the computer

t i o n conditions are v a l i d .

Given a set of

c u t p o i n t s which cut a l l the l o o p s , our task i s t o a t ­

The d e s i r a b i l i t y of proving t h a t a given program literature,

In a d d i t i o n ,

Elspas, et a l .

[ 1 9 7 2 ] , used " d i f f e r e n c e equations" derived from the

However,

program's statements which i s , in essence, a bottom-up

as King puts i t , f i n d i n g a set of assertions to ' c u t '

approach.

each loop of the program "depends on our deep under­

We handle programs w i t h arrays s e p a r a t e l y , since

standing of the program's performance and requires some

generating assertions i n v o l v i n g q u a n t i f i c a t i o n over

s o p h i s t i c a t e d i n t e l l e c t u a l endeavor".

the indices of arrays r e q u i r e s s p e c i a l treatment. In t h i s paper we show some general h e u r i s t i c t e c h ­

Thus

in Section II we discuss h e u r i s t i c techniques f o r f l o w ­

niques f o r a u t o m a t i c a l l y f i n d i n g a set of i n d u c t i v e

chart programs w i t h o u t a r r a y s , w h i l e in Section I I I we

assertions which w i l l allow proving p a r t i a l c o r r e c t ­

extend the treatment to programs w i t h a r r a y s .

ness of a given program.

More p r e c i s e l y , we are given

In Sec­

t i o n IV (conclusion) we discuss open problems and pos­

a f l o w c h a r t program w i t h input v a r i a b l e s x (which are

s i b l e i m p l i c a t i o n s of our techniques.

not changed during e x e c u t i o n ) , program v a r i a b l e s y

where these approaches seem a p p l i c a b l e include proving

(used as temporary storage during the execution of the

t e r m i n a t i o n of programs, and discovering the input and 500

Related problems

output assertions of a program. Our emphasis in t h i s paper is on the e x p o s i t i o n of the r u l e s themselves and we are purposely somewhat vague on other problems, such as c o r r e c t l y l o c a t i n g the cutpoints or o r d e r i n g the a p p l i c a t i o n of the r u l e s .

Though we do not enter i n t o d e t a i l s , we

assume t h a t whenever possible we conduct immediate t e s t s on the consistency ( w i t h known information) of a new component for an a s s e r t i o n as soon as it is gene­ r a t e d , and that algebraic s i m p l i f i c a t i o n s and manipu­ l a t i o n s are done whenever necessary.

II H e u r i s t i c s A.

for

Programs

without

Arrays

Top-down approach. We begin by l i s t i n g the top-down r u l e s , which may

be divided i n t o two classes:

entry rules and e x i t

rules. 1.

Entry r u l e s .

These r u l e s are i n t u i t i v e l y ob­

v i o u s , but provide valuable i n f o r m a t i o n in a s u r p r i s ­ ing number of cases. r u l e En`1. Any conjunct* in the input predicate $(x) may be added to any Q,

It need not be proven since

the input v a r i a b l e s are not changed i n s i d e the program, rule En2.

Any predicate known to be true upon f i r s t

reaching a cutpoint i should be t r i e d in Q i . 2.

Exit rules.

For s i m p l i c i t y in the statement

of these r u l e s , we assume that a c u t p o i n t is attached to the arc immediately before an e x i t t e s t of the loop. Thus we may consider an e x i t from a loop to be of the form

where t. is the e x i t t e s t , p, is some conjunct of a predicate known to be t r u e when the e x i t t e s t f i r s t holds, 1 is the c u t p o i n t on the arc leading i n t o the e x i t t e s t , and Q1 is the a s s e r t i o n which we wish to discover.

We attempt to e x t r a c t i n f o r m a t i o n from p.

If a predicate is expressed as a conjunction A 1 AA 2 ^........A N , then each A is a conjunct, of the predicate.

501

I t I s possible t o continue t o design r u l e s f o r o b t a i n i n g R f o r s p e c i f i c forms of p. , but since our aim is to explain the general tone of these techniques, we w i l l not go i n t o f u r t h e r d e t a i l s in t h i s d i r e c t i o n . B.

Bottom-up Approach. A l l of the rules given above have in common t h a t

they expect to be provided w i t h some i n f o r m a t i o n on e i t h e r what conditions were true upon e n t e r i n g the loop or what conditions were expected to hold upon completing the loop (or b o t h ) .

However., it is possible

to produce conjuncts of the a s s e r t i o n Q without con­ s i d e r i n g predicates already established elsewhere in the program.

In order to accomplish t h i s goal we

s h a l l look f o r a predicate which is an i n v a r i a n t of the loop L , i . e . , it remains t r u e upon repeated exe­ cutions of the loop. C l e a r l y , any conjunct in the i n d u c t i v e a s s e r t i o n of a loop must be an i n v a r i a n t of the loop.

However,

i n the top-down r u l e s t h i s i s u s u a l l y the l a s t f a c t which is established about a prospective a s s e r t i o n . In the pure bottom-up approach, a s s e r t i o n s which a r i s e " n a t u r a l l y " from the computations in the loop are d i ­ r e c t l y generated — and only afterward checked f o r relevance to the o v e r a l l p r o o f . Most i n v a r i a n t s may be traced back to the f a c t t h a t at any stage of the computation, those assignment statements which are on the same paths through the loop have been executed an i d e n t i c a l number of t i m e s , and t h i s is a ' c o n s t a n t ' which may be used to r e l a t e the v a r i a b l e s i t e r a t e d . For an assignment statement y. ■*■ f ( x , y ) we l e t y1

denote the value o f y . a f t e r n executions o f the (01 statement, w h i l e y; ' i n d i c a t e s the " i n i t i a l " value of

y 1 upon f i r s t reaching a given c u t p o l n t of the loop. Our technique f o r f i n d i n g i n v a r i a n t s involves c o n s t r u c t i n g an "operator t a b l e " in which we record u s e f u l i n f o r m a t i o n f o r each operator. Among the e i t r i e s f o r an operator are i t s d e f i n i t i o n (using "weaker" ope­ r a t o r s ) , a d e s c r i p t i o n of a general computation a f t e r n i t e r a t i o n s , and other common i d e n t i t i e s which f a c i ­ l i t a t e s i m p l i f i c a t i o n s . For example, f o r + our t a b l e w i l l include the f a c t t h a t f o r an assignment statement of the form y 1 + y 1 +k, in general where k

VJ

y 1 ( n ) =y ( 0 ) +

T k^"15

is the value of k before the j - t h i t e r a ­

t i o n of the assignment statement. Important i d e n t i ­ t i e s are also noted i n c l u d i n g t h a t f o r a constant c , n n , .. I c » c n , and t h a t £ i» ' -i—*- . Rules f o r producing i-1 i-1 2 i n v a r i a n t s l i n k i n g v a r i a b l e s which receive assignments on d i f f e r e n t paths through the loop are p r e s e n t l y being developed. Here we present r u l e s only f o r the 502

Our h e u r i s t i c r u l e s are a l l r e l e v a n t to programs having an a r b i t r a r y number of loops, and an a r b i t r a r y complex ' t o p o l o g y ' , although, of course, they w i l l y i e l d v a l i d i n d u c t i v e assertions more o f t e n and more immediately in a simple program. One of the problems in applying the r u l e s is d e c i ­ ding what order is p r e f e r a b l e . In p a r t i c u l a r , it has been found t h a t many terms of the a s s e r t i o n may be ob­ tained both by the bottom-up r u l e s and by repeated use of the top-down r u l e s .

However, u s u a l l y one method

w i l l y i e l d the r e s u l t immediately, w h i l e considerable e f f o r t is expended if the other method is applied first.

Experience shows that there is a need f o r i n ­

t e r a c t i o n between the top-down and bottom-up ap­ proaches.

For example, we may use established i n v a r ­

iants to deduce the r e l a t i o n R in the top-down r u l e Ex2, and on the other hand, we may d i r e c t the search for p a r t i c u l a r i n v a r i a n t s based on v a r i a b l e s or opera­ tors which appear in p. .

503

504

505

i . e . , the claim which can be made about the e f f e c t on the arrays of one c i r c u i t through the loop.

This claim

i s o f t e n not d i f f i c u l t t o e s t a b l i s h , i n p a r t i c u l a r f o r loops which do not contain other loops since then the c i r c u i t through the loop is a simple sequence of s t a t e ­ ments.

The a s s e r t i o n can be most e a s i l y established by

the known technique of "backward s u b s t i t u t i o n " , moving backwards around the loop past each assignment s t a t e ­ ment.

506

assertions w i l l show the program p a r t i a l l y c o r r e c t .

i n i t i a l attempt to f i n d an a s s e r t i o n , and would " g i v e -

We c l e a r l y could have used the t r a n s i t i v i t y r u l e here,

up" r a p i d l y if it d i d not provide an almost immediate

but f o r t h i s example, the amount of work required is

solution.

about the same.

may be t r i e d on the c u t p o i n t .

Then o t h e r , possibly more a p p r o p r i a t e , r u l e s Only if a l l r u l e s f a i l e d

to add relevant i n f o r m a t i o n , would the " s t r o n g " v e r s i o n IV.

be a p p l i e d .

Conclusion

C l e a r l y , the r u l e s and examples given in t h i s paper are f a r from being a general system f o r f i n d i n g induc­ t i v e assertions.

This d i v i s i o n is p a r a l l e l to the human

attempt to f i r s t f i n d what Is " o b v i o u s l y " true In the

More and b e t t e r r u l e s are needed,

loop, and only afterwards b r i n g out the f i n e p o i n t s . The o v e r a l l strategy we have adopted in t h i s paper has been to f i n d assertions strong enough to prove the

p a r t i c u l a r l y f o r array a s s e r t i o n s , which tend to be

p a r t i a l correctness in as few steps as p o s s i b l e .

complex and unwieldy.

in general, we attempt to d i r e c t l y produce a near-exact

In a d d i t i o n , before the r u l e s can be incorporated i n t o a p r a c t i c a l framework, we must order t h e i r a p p l i ­ cation.

That i s , at each step we must provide more

exact c r i t e r i a f o r deciding which r u l e to apply and on which c u t p o i n t of the program.

The order in which the

d e s c r i p t i o n of the operation of a l o o p , without going through numerous intermediate stages where we are u n ­ able to shorn e i t h e r v a l i d i t y or u n s a t l s f i a b i l i t y .

If

our h e u r i s t i c is wrong, t h i s f a c t w i l l be revealed r e l a t i v e l y r a p i d l y by generating an u n s a t i s f i a b l e v e r i ­

r u l e s are presented in each subclass does i m p l i c i t l y

f i c a t i o n condition.

provide a p a r t i a l s p e c i f i c a t i o n .

native claim.

Thus we presently

Thus,

We then may t r y a weaker a l t e r ­

We f e e l t h a t t h i s is the approach which

would t r y t o apply E x 1 , and only i f i t f a i l e d t r y Ex2,

should be taken in order to construct a p r a c t i c a l

etc.

system which could be added to a program v e r i f i e r .

Moreover, we generally would t r y to gather i n ­

formation on simple v a r i a b l e s using the rules of Sec­ t i o n I I before attempting t o t r e a t array a s s e r t i o n s . The more basic (and open) questions are (a) whether

We believe that the bottom-up approach may also be used to solve other problems.

For example, in the

p a r t i t i o n program (Example 5 ) , the i n d u c t i v e a s s e r t i o n

to attempt top-down or bottom-up techniques f i r s t for

was a c t u a l l y found without using the w given by the

a given l o o p , and (b) which loop of a program should

programmer.

be t r e a t e d f i r s t .

Although we experimented w i t h

In one s i n g l e step 1)1 may be generated

from Q , and thus we have ' d i s c o v e r e d ' what the program

various orderings in the examples In t h i s paper, we

does without the use of a d d i t i o n a l i n f o r m a t i o n .

have t e n t a t i v e l y formulated a more f i x e d approach.

feature of the bottom-up approach can probably be most

Our present I n c l i n a t i o n is to f i r s t use top-down r u l e s

useful f o r strengthening a too-weak a s s e r t i o n , i . e . ,

from the ( p h y s i c a l ) beginning of the program.

revealing that the program does more than is claimed

(Since

in general there is more than one outer loop, usually only entrance r u l e s are a p p l i c a b l e . )

Then we use

This

in w . Another apparent a p p l i c a t i o n is f o r proving t e r m i ­

bottom-up r u l e s f o r the same l o o p , to create a p true

nation using well-founded s e t s .

a f t e r e x i t from the f i r s t loop containing as much i n ­

predicates Q i and functions u i are r e q u i r e d , where u i

formation as p o s s i b l e .

(a mapping to the well-founded s e t ) has i t s domain

We continue w i t h the next

outer loop In a s i m i l a r manner.

I f , however, we are

bounded by Qi and descends each time the loop Is exe­

stymied and unable to find a loop a s s e r t i o n , we start

cuted.

w i t h top-down r u l e s from the end of the program, and

since no w Is provided.

t r y to work backwards towards the beginning. A more s o p h i s t i c a t e d approach would require a weighted e v a l u a t i o n f u n c t i o n capable of making a very cursory scan of the program.

This f u n c t i o n would

i d e n t i f y loops which seemed ' p r o m i s i n g 1 , i . e . l i k e l y

For t e r m i n a t i o n ,

Here again the bottom-up approach is useful We have already begun inves-

t i g a t i n g bottom-up methods f o r generating both the Q i 's and the u i 's which w i l l ensure t e r m i n a t i o n . The u l t i m a t e goal of automatic a s s e r t i o n generation is almost c e r t a i n l y u n a t t a i n a b l e ; thus the optimal system would involve man-machine i n t e r a c t i o n .

Whenever

to y i e l d v a l u a b l e i n f o r m a t i o n r a p i d l y , and apply selec­

it was unable to generate the proper a s s e r t i o n , the

ted r u l e s f i r s t t o these loops.

machine would supply d e t a i l e d questions on problematic

Since some of the r u l e s could continue searching

r e l a t i o n s among v a r i a b l e s and possible f a i l u r e p o i n t s

f o r a p o s s i b l y n o n - e x i s t e n t form of a s s e r t i o n almost

( i n c o r r e c t loops) of the program.

I n d e f i n i t e l y (the t r a n s i t i v i t y r u l e , f o r example),

s p e c i f i c a t i o n of the a s s e r t i o n s , provided by the

such r u l e s would have a "weak" v e r s i o n and a " s t r o n g "

programmer, could shorten t h i s e n t i r e process.

version.

The "weak" v e r s i o n would be used in the

Clearly, a p a r t i a l

REFERENCES COOPER [ 1 9 7 1 ] .

D. C. C o o p e r , " P r o g r a m s f o r M e c h a n i c a l

Program V e r i f i c a t i o n " , 6,

i n Machine

American E l s e v i e r , pp.

ELSPAS e t a l .

[1972].

L e v i t t and R . J .

43-59

(1971).

B . E l s p a s , M.W. G r e e n , K . N . Waldinger,

"Research

a c t i v e Program-Proving Techniques", Park, Cal. FLOYD [ 1 9 6 7 ] .

Floyd,

in P r o c .

Mathematics,

Vol.

AMS, p p . 1 9 - 3 2 KING

[1969].

J. King,

Thesis, Pa. WEGBREIT

in I n t e r ­

S R I , Menlo

(May 1 9 7 2 ) .

R.W.

Programs",

Intelligence

" A s s i g n i n g Meanings t o o f a Symposium i n A p p l i e d

19

(J.T.

Schwartz - e d i t o r ) ,

(1967). " A Program V e r i f i e r " , P h . D .

Carnegie-Mellon U n i v e r s i t y ,

Pittsburgh,

(1969).

[1973].

B. W e g b r e l t ,

" H e u r i s t i c Methods

f o r Mechanically Deriving

nductive Assertions",

U n p u b l i s h e d memo, B o l t , Beranek and Newman, Inc.,

Cambridge, M a s s . ,

(February 1973).

512

Session 18 ITERATED L I M I T I N G RECURSION AND THE PROGRAM MINIMIZATION PROBLEM.

machines.

L.K.

KEY WORDS AND PHRASES:

Schubert

minimal programs,

Department of Computing Science, U n i v e r s i t y o f A l b e r t a , Edmonton, A l b e r t a , Canada. ABSTRACT:

The g e n e r a l p r o b l e m o f

minimal programs

r e a l i z i n g given

descriptions"

considered,

is

descriptions

may

properties.

The p r o b l e m o f

programs

consistent with

input-output infinite o f E.M.

specify

lists

is

a

input-output Gold's

problem;

another

tne grammatical

arbitrary

finite

or

this

is

more

difficult

whether

than

they are

the

programs

in

or

nondecreasing

(whichever

is

harder).

formulated

in

terms

predicates

and f u n c t i o n a l s ,

of

This

k-limiting

of

Gold's

A simple

consequence

is

that

is

limiting

minimization

problem

solvable

for

finite

limiting

recursively

about

lists,

limit

input-output solvable w i t h weak

for

limiting

(more g e n e r a l l y ,

function

iterated

limiting

regarded

as

of

infinite

assumptions Gold

identification)

thought.

Intuitively,

i d e n t i f i c a t i o n might be

higher-order

inductive

performed c o l l e c t i v e l y by an community

2-

identification

" b l a c k box"

as a model of i n d u c t i v e

and

lower-order

inference

in the

larger

there

class.

index

portions

lists

all

in

the

l e a r n i n g problem i n

of

an

elements

of

an

are

pattern

adaptive pattern "learn"

a mapping

in

an

identifiable of

total

fiable partial fiable

recursive

in

the

index)

recursive

in the

in

the

functions

limit

recursive

and

(hence

functions

considered.

replacement of

that

for

it.

t h a t any r . e . is

that

the class

is

not

identi­

also

the

class of

is

not

identi­

limit).

Here a m o d i f i e d v e r s i o n o f is

the

found an

functions

limit,

a

the

identified

Two o f G o l d ' s m a i n r e s u l t s w e r e total

A l l of

t h a t i t has

(equivalently,

it

patterns

conform w i t h

i t has

t h e sense

from

to

i.e.,

t h e i r appropriate responses.

once

the

recognition

labelled patterns,

responses w i l l

is

recognition.

of

of

the

relevant

sequence

class

the

information

to responses by p r e s e n t i n g

algorithm

"guesses"

for

patterns

mapping,

limit

An example of a p r a c t i c a l p r o b l e m these concepts

machine's

for

"identifying

function

an

was

s u c c e s s i v e guesses b e i n g based o n

function.

inference

ever growing

inductive

convergent to

d e s i r e d mapping

recursively lists

any member o f

successively

with

the program

[1]

functions

succeed

system is caused to

operator.

program

c o n s i s t s of g e n e r a t i n g a sequence of

Typically

length

inductive

unsolvability.

computable which

limit"

to which

given

recursive

t h e measure o f program s i z e .

regarded

deciding

d e f i n e d by

application

of

sequence w h i c h

result is

repeated

input-output

are not

the problem of enumerating

order of

the

function,

Although

any

classes

(integers)

found to be no

problem of

of

I d e n t i f y i n g a computable

r e l a t e d problem is

any g i v e n p r o g r a m r e a l i z e s

description,

in

(for

identification

most p r o g r a m m i n i m i z a t i o n p r o b l e m s

degree

e x i s t machines

a variant

inference problem).

recursion,

INTRODUCTION

what

infinite

case

limiting

A q u e s t i o n c o n s i d e r e d by Gold

program

f i n d i n g minimal

lists,

closely

recursively solvable,

1.

function identification,

program l e n g t h measures,

properties,

"program

where program

special

function

inference,

finding

Automatic Programming

Gold's problem

The f i r s t m o d i f i c a t i o n information

sequences

by

is

the

( f i n i t e or i n f i n i t e )

"program d e s c r i p t i o n s "

i s a l s o suggested b y the work o f Kolmogorov

which may s p e c i f y a r b i t r a r y program p r o p e r t i e s .

[ 6 ] , Martin-Lof

D e s c r i p t i o n s which l i s t i n p u t - o u t p u t p a i r s are

t h e number o f symbols i n the s h o r t e s t program

then regarded as a s p e c i a l case.

f o r g e n e r a t i n g a f i n i t e sequence can be taken

The second

[7] and o t h e r s , showing t h a t

m o d i f i c a t i o n i s t h a t i t e r a t e d l i m i t procedures

a s a measure o f the i n f o r m a t i o n c o n t e n t o f the

(It-limiting recursive functionals)

sequence, and t h i s measure p r o v i d e s a l o g i c a l

are

admitted f o r program-finding, since f i n d i n g

b a s i s f o r i n f o r m a t i o n t h e o r y and p r o b a b i l i t y

s u i t a b l e programs i n the n o n - i t e r a t e d l i m i t

theory.

i s i m p o s s i b l e f o r many c l a s s e s o f program descriptions.

I n the f o l l o w i n g t h e u n s o l v a b i l i t y o f most

For t h i s purpose k - l i m i t i n g

n o n t r i v i a l program m i n i m i z a t i o n problems i s

recursiveness is defined by s t r a i g h t - f o r w a r d

f i r s t noted.

g e n e r a l i z a t i o n o f G o l d ' s concept o f l i m i t i n g

basic properties of k - l i m i t i n g recursive

recursiveness.

p r e d i c a t e s and f u n c t i o n a l s ,

The t h i r d m o d i f i c a t i o n i s the

A f t e r e s t a b l i s h m e n t o f some

i t i s shown t h a t

added r e q u i r e m e n t t h a t programs found i n the

any program m i n i m i z a t i o n problem i s k - l i m i t i n g

(iterated)

r e c u r s i v e l y s o l v a b l e i f the problem o f d e t e r m -

l i m i t b e m i n i m a l a c c o r d i n g t o some

p r e s c r i b e d measure of program s i z e .

i n i n g whether any g i v e n program s a t i s f i e s any

A c c o r d i n g l y problems o f t h i s m o d i f i e d type are

given d e s c r i p t i o n is k - l i m i t i n g recursively

c a l l e d program m i n i m i z a t i o n problems.

s o l v a b l e and programs are k - l i m i t i n g r . e .

There are v a r i o u s reasons f o r a n i n t e r e s t i n f i n d i n g m i n i m a l - l e n g t h programs.

o r d e r o f nondecreasing s i z e .

I n work

Simple conse-

quences are t h a t the problem o f f i n d i n g

o n grammatical i n f e r e n c e c l o s e l y r e l a t e d t o

m i n i m a l programs f o r f i n i t e f u n c t i o n s i s

G o l d ' s i d e n t i f i c a t i o n p r o b l e m , Feldman [2]

l i m i t i n g recursively solvable,

c o n s i d e r s i n f e r e n c e schemes which t r y t o f i n d

problem o f f i n d i n g minimal programs f o r

"good" grammars c o n s i s t e n t w i t h a v a i l a b l e

a r b i t r a r y computable f u n c t i o n s

i n f o r m a t i o n about a language.

explicit

One measure

listing)

is

and t h a t the

(given an

2-limiting recursively

o f goodness i s t h e i n t r i n s i c c o m p l e x i t y , o r

s o l v a b l e , w i t h weak assumptions about the

s i z e , of a grammar.

measure of program s i z e .

In terms of the f u n c t i o n

i d e n t i f i c a t i o n p r o b l e m , t h i s corresponds t o f i n d i n g programs which are s m a l l a c c o r d i n g t o some measure o f program s i z e .

in

Lower bounds on the

d i f f i c u l t y o f these problems are a l r e a d y known from

I n d e e d , the

the work of Pager [8] and Gold [ 1 ] . Finally,

the p o i n t i s emphasized i n the

use o f s m a l l programs f o r i n d u c t i v e i n f e r e n c e

c o n c l u d i n g remarks t h a t l i m i t i n g r e c u r s i v e l y

i s a r e c u r r i n g theme i n the l i t e r a t u r e

solvable i n d u c t i o n problems,

f o r example R e f s .

(see

3-5); allusion is usually

though s t r i c t l y

" u n s o l v a b l e " i n g e n e r a l , are n o n e t h e l e s s w i t h -

made to the s c i e n t i f i c maxim knows as

in the reach of mechanical procedures in the

"Occam's Razor", a c c o r d i n g t o which " i t i s

i m p o r t a n t sense d e s c r i b e d b y G o l d , and t h a t

vain

even problems u n s o l v a b l e in the l i m i t may be

to do w i t h more what can be done w i t h

fewer" in a c c o u n t i n g f o r known phenomena.

r e g a r d e d as s o l v a b l e in a weakened sense by an

The s p e c i a l importance of minimal programs

expanding community of mechanisms p e r f o r m i n g 514

l i m i t computations. 2.

assumed t o b e e f f e c t i v e l y enumerable

PROGRAM MINIMIZATION PROBLEMS To

f i x ideas,

of

n°ndecreasing l e n g t h .

any programmable machine M

number

may be t h o u g h t of as a 2 - t a p e T u r i n g machine, w i t h one

tape regarded as

input-output

tape and t h e o t h e r as program t a p e . both tapes

also

s e r v e as w o r k i n g

computation begins w i t h the control

of

t h e machine

in

provides

(I/O)

In

One or

tape.

A

the

halts,

the

put.

It

is

1-1

coding

I/O t a p e . I/O

tape

assumed t h a t

f o r b o t h tapes)

onto

the

gives

(same s y n t a x

i n t e g e r s N.

used

should

The

be

in a program

o b v i o u s l y machine and

a

of

the

minimal

Whose

otherr

Program f o r

length

program

function

for

a

This

correct

inter-

Program f o r a f u n c t i o n O i s one

out-

a

l e n g t h measure.

to a

results.

the

minimal

some-

without e x p l i c i t reference

k e p t i n mind

pretation

t h e r e i s an e f f e c t i v e

from t a p e e x p r e s s i o n s

be

t h e machine

expression

following,

p a r t i c u l a r machine o r

s t a r t state

I f and when

symbols

the

such a l e n g t h measure,

the

times

and w i t h a program on t h e program tape and an i n p u t on

For example,

l e n g t h - m e a s u r e dependent concepts w i l l

finite-state

a unique

of elementary

in order

O

does

4,.

not

exeed

the

The problem o f

length

any

finding a

(or a l l m i n i m a l programs)

given

a

(possibly

for

infinite)

list

program (or I/O t a p e e x p r e s s i o n ) c o r r e s p o n d i n g of the elements of ♦, is an example of a code number (index) X be written as ■i.i. ~ pro t o code number ( i n d e x ) x w i l l be w r i t t e n as x . Program problem generally ,. . . , , . . ... . - ,,u_„ If M e v e n t u a l l y h a l t s w i t h o u t p u t z when . - , . .. , . . „.,„ s u p p l i e d w i t h program x and i n p u t y, one may

a program m i n i m i z a t i o n problem is t h e problem r ■» c e of f i n d i n g a m i n i m a l program (or a l l m i n i m a l

TC

3

M

,M, . , . , . . J. /,,\ T . ,„ w r i t e 4> (y) = z. If M does n o t h a l t , t (Y) x x - ^. -, ., i. • i m. is u n d e f i n e d . Thus M computes a p a r t i a l ,M . , . . ■ ,i f u n c t i o n $ x w i t h program x . However, i t w i l l ,_ . be c o n v e n i e n t to t h i n k of x n o t merely as a M program f o r tj> , b u t as a program f o r any

programs)

domain.

$

are a

M .

If such an x e x i s t s f o r a g i v e n q>,

o n which

functions

are

all partial

programmable

that only

integral a

f i n i t e number of

wff

, ,,......Which p a r t i a l recursive function

enumerates some O.

A t any t i m e

d e s c r i p t i o n s ; one d i g i t , say 0, would be r e s e r v e d a s t e r m i n a t o r and a l l f u n c t i o n v a l u e s

c o r r e s p o n d i n g tp p o i n t s beyond the end of a

Memo AI-93

f i n i t e d e s c r i p t i o n would b e s e t t o 0 .

2_0, 244-262 (1972) .

(1969); a l s o I n f . and C o n t r o l

2

For t h e f i n i t e d e c i s i o n f u n c t i o n s , note t h a t

3.

R. Solomonoff, "A Formal Theory of

t h e f u n c t i o n s computed w i t h any f i n i t e s e t o f

Inductive Inference," Inf.

programs can be d i a g o n a l i z e d to y i e l d a f i n i t e

1-22, 224-254

d e c i s i o n f u n c t i o n which r e q u i r e s a program not i n t h e g i v e n s e t .

4.

(1964),

G. C h a i t i n , "On the D i f f i c u l t y of Computations," IEEE Trans. I n f . Theory

T o prove i n f i n i t e

I T - 1 6 , 5-9

d i v e r s i t y f o r t h e 2-element d e c i s i o n f u n c t i o n s , i t i s s u f f i c i e n t t o show t h a t n o s e t o f n

and C o n t r o l 7 ,

5.

(1970).

D.G. W i l l i s ,

"Computational Complexity

programs can i n c l u d e a program f o r each 2-

and P r o b a b i l i t y C o n s t r u c t i o n s , " J . Assoc.

element d e c i s i o n f u n c t i o n whose arguments are

Comp. Mach. 17, 241-259 (1970).

in a f i x e d set of 2

integers;

but f o r t h i s

6.

A.N. Kolmogorov, "Three Approaches to the

many arguments at l e a s t 2 of the programs

quantitative Definition of Information,"

must g i v e i d e n t i c a l r e s u l t s

I n f o r m . Transmission 1 , 3-11 (1965); a l s o

{ i f any),

so that

I n t . J. Comp. Math 2, 157-168 (1968).

two unsymmetric d e c i s i o n f u n c t i o n s are missed. 3

F u n c t i o n s a r e regarded as a s p e c i a l case of

7.

"The D e f i n i t i o n of Random

Sequences," I n f . and C o n t r o l 9_, 602-619

functionals.

(1966).

The e q u i v a l e n c e f o l l o w s from t h e f a c t t h a t F(6)

P. M a r t i n - L o f ,

i s independent o f n . , . . . , n , ,

s o t h a t the

8.

D. Pager,

"On the Problem of F i n d i n g

Minimal Programs f o r T a b l e s " ,

i t e r a t e d l i m i t o f G must e x i s t .

I n f . and

C o n t r o l 14, 550-554 (1969) .

5

This d i f f e r s from G o l d ' s d e f i n i t i o n , which expresses l i m i t i n g r e c u r s i v e e n u m e r a b i l i t y i n terms o f l i m i t i n g s e m i - d e c i d a b i l i t y . However, t h e d e f i n i t i o n s can be shown to be equivalent

(for k = l ) .

A c t u a l l y , Feldman was concerned w i t h "occams enumerations" of f o r m a l grammars, but the problem o f f i n d i n g minimal grammars f o r languages is e s s e n t i a l l y the same as t h a t of f i n d i n g m i n i m a l programs f o r d e c i s i o n functions. References 1.

E.M. G o l d ,

"Limiting Recursion," J.

Symb. L o g i c 30, 28-48 (1965). 2.

J . Feldman,

"Some D e c i d a b i l i t y Results

on Grammatical I n f e r e n c e and C o m p l e x i t y , " Stanford A r t i f i c i a l I n t e l l i g e n c e Project

9.

D. Pager, " F u r t h e r Results on the Problem of F i n d i n g the S h o r t e s t Program f o r a D e c i s i o n T a b l e , " presented at Symposium on Computational C o m p l e x i t y , O c t . 25-26, 1971; a b s t r a c t e d in SIGACT News, No. 13 (Dec. 1971) .

10. H. Putnam, " T r i a l and E r r o r P r e d i c a t e s and the S o l u t i o n of a

Problem of

M o s t o w s k i , " J. Symb. Logic 30, 49-57 (1965).