Data-Driven Detection of Recursive Program Schemes Martin Hofmann and Ute Schmid
[email protected],
[email protected]
Approaches
Inductive Programming (IP)
Igor II [1, 2]
• Analytical
• Inductive Synthesis of Functional Programs – not only concepts, as in ILP, but recursive programs – from incomplete specifications (IO examples) – 100% accuracy on training data – challenging subfield of machine learning – still little researched niche • Potential Applications – End-user programming, Programming-by-example – Test-Driven Development – AI & Cognitive Psychology (planning, human learning)
• Analytical system with integrated search. • Initial hypothesis is least-general generalisation of IOs.
– data-driven, detect regularities in IOs – fold them into recursive definition
• Successively find a closed rule for a (sub-)set of the IOs.
• Generate & Test
• Unbound variables are induction cue.
– use IOs for test only – systematic, enumerative search – random search, genetic algorithms
• analytic operators to close rules: Partitioning: Introduces case distinction Invent Auxiliary function: Solves subproblem Background: Use use-provided knowledge
• I GOR II – combined approach – guide search by three analytical operators
• All operators are applied in parallel. • Preference bias: hypothesis with least cases.
reverse x = y
= = = =
[d] [d,c] [c,d,b] [d,c,b,a]
Least general generalisation of target IOs
= foldr fun 4 [] x = x3
1a
Check universal property of foldr ! foldr intro
Abducing IOs for fun 5
2a
fun 2 (d:[]) fun 2 (c:d:[]) fun 2 (b:c:d:[]) fun 2 (a:b:c:d:[])
= [] = (c:[]) = (c:b:[]) = (c:b:a:[])
d d d d
Replace open variables: fun 1 & fun 2 compute subterm at 1st & 2nd position
Results of fun 2 can be computed by reverse , fun 2 computes its inputs.
-- IO examples of target function reverse :: [α] → [α] reverse [] = [] reverse (d:[]) = (d:[]) reverse (c:d:[]) = (d:c:[]) reverse (b:c:d:[]) = (d:c:b:[]) reverse (a:b:c:d:[]) = (d:c:b:a:[])
fun 3 (d:[]) fun 3 (c:d:[]) fun 3 (b:c:d:[]) fun 3 (a:b:c:d:[])
-- IOs of background knowledge last :: [α] → α last (a:[]) = a last (a:b:[]) = b last (a:b:c:[]) = c
= [] = (c:[]) = (b:c:[]) = (a:b:c:[])
reverse reverse fun 1 fun 2 fun 3
6
[] fun 1 i : fun 2 i last i x3
[] i@(x 1 :x 2 ) i@(x 1 :x 2 ) i@(x 1 :x 2 ) i@(x 1 :x 2 )
Close rule by recursive call to fun 3
[] fun 1 i : fun 2 i last i reverse (fun 3 i) [] fun 3 (x 2 :x 3 )
= = = =
3
-- data type definitions data [α] = [] | α : [α]
= = = = = =
from background knowledge
recursive call
reverse reverse fun 1 fun 2 fun 3 fun 3
[] i@(x 1 :x 2) i@(x 1 :x 2) i@(x 1 :x 2) [x] i@(x 1 :x 2)
= = = = = =
= = = = =
4 partitioning
= foldr fun 1 [] x = foldr fun 2 [x 1 ] x 2 = x1 : x2 : x3
= = = =
reverse [] reverse i@(x 1 :x 2 ) fun 1 i@(x 1 :x 2 ) fun 2 i@(x 1 :x 2 )
fun 5 b [a] = [b, a] fun 5 c [b, a] = [c, b, a] fun 5 d [c, b, a] = [d, c, b, a]
reverse x fun 4 x 1 x 2 fun 5 x 1 (x 2 : x 3)
fun 1 (d:[]) fun 1 (c:d:[]) fun 1 (b:c:d:[]) fun 1 (a:b:c:d:[])
Specification
reverse [] reverse i@(x 1 :x 2) fun 1 i@(x 1 :x 2) fun 2 i@(x 1 :x 2) fun 3 [x] fun 3 (x 1 :x 2 :x 3 )
fun 1 is last
2
recursive call
reverse x fun 4 x 1 x 2
al is iti es in th po
d [] c (d:[]) b (c:d:[]) a (b:c:d:[])
(:) ,[]
[] fun 1 i : fun 2 i x3 x3
reverse [] reverse i@(x 1 :x 2 ) fun 1 (x 1 :x 2 ) fun 2 (x 1 :x 2 )
auxiliary introduction
Partition IOs w.r.t. symbol at root position of initial rule.:
hy
fun 4 fun 4 fun 4 fun 4
1
partitioning
foldr intro
Abducing IOs for fun 4
I
reverse [] = [] reverse (x 1 :x 2) = (y:ys)
= = = =
background intro
One rule always covers a (sub-) set of IOs.
[] fun 1 i : fun 2 i last i reverse (fun 3 i) [] x3
5
[] fun 1 i : fun 2 i last i reverse (fun 3 i) x3
Partition IOs w.r.t. symbol at root position of 2nd subterm New
Exponential Complexity
Type Morphisms as Program Schemes
Old
Empirical Results Iterations of the I GOR II-algorithm
IP is a search problem in the space of all candidate programs. Alternative strategies to deal with comlexity: • Search full space . . . • Hard coded a priori knowledge reduces expressiveness. • Extensive specification requires expert knowledge. Type morphisms as program schemes allow to reduce complexity without losing expressiveness!
References [1] Martin Hofmann and Emanuel Kitzelmann. I/O Guided Detection of List Catamorphisms: Towards Problem Specific Use of Program Templates in IP. In John Gallagher and Janis Voigtläder, editors, Proceedings of the 2010 ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation (PEPM’2010), pages 93–100, New York, NY, USA, 2010. ACM.
Exist uniquely for any inductive type! [3] Structural Recursion on lists (reduce-map-filter) • Recursive higher-order schemes: foldr :: (α → β → β) → β → [α] → β foldr f v [] = v foldr f v (x:xs) = x ‘f‘ (fold f v xs) map :: (α → β) → [α] → [β] map _ [] = [] map f (x:xs) = (f x) : map f xs -- map f = fold ((:) . f) [] filter :: (α → Bool) → [α] → [α] filter p = fold f [] where f = (λx xs → if p x then x:xs else xs)
• Universal Property of foldr : g [] = v g (x:xs) = f x (g xs)
⇐⇒
g = foldr f v
[2] Emanuel Kitzelmann and Ute Schmid. Inductive synthesis of functional programs: An explanation based generalization approach. Journal of Machine Learning Research, 7:429–454, 2006.
Catamorphism generalise structural recursion for arbitrary inductive data types (Nat, Tree, etc.).
[3] Erik Meijer, Maarten Fokkinga, and Ross Paterson. Functional programming with bananas, lenses, envelopes and barbed wire. In Proceedings of th 5th ACM Conference on Functional Programming Languages and Computer Architecture, Cambridge, MA, USA, pages 124–144. Springer-Verlag, 1991.
Paramorphisms generalise primitive recursion.
⇒ Detect applicability in IOs! ⇐
http://www.cogsys.wiai.uni-bamberg.de/effalip/ Cognitive Systems Group Faculty Information Systems and Applied Computer Science University of Bamberg
Old addN 13 allodd and drop evens 23 fib , add 173 preorder 5 hanoi 5 lasts 5 lengths 1581⊥ mirror 4 odd/even 2/2 powset , ++ 76⊥ reverse 10 rev , last 11 rev , ++ 159 sum 6 zeros 6
New 2 3 2 2 3 173 3 5 3 2 1 2/2 5 2 2 6 3 2
speedup description 7 – – – 8 1 2 1 2 791 4 1/1 15 5 6 6 2 3
λn → map (+n) all odd all id drop n elems filter even
Fibonacci numbers tree traversal towers of hanoi map last map length
mirror a tree mutual rec. def power set reverse a list reverse with last reverse with append fold (+) 0 filter (== 0)
⊥ wrong solution, timeout Runtime statistics in seconds 0.001 min, 139.761 max, 4.509 avg., 0.008 median, 24.295 std.dev.
