A Framework for Distributed Proof Search - Carnegie Mellon ...

A Framework for Distributed Proof Search? David Renshaw, Andr´e Platzer, Chris Martens, and Michelle Mazurek Carnegie Mellon University, Computer Science Department, Pittsburgh, PA, USA {renshaw|aplatzer|cmartens}@cs.cmu.edu, [email protected]

Abstract. In automated theorem proving, the nondeterminism of proof rule application naturally gives rise to nested and/or branching of independent subgoals. This problem structure should be amenable to parallel exploration, but there are at present few theorem prover implementations which actually exploit this potential. We propose a new programming model to make it easier to write such parallel provers. In it, details such as communication and work distribution among processors are hidden from the programmer. Banyan, our implementation of this model, only needs problem-specific input such as how a tree of proof goals gets constructed for the proof search algorithm and what should be the relative amount of processor time spent at each subgoal. This allows the programmer to focus on high level proof search and finely tune a search strategy, for example by giving more time to branches that are more promising. The Banyan runtime system automatically distributes and dynamically redistributes proof goals among distributed processors and performs weighted round-robin scheduling among active goals. We have used Banyan to write a theorem prover for hybrid systems, DLBanyan. DLBanyan outperforms our existing sequential prover for hybrid systems and achieves a near linear speedup in the number of processors used.

1

Introduction

Efficient proof search is the key to turning theoretically complete proof systems into practically useful automatic proof procedures. For problems like system verification that have no effective axiomatization, efficient proof search is even more important. The efficiency of a prover depends critically upon how it deals with nondeterminisms, which may be choices between possible proof rule applications or between possible invariants and induction hypotheses. Some are don’t care nondeterminisms, e.g. the order of α-rule applications, and have virtually no practical impact on proving. Others are more significant don’t know nondeterminisms, e.g. choices between different invariants. It is not easy to come up with good exploration strategies for those nondeterminisms. As an example domain, we consider hybrid systems [1], i.e. systems with joint discrete and continuous dynamics, such as cars and aircraft. In hybrid systems verification, the difference in ?

This material is based upon work supported by the National Science Foundation under Grant Nos. CNS-0926181, CNS-0931985, NASA grant NNG-05GF84H, ONR award N00014-10-1-0188.

2

David Renshaw, Andr´e Platzer, Chris Martens, and Michelle Mazurek

performance between good and bad proof search strategies can be two orders of magnitude or more [2]. Theorem proving for hybrid systems [3] involves decision procedures for real arithmetic [4]. Because real arithmetic has a high theoretical and practical computational complexity [5], neither depth-first nor breadth-first proof search is practically complete, even for decidable subproblems of hybrid systems verification. Both exploration orders can be stuck for several days on the wrong proof choices without finding an efficient 30 minute proof. When using computationally expensive decision procedures, inefficient proof search can turn a theoretically decidable problem into a practically infeasible problem. Practically complete procedures need to follow a fair exploration order of all possible proof options for arbitrarily long times, including the time for background decision procedures. Thus proof search has an inherently parallel problem structure, but few theorem provers actually exploit this potential. One reason is that general parallel (and distributed) programming can be tricky. Simplified distributed programming models like MapReduce list processing, on the other hand, do not match the problem structure of proof search very well. Flat parallel process structures are inefficient for parallel proving problems. Neither eager parallel exploration nor lazy work-stealing exploration really match. Eagerly forking one parallel proving process for each subproblem will quickly clutter the machine with too many processes competing for computing and memory resources, thereby effectively spoiling progress. This effect is particularly counterproductive in the presence of background decision procedures that are computationally expensive and memory intensive. Lazy work-stealing, which proves as many subgoals at once as there are processors and moves on to the next subproblem whenever one subproblem terminates, is also unlikely to work. Again, this problem intensifies when using decision procedures, because they can run for days on one subproblem without terminating, even for decidable theories. For other problem domains, where all subproblems need to be solved, these simple exploration orders still work. The theorem proving problem is inherently different, though, because answers to some subproblems invalidate the need to consider others, but it is not clear which subproblems terminate most quickly. Consequently, static exploration orders are insufficient and automated theorem proving needs a more dynamic exploration order that fits to the proving domain. In this paper we propose a new programming model that naturally fits the actual problem structure in proof search procedures of automated theorem provers. Our model makes it easier to implement distributed parallel proof search procedures and hides details about communication and work distribution among processors. The proposed model is designed for proof search problems and makes it easy to specify parallel exploration strategies that are sufficiently focused to make progress but sufficiently broad to ensure the fairness required for completeness reasons. In this paper we consider the automated theorem proving problem for hybrid systems [3], but our approach is relevant for other domains as well.

A Framework for Distributed Proof Search

3

client_1 time = 29

A_0 time = 3

A_10 time = 0

A_81 time = 0

A_269 time = 0

A_297 time = 0

A_11 time = 4

A_12 time = 13

A_16 time = 2

A_43 time = 1292

A_65 time = 0

A_83 time = 1

A_106 time = 0

A_2 time = 4

A_18 time = 1

A_44 time = 0

A_63 time = 4

A_82 time = 0

A_105 time = 0

A_1 time = 1

A_17 time = 0

A_40 time = 1

A_107 time = 0

A_19 time = 1

A_84 time = 0

A_108 time = 0

A_109 time = 0

A_101 time = 0

A_20 time = 2

A_89 time = 0

A_110 time = 0

A_104 time = 0

A_135 time = 1

A_133 time = 0

A_134 time = 0

A_102 time = 0

A_163 time = 1

A_150 time = 1

A_148 time = 1

A_149 time = 0

A_187 time = 7

A_168 time = 1

A_169 time = 0

A_164 time = 1

A_165 time = 0

A_166 time = 1

A_167 time = 0

A_192 time = 0

A_256 time = 0

A_189 time = 0

A_188 time = 3

A_190 time = 3

A_191 time = 1

A_270 time = 0

A_271 time = 5

A_235 time = 1

A_236 time = 0

A_298 time = 2

A_461 time = 0

A_462 time = 0

A_103 time = 0

A_3 time = 1

A_21 time = 2

A_46 time = 0

A_45 time = 8

A_66 time = 0

A_68 time = 0

A_90 time = 0

A_91 time = 0

A_25 time = 0

A_26 time = 1

A_4 time = 2

A_27 time = 0

A_49 time = 0

A_22 time = 0

A_50 time = 7

A_23 time = 1

A_92 time = 0

A_114 time = 1

A_111 time = 0

A_137 time = 0

A_136 time = 0

A_85 time = 0

A_112 time = 0

A_86 time = 0

A_5 time = 3

A_24 time = 0

A_47 time = 0

A_13 time = 0

A_48 time = 6

A_67 time = 0

A_115 time = 0

A_116 time = 0

A_117 time = 0

A_118 time = 0

A_193 time = 0

A_238 time = 0

A_194 time = 2

A_237 time = 0

A_258 time = 1

A_257 time = 0

A_199 time = 1

A_463 time = 0

A_152 time = 1

A_202 time = 0

A_731 time = 0

A_734 time = 0

A_464 time = 0

A_736 time = 0

A_733 time = 0

A_737 time = 0

A_465 time = 1

A_466 time = 1

A_865 time = 0

A_740 time = 3

A_470 time = 1

A_694 time = 1

A_866 time = 0

A_741 time = 0

A_743 time = 0

A_1225 time = 0

A_300 time = 1

A_325 time = 0

A_323 time = 0

A_868 time = 0

A_867 time = 0

A_744 time = 1

A_1227 time = 1

A_537 time = 0

A_467 time = 0

A_870 time = 0

A_869 time = 0

A_745 time = 8

A_1229 time = 0

A_871 time = 0

A_471 time = 0

A_375 time = 0

A_472 time = 0

A_538 time = 1

A_468 time = 0

A_469 time = 0

A_873 time = 0

A_874 time = 1

A_539 time = 0

A_696 time = 2

A_695 time = 0

A_876 time = 0

A_877 time = 1

A_742 time = 1

A_1232 time = 0

A_875 time = 0

A_1236 time = 0

A_473 time = 0

A_540 time = 0

A_697 time = 1

A_698 time = 0

A_880 time = 1

A_881 time = 0

A_882 time = 0

A_878 time = 2

A_382 time = 0

A_474 time = 0

A_541 time = 0

A_700 time = 0

A_699 time = 0

A_884 time = 0

A_885 time = 0

A_886 time = 0

A_883 time = 4

A_475 time = 0

A_542 time = 0

A_887 time = 0

A_888 time = 0

A_889 time = 0

A_890 time = 0

A_476 time = 0

A_543 time = 1

A_893 time = 2

A_894 time = 0

A_477 time = 1

A_544 time = 1

A_478 time = 0

A_545 time = 0

A_891 time = 0

A_895 time = 0

A_899 time = 0

A_892 time = 0

A_896 time = 1

A_897 time = 0

A_900 time = 1

A_898 time = 0

A_902 time = 0

A_547 time = 0

A_906 time = 0

A_907 time = 0

A_703 time = 0

A_548 time = 0

A_908 time = 0

A_909 time = 0

A_903 time = 0

A_904 time = 0

A_942 time = 0

A_905 time = 0

A_704 time = 0

A_549 time = 0

A_910 time = 0

A_911 time = 0

A_1117 time = 0

A_705 time = 0

A_550 time = 0

A_912 time = 1

A_551 time = 1

A_941 time = 0

A_1415 time = 1

A_1418 time = 1

A_1438 time = 0

A_1443 time = 1

A_1424 time = 0

A_1444 time = 0

A_913 time = 0

A_1425 time = 1

A_1445 time = 1

A_914 time = 0

A_915 time = 0

A_1426 time = 0

A_916 time = 0

A_917 time = 0

A_1427 time = 0

A_710 time = 0

A_709 time = 1

A_554 time = 0

A_555 time = 0

A_918 time = 0

A_919 time = 0

A_1428 time = 0

A_556 time = 1

A_557 time = 0

A_920 time = 0

A_921 time = 0

A_1429 time = 0

A_716 time = 0

A_714 time = 0

A_715 time = 0

A_720 time = 1

A_717 time = 0

A_718 time = 0

A_723 time = 0

A_721 time = 0

A_722 time = 0

A_724 time = 0

A_725 time = 0

A_568 time = 0

A_726 time = 0

A_727 time = 0

A_569 time = 0

A_719 time = 0

A_558 time = 0

A_559 time = 0

A_560 time = 0

A_561 time = 1

A_562 time = 0

A_563 time = 0

A_564 time = 0

A_566 time = 0

A_567 time = 0

A_931 time = 0

A_935 time = 0

A_565 time = 0

A_924 time = 0

A_1124 time = 0

A_1122 time = 1

A_729 time = 0

A_939 time = 7

A_1126 time = 0

A_1247 time = 0

A_344 time = 6

A_1239 time = 1

A_1441 time = 0

A_1448 time = 0

A_1450 time = 0

A_1413 time = 0

A_282 time = 0

A_307 time = 4

A_328 time = 0

A_331 time = 1

A_346 time = 0

A_570 time = 0

A_345 time = 0

A_571 time = 0

A_355 time = 0

A_356 time = 0

A_364 time = 3

A_608 time = 1

A_610 time = 1

A_611 time = 0

A_1409 time = 0

A_380 time = 0

A_575 time = 0

A_613 time = 0

A_614 time = 0

A_383 time = 0

A_576 time = 0

A_616 time = 0

A_615 time = 0

A_385 time = 1

A_577 time = 1

A_617 time = 0

A_618 time = 0

A_578 time = 0

A_619 time = 0

A_1414 time = 0

A_1417 time = 6

A_387 time = 0

A_1439 time = 0

A_178 time = 0

A_210 time = 0

A_211 time = 0

A_479 time = 0

A_1131 time = 0

A_1140 time = 0

A_205 time = 0

A_207 time = 0

A_1132 time = 1

A_1137 time = 0

A_227 time = 1

A_500 time = 0

A_358 time = 0

A_359 time = 1

A_366 time = 0

A_501 time = 0

A_502 time = 3

A_503 time = 0

A_1157 time = 4

A_1162 time = 0

A_372 time = 1

A_504 time = 0

A_505 time = 0

A_1165 time = 0

A_377 time = 0

A_507 time = 0

A_508 time = 1

A_1166 time = 0

A_1167 time = 0

A_1168 time = 0

A_510 time = 0

A_509 time = 1

A_1169 time = 0

A_1170 time = 0

A_1171 time = 0

A_1172 time = 0

A_512 time = 0

A_511 time = 0

A_1175 time = 7

A_1176 time = 0

A_1174 time = 1

A_1177 time = 0

A_1178 time = 0

A_1179 time = 0

A_1180 time = 0

A_1181 time = 0

A_1195 time = 0

A_1182 time = 0

A_1493 time = 0

A_1183 time = 0

A_1196 time = 0

A_514 time = 0

A_513 time = 0

A_516 time = 0

A_515 time = 0

A_587 time = 0

A_588 time = 0

A_1189 time = 0

A_1202 time = 2

A_758 time = 0

A_756 time = 0

A_589 time = 0

A_590 time = 0

A_760 time = 1

A_759 time = 0

A_592 time = 0

A_594 time = 1

A_593 time = 0

A_762 time = 0

A_761 time = 0

A_595 time = 0

A_598 time = 0

A_596 time = 0

A_597 time = 0

A_763 time = 0

A_599 time = 1

A_601 time = 0

A_600 time = 1

A_1193 time = 0

A_1194 time = 0

A_1245 time = 1

A_1246 time = 0

A_764 time = 0

A_603 time = 0

A_602 time = 0

A_1204 time = 1

A_1205 time = 0

A_1485 time = 0

A_1486 time = 0

A_765 time = 0

A_604 time = 0

A_1206 time = 0

A_1207 time = 0

A_1208 time = 0

A_1487 time = 0

A_1488 time = 0

A_1209 time = 0

A_1499 time = 0

A_1504 time = 0

A_1490 time = 0

A_1210 time = 1

A_1500 time = 0

A_1505 time = 0

A_1211 time = 7

A_1501 time = 0

A_940 time = 6

A_1125 time = 0

A_1461 time = 1

A_1473 time = 0

A_1471 time = 0

A_1472 time = 0

A_1248 time = 0

A_1127 time = 1

A_1475 time = 0

A_1476 time = 0

A_1606 time = 0

A_1474 time = 1

A_1603 time = 0

A_1627 time = 0

A_1249 time = 0

A_1129 time = 0

A_1478 time = 0

A_1479 time = 1

A_1607 time = 1

A_1608 time = 0

A_1477 time = 0

A_1604 time = 1

A_1628 time = 1

A_1482 time = 0

A_1610 time = 0

A_1605 time = 0

A_1630 time = 0

A_1629 time = 0

A_1191 time = 1

A_1243 time = 0

A_1192 time = 0

A_1244 time = 0

A_1503 time = 0

A_225 time = 0

A_506 time = 0

A_226 time = 1

A_767 time = 0

A_776 time = 0

A_288 time = 1

A_771 time = 0

A_318 time = 0

A_351 time = 0

A_795 time = 5

A_361 time = 0

A_352 time = 0

A_353 time = 0

A_267 time = 0

A_320 time = 0

A_321 time = 1

A_369 time = 0

A_628 time = 0

A_947 time = 1

A_1647 time = 0

A_407 time = 3

A_406 time = 3

A_645 time = 0

A_644 time = 0

A_948 time = 0

A_1652 time = 0

A_648 time = 0

A_949 time = 0

A_655 time = 0

A_662 time = 0

A_663 time = 1

A_669 time = 0

A_668 time = 0

A_677 time = 0

A_678 time = 0

A_676 time = 0

A_802 time = 0

A_803 time = 0

A_804 time = 0

A_381 time = 0

A_421 time = 0

A_423 time = 0

A_422 time = 0

A_682 time = 0

A_1084 time = 0

A_681 time = 1

A_806 time = 0

A_807 time = 0

A_424 time = 0

A_425 time = 3

A_426 time = 0

A_687 time = 0

A_1085 time = 1

A_685 time = 0

A_386 time = 1

A_427 time = 1

A_428 time = 0

A_1086 time = 0

A_1063 time = 2

A_810 time = 0

A_811 time = 3

A_813 time = 0

A_814 time = 0

A_815 time = 0

A_817 time = 0

A_809 time = 0

A_818 time = 0

A_820 time = 0

A_822 time = 0

A_826 time = 0

A_522 time = 0

A_532 time = 6

A_827 time = 0

A_828 time = 0

A_523 time = 0

A_766 time = 0

A_829 time = 0

A_830 time = 0

A_831 time = 1

A_832 time = 1

A_526 time = 0

A_833 time = 0

A_834 time = 0

A_527 time = 0

A_528 time = 0

A_835 time = 0

A_836 time = 0

A_533 time = 0

A_529 time = 0

A_837 time = 0

A_838 time = 0

A_839 time = 0

A_840 time = 0

A_812 time = 0

A_816 time = 0

A_656 time = 0

A_1386 time = 0

A_1383 time = 0

A_1384 time = 0

A_1655 time = 0

A_1864 time = 0

A_664 time = 0

A_1388 time = 0

A_1387 time = 0

A_1731 time = 0

A_1654 time = 0

A_1657 time = 0

A_1865 time = 0

A_670 time = 3

A_1390 time = 0

A_1389 time = 0

A_1732 time = 0

A_1977 time = 0

A_1656 time = 0

A_1659 time = 1

A_1866 time = 0

A_1392 time = 0

A_1391 time = 0

A_1733 time = 0

A_1978 time = 0

A_1653 time = 0

A_1514 time = 1

A_1662 time = 0

A_1667 time = 0

A_1869 time = 0

A_1397 time = 0

A_1398 time = 1

A_1551 time = 0

A_1736 time = 1

A_1737 time = 0

A_1981 time = 0

A_951 time = 0

A_952 time = 0

A_1071 time = 1

A_989 time = 0

A_995 time = 1

A_1361 time = 1

A_1663 time = 0

A_1668 time = 5

A_1870 time = 0

A_1399 time = 0

A_1555 time = 0

A_1552 time = 0

A_1738 time = 2

A_1859 time = 0

A_1982 time = 0

A_1255 time = 0

A_1293 time = 0

A_1072 time = 0

A_991 time = 0

A_998 time = 5

A_1362 time = 0

A_1554 time = 0

A_1860 time = 0

A_1861 time = 0

A_993 time = 0

A_994 time = 0

A_1863 time = 2

A_1862 time = 2

A_1664 time = 0

A_1665 time = 1

A_1669 time = 0

A_1670 time = 0

A_1871 time = 0

A_1682 time = 0

A_1749 time = 0

A_1890 time = 0

A_1889 time = 0

A_396 time = 1

A_397 time = 0

A_437 time = 11

A_438 time = 0

A_956 time = 0

A_1095 time = 0

A_1751 time = 0

A_1894 time = 0

A_1893 time = 0

A_441 time = 0

A_442 time = 0

A_439 time = 0

A_440 time = 0

A_957 time = 0

A_1096 time = 0

A_445 time = 0

A_446 time = 1

A_958 time = 1

A_1097 time = 0

A_458 time = 0

A_457 time = 0

A_851 time = 0

A_852 time = 0

A_856 time = 0

A_854 time = 0

A_855 time = 0

A_459 time = 1

A_857 time = 0

A_858 time = 1

A_859 time = 0

A_1212 time = 0

A_460 time = 0

A_860 time = 0

A_861 time = 1

A_862 time = 0

A_1213 time = 0

A_965 time = 0

A_1507 time = 0

A_1492 time = 0

A_863 time = 0

A_1214 time = 0

A_864 time = 0

A_1215 time = 0

A_1331 time = 6

A_1671 time = 0

A_1673 time = 0

A_1674 time = 0

A_1739 time = 0

A_1675 time = 0

A_1380 time = 0

A_1676 time = 0

A_1740 time = 0

A_1677 time = 0

A_1678 time = 0

A_1382 time = 0

A_1679 time = 1

A_1741 time = 0

A_1742 time = 5

A_1743 time = 0

A_1680 time = 0

A_1744 time = 0

A_1745 time = 0

A_1683 time = 0

A_1684 time = 0

A_1753 time = 0

A_1685 time = 0

A_1755 time = 0

A_1400 time = 0

A_1872 time = 0

A_1672 time = 0

A_1069 time = 0

A_1873 time = 0

A_1746 time = 0

A_1401 time = 0

A_1556 time = 0

A_1543 time = 0

A_1573 time = 0

A_1557 time = 0

A_1544 time = 0

A_1574 time = 0

A_1558 time = 0

A_1874 time = 0

A_1875 time = 0

A_1545 time = 0

A_1575 time = 0

A_1877 time = 0

A_1876 time = 0

A_1546 time = 0

A_1576 time = 0

A_1879 time = 0

A_1880 time = 0

A_1878 time = 0

A_1547 time = 0

A_1883 time = 0

A_1882 time = 0

A_1881 time = 0

A_1548 time = 1

A_1887 time = 0

A_1886 time = 0

A_1884 time = 0

A_1897 time = 0

A_1891 time = 0

A_1896 time = 0

A_1748 time = 0

A_1579 time = 0

A_1580 time = 0

A_1581 time = 0

A_1566 time = 0

A_1997 time = 0

A_1996 time = 0

A_1998 time = 1

A_1999 time = 0

A_1263 time = 0

A_1301 time = 0

A_1017 time = 0

A_2002 time = 0

A_1264 time = 0

A_1302 time = 0

A_1019 time = 0

A_2006 time = 0

A_1265 time = 0

A_1303 time = 0

A_1022 time = 0

A_1023 time = 0

A_1266 time = 0

A_1304 time = 0

A_1026 time = 1

A_1027 time = 0

A_1567 time = 0

A_1568 time = 0

A_2001 time = 0

A_2000 time = 0

A_1569 time = 2

A_1570 time = 0

A_2004 time = 0

A_2003 time = 0

A_2008 time = 0

A_2007 time = 0

A_1892 time = 0

A_1750 time = 0

A_1895 time = 0

A_1752 time = 0

A_1582 time = 0

A_1571 time = 0

A_1572 time = 0

A_1899 time = 0

A_1754 time = 0

A_1583 time = 0

A_1776 time = 0

A_1777 time = 0

A_2011 time = 0

A_2010 time = 0

A_1902 time = 0

A_1756 time = 0

A_2015 time = 0

A_2014 time = 0

A_1906 time = 0

A_1758 time = 0

A_1585 time = 0

A_2018 time = 0

A_2017 time = 0

A_1909 time = 0

A_1760 time = 0

A_1586 time = 0

A_2022 time = 0

A_2021 time = 0

A_1913 time = 0

A_1762 time = 0

A_1587 time = 0

A_2025 time = 0

A_2024 time = 0

A_1688 time = 0

A_1761 time = 0

A_1911 time = 0

A_1910 time = 0

A_1334 time = 0

A_1105 time = 0

A_1103 time = 0

A_1104 time = 0

A_1689 time = 0

A_1763 time = 0

A_1915 time = 0

A_1914 time = 0

A_1336 time = 0

A_1108 time = 0

A_1106 time = 1

A_1107 time = 0

A_1690 time = 0

A_1766 time = 0

A_1917 time = 0

A_1916 time = 0

A_1764 time = 1

A_1765 time = 0

A_1588 time = 0

A_1589 time = 0

A_2029 time = 0

A_1339 time = 0

A_1112 time = 8

A_1109 time = 0

A_1110 time = 27

A_1111 time = 0

A_1691 time = 0

A_1767 time = 0

A_1919 time = 0

A_1918 time = 0

A_1775 time = 0

A_1778 time = 0

A_1591 time = 0

A_1590 time = 0

A_2031 time = 0

A_1341 time = 0

A_1115 time = 6

A_1113 time = 1

A_1114 time = 1

A_1692 time = 1

A_1693 time = 0

A_1768 time = 1

A_1921 time = 0

A_1920 time = 0

A_1779 time = 0

A_1780 time = 0

A_1781 time = 0

A_1594 time = 0

A_1592 time = 0

A_1344 time = 0

A_1345 time = 0

A_1116 time = 0

A_1402 time = 0

A_1770 time = 0

A_1804 time = 0

A_1803 time = 0

A_1923 time = 0

A_1922 time = 0

A_1782 time = 0

A_1783 time = 1

A_1784 time = 0

A_1596 time = 0

A_1595 time = 1

A_1217 time = 0

A_1347 time = 0

A_1349 time = 0

A_1348 time = 1

A_1609 time = 0

A_1480 time = 8

A_1483 time = 0

A_1613 time = 0

A_1611 time = 0

A_1612 time = 0

A_1632 time = 0

A_1633 time = 0

A_1634 time = 1

A_1351 time = 0

A_1928 time = 0

A_1929 time = 0

A_1616 time = 0

A_1615 time = 0

A_1614 time = 0

A_1635 time = 0

A_1636 time = 0

A_1637 time = 0

A_1353 time = 0

A_1773 time = 0

A_1809 time = 0

A_1810 time = 0

A_1811 time = 0

A_1933 time = 0

A_1932 time = 0

A_1931 time = 0

A_1930 time = 0

A_1619 time = 0

A_1618 time = 0

A_1617 time = 0

A_1638 time = 0

A_1639 time = 0

A_1640 time = 0

A_1354 time = 0

A_1774 time = 0

A_1812 time = 0

A_1814 time = 0

A_1813 time = 0

A_1939 time = 1

A_1937 time = 0

A_1938 time = 1

A_1936 time = 0

A_1934 time = 0

A_1935 time = 0

A_1622 time = 0

A_1621 time = 0

A_1620 time = 0

A_1641 time = 0

A_1642 time = 0

A_1643 time = 1

A_1355 time = 0

A_1815 time = 0

A_1818 time = 0

A_1816 time = 0

A_1817 time = 0

A_1945 time = 0

A_1944 time = 0

A_1943 time = 0

A_1942 time = 0

A_1941 time = 0

A_1940 time = 0

A_1624 time = 0

A_1623 time = 1

A_1644 time = 0

A_1645 time = 0

A_1819 time = 2

A_1821 time = 0

A_1820 time = 0

A_1953 time = 0

A_1952 time = 0

A_1950 time = 0

A_1951 time = 0

A_1949 time = 0

A_1948 time = 0

A_1946 time = 1

A_1947 time = 0

A_1626 time = 0

A_1625 time = 0

A_1646 time = 0

A_1823 time = 1

A_1961 time = 18

A_1352 time = 0

A_1404 time = 0

A_1405 time = 0

A_1771 time = 0

A_1772 time = 0

A_1805 time = 0

A_1807 time = 1

A_1824 time = 0

A_1825 time = 0

A_1806 time = 0

A_1925 time = 0

A_1808 time = 0

A_1822 time = 0

A_1924 time = 0

A_1926 time = 0

A_1785 time = 0

A_1584 time = 0

A_1484 time = 0

A_1350 time = 0

A_1403 time = 0

A_1769 time = 0

A_1927 time = 0

A_1789 time = 9

A_1788 time = 0

A_1790 time = 1

A_1791 time = 0

A_1793 time = 0

A_1786 time = 0

A_1794 time = 0

A_1787 time = 0

A_1797 time = 0

A_1960 time = 0

A_1958 time = 0

A_1959 time = 0

A_1957 time = 43

A_1956 time = 0

A_1955 time = 0

A_1954 time = 0

A_1969 time = 0

A_1966 time = 0

A_1967 time = 0

A_1968 time = 0

A_1965 time = 0

A_1964 time = 1

A_1962 time = 0

A_1973 time = 4

A_1972 time = 0

A_2109 time = 0

A_1971 time = 3

A_1970 time = 1

A_1974 time = 0

A_1975 time = 3

A_2110 time = 0

A_2111 time = 0

A_2107 time = 1

A_2108 time = 0

A_1593 time = 0

A_2005 time = 0

A_2009 time = 0

A_2012 time = 0

A_2026 time = 0

A_2028 time = 0

A_2030 time = 0

A_2033 time = 0

A_2032 time = 0

A_2035 time = 0

A_2034 time = 0

A_1032 time = 0

A_1524 time = 1

A_1036 time = 0

A_1525 time = 0

A_1365 time = 0

A_1526 time = 0

A_1367 time = 0

A_1527 time = 0

A_1718 time = 1

A_1719 time = 0

A_1697 time = 1

A_1720 time = 0

A_1698 time = 0

A_1528 time = 0

A_1271 time = 0

A_1308 time = 1

A_1309 time = 0

A_1037 time = 0

A_1038 time = 0

A_1529 time = 1

A_1702 time = 0

A_1272 time = 0

A_1310 time = 0

A_1311 time = 0

A_1039 time = 0

A_1040 time = 0

A_1530 time = 0

A_1703 time = 0

A_2023 time = 0

A_1270 time = 1

A_2027 time = 0

A_1273 time = 0

A_1366 time = 0

A_1274 time = 0

A_1275 time = 0

A_1312 time = 0

A_1313 time = 0

A_1314 time = 0

A_1041 time = 0

A_1042 time = 1

A_1531 time = 0

A_1279 time = 0

A_1277 time = 1

A_1278 time = 0

A_1315 time = 0

A_1317 time = 0

A_1316 time = 0

A_1043 time = 0

A_1044 time = 0

A_1532 time = 0

A_1318 time = 0

A_1321 time = 0

A_1319 time = 1

A_1322 time = 0

A_1324 time = 0

A_1323 time = 1

A_1325 time = 0

A_1371 time = 0

A_1283 time = 0

A_1280 time = 0

A_1281 time = 0

A_1286 time = 0

A_1284 time = 1

A_1285 time = 0

A_1287 time = 0

A_1288 time = 0

A_1326 time = 0

A_1289 time = 0

A_1290 time = 0

A_1328 time = 0

A_2041 time = 0

A_2038 time = 0

A_2039 time = 0

A_2044 time = 0

A_2043 time = 3

A_2042 time = 0

A_2049 time = 1

A_2050 time = 0

A_2048 time = 0

A_2036 time = 0

A_2046 time = 0

A_2047 time = 0

A_1800 time = 0

A_1801 time = 0

A_2056 time = 0

A_2054 time = 0

A_2052 time = 0

A_2053 time = 0

A_2058 time = 1

A_2059 time = 0

A_1045 time = 1

A_1049 time = 1

A_1369 time = 0

A_1533 time = 1

A_1706 time = 1

A_1051 time = 0

A_1534 time = 0

A_1707 time = 0

A_1053 time = 0

A_1054 time = 0

A_1535 time = 0

A_1056 time = 0

A_1536 time = 0

A_1537 time = 0

A_1058 time = 0

A_1538 time = 0

A_1711 time = 0

A_1060 time = 0

A_1539 time = 1

A_1712 time = 1

A_1713 time = 0

A_1827 time = 0

A_1832 time = 0

A_1061 time = 0

A_1540 time = 0

A_1840 time = 0

A_1845 time = 0

A_1828 time = 0

A_1834 time = 0

A_1835 time = 0

A_1062 time = 0

A_1541 time = 0

A_1841 time = 0

A_1846 time = 0

A_1847 time = 0

A_1829 time = 1

A_1836 time = 0

A_2094 time = 1

A_1542 time = 20

A_1374 time = 0

A_1057 time = 1

A_1377 time = 7

A_1376 time = 0

A_1059 time = 7

A_1378 time = 0

A_1379 time = 0

A_2072 time = 0

A_2069 time = 43

A_2066 time = 1

A_2067 time = 0

A_2068 time = 0

A_2081 time = 0

A_2074 time = 0

A_2075 time = 0

A_2076 time = 0

A_2112 time = 1

A_2084 time = 18

A_2085 time = 0

A_2091 time = 0

A_2082 time = 10

A_2083 time = 0

A_1730 time = 1

A_1052 time = 1

A_1372 time = 0

A_1375 time = 1

A_2080 time = 0

A_1728 time = 0

A_1055 time = 0

A_1373 time = 0

A_2071 time = 0

A_1727 time = 0

A_1729 time = 0

A_1705 time = 0

A_1048 time = 0

A_1327 time = 0

A_2079 time = 0

A_1726 time = 0

A_1704 time = 0

A_1050 time = 2

A_1370 time = 0

A_1724 time = 0

A_1725 time = 0

A_1701 time = 0

A_1047 time = 0

A_1330 time = 0

A_2070 time = 0

A_1723 time = 0

A_1700 time = 0

A_1368 time = 0

A_1329 time = 0

A_2060 time = 0

A_1699 time = 0

A_1046 time = 0

A_1292 time = 0

A_2078 time = 1

A_2088 time = 4

A_1320 time = 0

A_1291 time = 0

A_1963 time = 0

A_2086 time = 0

A_1282 time = 0

A_1721 time = 0

A_1722 time = 0

A_1034 time = 0

A_1276 time = 0

A_2093 time = 0

A_1717 time = 0

A_1696 time = 0

A_1031 time = 0

A_2045 time = 0

A_2090 time = 14

A_1523 time = 0

A_1029 time = 0

A_2040 time = 0

A_2092 time = 0

A_1025 time = 0

A_1033 time = 0

A_2051 time = 0

A_2087 time = 0

A_1716 time = 0

A_1695 time = 0

A_1028 time = 0

A_1601 time = 0

A_2114 time = 0

A_1035 time = 1

A_1715 time = 0

A_1522 time = 0

A_1030 time = 0

A_1599 time = 0

A_2113 time = 0

A_1024 time = 0

A_1714 time = 0

A_1520 time = 0

A_1694 time = 0

A_1021 time = 0

A_1307 time = 0

A_1597 time = 0

A_2073 time = 33

A_1020 time = 0

A_1517 time = 0

A_1518 time = 0

A_1521 time = 0

A_1306 time = 0

A_1602 time = 0

A_2061 time = 1

A_1516 time = 0

A_1018 time = 0

A_1519 time = 0

A_1014 time = 0

A_1305 time = 0

A_1796 time = 0

A_2064 time = 0

A_1515 time = 0

A_1013 time = 1

A_1267 time = 0

A_1600 time = 8

A_2063 time = 0

A_1016 time = 0

A_1268 time = 1

A_1598 time = 0

A_2062 time = 0

A_1012 time = 0

A_1269 time = 0

A_1799 time = 1

A_2065 time = 0

A_1011 time = 0

A_2020 time = 0

A_1792 time = 0

A_2055 time = 0

A_1083 time = 0

A_2016 time = 0

A_1795 time = 0

A_2057 time = 0

A_2037 time = 0

A_2013 time = 0

A_2019 time = 0

A_1798 time = 0

A_1802 time = 0

A_1009 time = 0

A_1015 time = 0

A_1101 time = 0

A_1342 time = 0

A_1010 time = 6

A_1300 time = 0

A_1102 time = 0

A_1346 time = 0

A_1006 time = 0

A_1262 time = 0

A_1332 time = 0

A_1340 time = 0

A_999 time = 0

A_1995 time = 0

A_1757 time = 1

A_1343 time = 0

A_1005 time = 0

A_1993 time = 0

A_1759 time = 0

A_1216 time = 0

A_1002 time = 0

A_1008 time = 0

A_1994 time = 0

A_1687 time = 0

A_1338 time = 1

A_1004 time = 0

A_1565 time = 0

A_1299 time = 0

A_1082 time = 0

A_1364 time = 0

A_1001 time = 0

A_1003 time = 1

A_1007 time = 0

A_1261 time = 0

A_1081 time = 0

A_1363 time = 1

A_996 time = 0

A_1000 time = 0

A_1992 time = 0

A_1686 time = 0

A_1337 time = 0

A_997 time = 0

A_1078 time = 0

A_1079 time = 0

A_1991 time = 0

A_1100 time = 0

A_1912 time = 0

A_1075 time = 0

A_1990 time = 0

A_1099 time = 1

A_1905 time = 0

A_1074 time = 0

A_1077 time = 1

A_1080 time = 0

A_1298 time = 0

A_444 time = 0

A_1907 time = 0

A_1076 time = 2

A_1297 time = 0

A_961 time = 0

A_1903 time = 0

A_1296 time = 0

A_1260 time = 0

A_959 time = 1

A_1908 time = 0

A_1073 time = 1

A_1295 time = 0

A_1259 time = 0

A_1989 time = 0

A_443 time = 1

A_1904 time = 0

A_1898 time = 0

A_1888 time = 0

A_1564 time = 0

A_1294 time = 0

A_1257 time = 1

A_1258 time = 0

A_1987 time = 0

A_1563 time = 0

A_1578 time = 0

A_1256 time = 0

A_1985 time = 0

A_1988 time = 0

A_448 time = 0

A_1335 time = 0

A_1900 time = 0

A_1549 time = 0

A_1577 time = 0

A_1983 time = 0

A_1984 time = 0

A_672 time = 0

A_983 time = 0

A_1986 time = 0

A_1562 time = 0

A_960 time = 0

A_1333 time = 0

A_1901 time = 0

A_1885 time = 0

A_1553 time = 0

A_1559 time = 1

A_1560 time = 0

A_1561 time = 0

A_607 time = 0

A_963 time = 0

A_964 time = 1

A_1067 time = 0

A_1550 time = 0

A_606 time = 1

A_962 time = 0

A_1360 time = 0

A_1513 time = 1

A_1094 time = 0

A_456 time = 0

A_679 time = 4

A_1359 time = 1

A_992 time = 0

A_1070 time = 0

A_955 time = 0

A_849 time = 1

A_671 time = 0

A_990 time = 0

A_987 time = 0

A_675 time = 0

A_950 time = 1

A_436 time = 0

A_850 time = 0

A_665 time = 0

A_985 time = 0

A_674 time = 0

A_1980 time = 0

A_395 time = 0

A_848 time = 0

A_974 time = 2

A_1979 time = 0

A_1747 time = 0

A_847 time = 0

A_972 time = 0

A_973 time = 1

A_1358 time = 0

A_1735 time = 0

A_1066 time = 0

A_1098 time = 0

A_971 time = 1

A_1357 time = 0

A_988 time = 0

A_1734 time = 0

A_1681 time = 1

A_605 time = 0

A_673 time = 0

A_657 time = 0

A_986 time = 0

A_982 time = 1

A_1393 time = 1

A_1093 time = 0

A_454 time = 0

A_970 time = 0

A_1395 time = 0

A_954 time = 0

A_450 time = 0

A_969 time = 0

A_666 time = 1

A_1356 time = 1

A_1396 time = 0

A_435 time = 0

A_447 time = 0

A_658 time = 0

A_338 time = 4

A_984 time = 1

A_981 time = 1

A_1394 time = 1

A_1092 time = 0

A_453 time = 1

A_968 time = 1

A_315 time = 0

A_980 time = 0

A_1868 time = 0

A_953 time = 0

A_451 time = 0

A_652 time = 0

A_248 time = 0

A_635 time = 0

A_651 time = 0

A_979 time = 0

A_1867 time = 0

A_434 time = 1

A_455 time = 0

A_659 time = 0

A_667 time = 0

A_247 time = 0

A_634 time = 0

A_978 time = 0

A_1661 time = 0

A_1091 time = 0

A_452 time = 0

A_414 time = 2

A_977 time = 0

A_1666 time = 0

A_693 time = 0

A_449 time = 1

A_413 time = 1

A_653 time = 0

A_967 time = 0

A_976 time = 0

A_1658 time = 3

A_433 time = 1

A_846 time = 0

A_411 time = 0

A_646 time = 0

A_633 time = 0

A_1660 time = 0

A_1089 time = 0

A_842 time = 0

A_410 time = 0

A_975 time = 0

A_1512 time = 0

A_1090 time = 0

A_845 time = 0

A_350 time = 1

A_647 time = 0

A_638 time = 0

A_966 time = 0

A_1510 time = 0

A_691 time = 0

A_841 time = 0

A_408 time = 1

A_625 time = 0

A_637 time = 0

A_642 time = 1

A_1511 time = 1

A_692 time = 0

A_844 time = 0

A_409 time = 1

A_246 time = 0

A_624 time = 0

A_636 time = 0

A_1406 time = 1

A_432 time = 0

A_394 time = 1

A_339 time = 0

A_349 time = 0

A_639 time = 0

A_643 time = 0

A_684 time = 1

A_431 time = 0

A_393 time = 0

A_405 time = 0

A_680 time = 3

A_430 time = 0

A_1381 time = 0

A_623 time = 0

A_401 time = 1

A_404 time = 0

A_1234 time = 0

A_429 time = 0

A_392 time = 0

A_251 time = 1

A_399 time = 0

A_317 time = 0

A_1385 time = 0

A_1976 time = 0

A_234 time = 0

A_250 time = 0

A_294 time = 0

A_316 time = 0

A_1651 time = 0

A_388 time = 0

A_1068 time = 0

A_1221 time = 1

A_1233 time = 0

A_1064 time = 0

A_1065 time = 0

A_632 time = 2

A_1220 time = 0

A_661 time = 0

A_686 time = 0

A_1088 time = 0

A_266 time = 0

A_631 time = 0

A_650 time = 0

A_1650 time = 0

A_389 time = 0

A_690 time = 0

A_1087 time = 0

A_683 time = 0

A_1230 time = 0

A_1649 time = 0

A_390 time = 0

A_824 time = 0

A_843 time = 1

A_853 time = 0

A_688 time = 0

A_654 time = 1

A_660 time = 0

A_630 time = 0

A_1648 time = 1

A_391 time = 0

A_819 time = 0

A_823 time = 0

A_689 time = 0

A_622 time = 0

A_627 time = 0

A_640 time = 0

A_420 time = 1

A_821 time = 0

A_249 time = 0

A_621 time = 0

A_626 time = 0

A_641 time = 0

A_417 time = 0

A_825 time = 0

A_620 time = 0

A_629 time = 0

A_403 time = 0

A_374 time = 2

A_384 time = 0

A_255 time = 0

A_398 time = 0

A_402 time = 0

A_419 time = 0

A_808 time = 0

A_233 time = 1

A_254 time = 0

A_400 time = 0

A_649 time = 2

A_362 time = 1

A_368 time = 0

A_232 time = 0

A_253 time = 0

A_268 time = 0

A_296 time = 0

A_416 time = 0

A_531 time = 1

A_373 time = 1

A_1491 time = 1

A_1509 time = 0

A_186 time = 2

A_252 time = 0

A_314 time = 1

A_319 time = 1

A_341 time = 0

A_415 time = 0

A_1506 time = 0

A_1508 time = 0

A_313 time = 2

A_340 time = 0

A_412 time = 0

A_367 time = 0

A_799 time = 5

A_162 time = 19

A_231 time = 0

A_293 time = 2

A_310 time = 4

A_295 time = 0

A_790 time = 0

A_792 time = 1

A_772 time = 0

A_418 time = 0

A_517 time = 1

A_1489 time = 0

A_770 time = 0

A_147 time = 0

A_230 time = 0

A_378 time = 0

A_519 time = 0

A_535 time = 1

A_775 time = 0

A_229 time = 0

A_801 time = 0

A_521 time = 0

A_534 time = 0

A_769 time = 0

A_774 time = 0

A_220 time = 0

A_800 time = 0

A_518 time = 0

A_530 time = 0

A_768 time = 0

A_773 time = 0

A_219 time = 0

A_798 time = 0

A_520 time = 0

A_536 time = 0

A_132 time = 1

A_185 time = 0

A_796 time = 0

A_1496 time = 0

A_1498 time = 0

A_218 time = 0

A_161 time = 0

A_797 time = 0

A_805 time = 0

A_524 time = 0

A_1203 time = 0

A_224 time = 0

A_1495 time = 0

A_1497 time = 0

A_1190 time = 4

A_1502 time = 0

A_1631 time = 0

A_1494 time = 0

A_1197 time = 0

A_291 time = 0

A_336 time = 1

A_1161 time = 1

A_1173 time = 0

A_290 time = 0

A_311 time = 7

A_1164 time = 0

A_754 time = 0

A_1469 time = 0

A_791 time = 0

A_360 time = 0

A_755 time = 0

A_1470 time = 0

A_788 time = 0

A_794 time = 1

A_1156 time = 12

A_525 time = 1

A_1460 time = 0

A_789 time = 0

A_793 time = 0

A_347 time = 1

A_1201 time = 0

A_1459 time = 1

A_785 time = 0

A_499 time = 0

A_1188 time = 0

A_1253 time = 0

A_786 time = 0

A_348 time = 0

A_586 time = 0

A_1121 time = 0

A_498 time = 2

A_1153 time = 0

A_585 time = 0

A_938 time = 0

A_497 time = 1

A_1149 time = 1

A_753 time = 0

A_1458 time = 0

A_787 time = 1

A_1150 time = 0

A_752 time = 1

A_1457 time = 0

A_782 time = 0

A_750 time = 2

A_1200 time = 0

A_1435 time = 0

A_783 time = 0

A_749 time = 1

A_1187 time = 0

A_1434 time = 1

A_496 time = 0

A_333 time = 1

A_584 time = 0

A_1252 time = 0

A_495 time = 0

A_335 time = 0

A_751 time = 0

A_1118 time = 0

A_337 time = 1

A_334 time = 0

A_1199 time = 0

A_936 time = 0

A_784 time = 0

A_1151 time = 0

A_1186 time = 0

A_1119 time = 0

A_779 time = 0

A_1146 time = 0

A_583 time = 1

A_1456 time = 0

A_780 time = 0

A_1147 time = 3

A_488 time = 0

A_1433 time = 1

A_493 time = 0

A_748 time = 0

A_1464 time = 1

A_933 time = 0

A_494 time = 3

A_747 time = 0

A_1198 time = 0

A_932 time = 0

A_312 time = 0

A_332 time = 2

A_1185 time = 0

A_934 time = 0

A_781 time = 0

A_308 time = 0

A_582 time = 0

A_946 time = 0

A_777 time = 0

A_309 time = 0

A_487 time = 1

A_1455 time = 0

A_778 time = 0

A_1148 time = 0

A_1449 time = 0

A_1454 time = 0

A_492 time = 0

A_1143 time = 0

A_1184 time = 0

A_1453 time = 1

A_491 time = 0

A_1144 time = 0

A_581 time = 1

A_1431 time = 1

A_292 time = 0

A_746 time = 0

A_580 time = 0

A_1432 time = 0

A_289 time = 0

A_483 time = 1

A_1163 time = 0

A_591 time = 0

A_490 time = 0

A_306 time = 0

A_579 time = 0

A_757 time = 0

A_489 time = 0

A_265 time = 0

A_287 time = 0

A_486 time = 0

A_1452 time = 0

A_245 time = 0

A_264 time = 0

A_286 time = 1

A_485 time = 0

A_1451 time = 2

A_244 time = 0

A_263 time = 0

A_1145 time = 0

A_484 time = 0

A_1468 time = 0

A_243 time = 1

A_1141 time = 0

A_223 time = 0

A_183 time = 0

A_221 time = 1

A_1142 time = 0

A_1155 time = 0

A_303 time = 6

A_222 time = 0

A_182 time = 1

A_217 time = 0

A_482 time = 0

A_1152 time = 0

A_100 time = 0

A_160 time = 1

A_181 time = 0

A_481 time = 0

A_1154 time = 1

A_1136 time = 0

A_1442 time = 0

A_1467 time = 3

A_228 time = 0

A_279 time = 1

A_1135 time = 0

A_1446 time = 0

A_1466 time = 0

A_180 time = 0

A_283 time = 0

A_1160 time = 0

A_1134 time = 2

A_80 time = 0

A_131 time = 0

A_146 time = 0

A_157 time = 1

A_480 time = 2

A_1158 time = 0

A_1139 time = 0

A_61 time = 0

A_130 time = 0

A_144 time = 0

A_159 time = 0

A_1133 time = 0

A_1138 time = 0

A_39 time = 0

A_127 time = 0

A_184 time = 1

A_206 time = 0

A_38 time = 0

A_62 time = 0

A_79 time = 0

A_99 time = 0

A_261 time = 1

A_1159 time = 0

A_612 time = 0

A_929 time = 1

A_1481 time = 7

A_304 time = 0

A_285 time = 0

A_609 time = 0

A_1128 time = 0

A_1251 time = 0

A_284 time = 1

A_209 time = 0

A_241 time = 1

A_574 time = 0

A_1130 time = 0

A_1250 time = 0

A_242 time = 0

A_573 time = 0

A_1419 time = 0

A_1465 time = 0

A_1463 time = 0

A_214 time = 0

A_262 time = 0

A_281 time = 1

A_572 time = 0

A_213 time = 0

A_371 time = 1

A_927 time = 0

A_1462 time = 0

A_212 time = 2

A_177 time = 1

A_376 time = 0

A_923 time = 0

A_1254 time = 0

A_208 time = 1

A_176 time = 0

A_37 time = 0

A_60 time = 0

A_78 time = 2

A_97 time = 0

A_129 time = 0

A_158 time = 0

A_1240 time = 0

A_930 time = 0

A_1123 time = 0

A_175 time = 0

A_216 time = 2

A_75 time = 0

A_145 time = 1

A_154 time = 0

A_330 time = 0

A_357 time = 1

A_57 time = 0

A_128 time = 0

A_156 time = 0

A_329 time = 0

A_365 time = 0

A_926 time = 0

A_945 time = 1

A_937 time = 0

A_327 time = 0

A_922 time = 1

A_944 time = 1

A_728 time = 1

A_1410 time = 0

A_1447 time = 0

A_1430 time = 0

A_305 time = 0

A_925 time = 0

A_928 time = 0

A_1120 time = 0

A_326 time = 1

A_302 time = 0

A_36 time = 0

A_58 time = 0

A_77 time = 0

A_142 time = 0

A_240 time = 0

A_280 time = 0

A_35 time = 0

A_59 time = 0

A_98 time = 0

A_124 time = 1

A_260 time = 0

A_278 time = 1

A_1224 time = 0

A_1407 time = 0

A_1437 time = 1

A_1440 time = 0

A_215 time = 0

A_299 time = 4

A_1237 time = 0

A_1436 time = 0

A_553 time = 1

A_712 time = 0

A_1223 time = 0

A_1242 time = 1

A_1421 time = 1

A_1423 time = 0

A_197 time = 0

A_9 time = 1

A_34 time = 0

A_76 time = 1

A_126 time = 0

A_179 time = 0

A_1235 time = 0

A_1420 time = 0

A_552 time = 0

A_125 time = 0

A_1231 time = 0

A_1416 time = 0

A_708 time = 0

A_711 time = 0

A_277 time = 0

A_1412 time = 0

A_707 time = 0

A_713 time = 0

A_1219 time = 0

A_1411 time = 0

A_1422 time = 0

A_943 time = 0

A_259 time = 0

A_1238 time = 1

A_54 time = 0

A_95 time = 0

A_272 time = 1

A_739 time = 3

A_1228 time = 0

A_1241 time = 0

A_901 time = 0

A_546 time = 0

A_702 time = 0

A_123 time = 0

A_196 time = 0

A_1226 time = 0

A_1408 time = 0

A_701 time = 0

A_706 time = 0

A_879 time = 0

A_195 time = 0

A_1222 time = 1

A_33 time = 0

A_72 time = 0

A_96 time = 0

A_143 time = 0

A_239 time = 0

A_1218 time = 1

A_872 time = 0

A_379 time = 2

A_204 time = 0

A_738 time = 0

A_301 time = 0

A_370 time = 0

A_32 time = 0

A_55 time = 0

A_74 time = 0

A_174 time = 0

A_203 time = 0

A_342 time = 0

A_343 time = 0

A_122 time = 0

A_8 time = 1

A_31 time = 1

A_56 time = 1

A_73 time = 1

A_94 time = 0

A_155 time = 0

A_732 time = 0

A_735 time = 0

A_71 time = 0

A_153 time = 2

A_276 time = 2

A_363 time = 0

A_121 time = 1

A_173 time = 1

A_200 time = 0

A_730 time = 3

A_70 time = 0

A_141 time = 0

A_172 time = 0

A_198 time = 0

A_53 time = 1

A_120 time = 0

A_275 time = 0

A_324 time = 1

A_274 time = 0

A_201 time = 1

A_30 time = 0

A_52 time = 0

A_140 time = 1

A_322 time = 0

A_354 time = 0

A_273 time = 0

A_171 time = 0

A_29 time = 0

A_51 time = 0

A_93 time = 1

A_119 time = 0

A_139 time = 0

A_151 time = 1

A_170 time = 0

A_28 time = 0

A_42 time = 7

A_88 time = 0

A_138 time = 0

A_7 time = 0

A_15 time = 47

A_69 time = 1

A_87 time = 0

A_113 time = 0

A_6 time = 0

A_14 time = 0

A_41 time = 0

A_64 time = 1

A_1708 time = 0

A_1709 time = 1

A_1826 time = 0

A_1710 time = 0

A_1831 time = 0

A_1833 time = 1

A_2095 time = 0

A_1842 time = 0

A_1848 time = 1

A_1853 time = 0

A_1830 time = 83

A_1837 time = 0

A_2096 time = 0

A_2097 time = 0

A_2077 time = 1

A_1843 time = 0

A_1849 time = 0

A_1854 time = 0

A_1855 time = 0

A_1838 time = 0

A_2098 time = 0

A_2099 time = 0

A_2089 time = 0

A_1844 time = 3

A_1850 time = 0

A_1856 time = 0

A_1839 time = 4

A_2101 time = 0

A_2115 time = 0

A_2116 time = 0

A_1851 time = 0

A_1852 time = 6

A_1857 time = 0

A_1858 time = 0

A_2102 time = 1

A_2100 time = 0

A_2104 time = 0

A_2105 time = 0

A_2106 time = 0

A_2103 time = 0

Fig. 1. Proof search for verifying a switching damped oscillator

The basic principle of our programming model allows for distributed parallel proof search in the form of annotated and/or trees. Specifically, we consider a (tableau-style) backwards proof search procedure for a sequent calculus for hybrid systems [3, 2]. A proof is a tree of sequents in which all leaves are provable by axioms or arithmetic decision procedures. During proof search, however, the proof structure has multiple choice points, because there are choices in proof rule applications (or-nodes). Only one choice of proof rule applications or invariants needs to succeed for a successful proof. See Fig. 1 for an illustration of the exploration of the proof search space for verifying a switching damped oscillator control. Each node is either an and-node (all subgoals need to be proven, illustrated by rectangular bars in Fig. 1) or an or-node (at least one subgoal needs to be proven, illustrated by circles in Fig. 1). The dark blue nodes represent the (first) successful proof of the verification conjecture. The light gray nodes have neither been proven nor disproven yet, but became irrelevant after the dark blue proof has been found. The medium color nodes have been proven (medium blue) or disproven (medium orange) during proof search, but no consecutive subset of the proven nodes constitutes a full proof of the conjecture, because other subgoals of its and-nodes have been disproven or are still unknown. Proof search thus has a concurrent nature. Considering the trend toward multi-core, multi-processor, or even distributed cluster systems, it makes sense to exploit this concurrency and actually explore the concurrent proof search structure in parallel. (Henceforth, we will use the word “processor” to denote a “cpu” or core— a hardware unit, of which there may be several in a single chip, capable of executing code. A distributed three-node computer system with two quad-core chips would have 24 processors.) As long as the total number of processors exceeds the number of subgoals in the proof search structure and

4


communication cost is low, parallelization is fairly simple: Every subgoal could be explored by one processor. When there is more work than processors (the proof space in Fig. 1 has 2118 nodes) or when locality needs to be respected to avoid unnecessary communication cost, smart parallelization is more difficult, because tradeoffs must be made on how much time to spend on the various subgoals. Even for the extreme case of a single processor system, we show that significant speedups can be obtained by exploiting the concurrent problem structure in theorem proving compared to our previous strictly sequential implementation of KeYmaera [6], a theorem prover for hybrid systems. Note, in particular, that it is not clear what the best parallel exploration order would be for the various proof nodes in Fig. 1. For each of the proof options, we want to come up with a proven/disproven decision as quickly as possible in order to find the actual proof as early as possible. It is our goal to come up with a framework in which it is easy to specify good parallel exploration orders in a natural way, such that the framework takes care of parallelization, distribution, workload sharing, proof management, propagation of proof status information, and local proof hint communication. Our contributions are as follows: – We propose a new model for parallel computation. – We show how this model fits naturally to the theorem proving problem. – We identify simple ways how fair parallel exploration strategies can be described with our framework. – We implement the framework Banyan that makes it easy to develop parallel provers. – We implement a theorem prover for hybrid systems in Banyan. – In experimental results we evaluate the performance of our parallel exploration strategies. We show that our new framework outperforms a previous theorem prover for hybrid systems and demonstrate that our new parallel proof framework generally finds proofs much faster.

2

Related Work

Parallel Processing Languages and Frameworks. The primary inspiration for Banyan comes from frameworks for high performance data-parallel computing such as MapReduce [7] and Dryad [8]. Using the “single program, multiple data” programming model, these systems hide the low-level details of parallelization from the application programmer, who need only worry about how to express a problem within the framework. Both have proven to be effective in exploiting the computing power of large clusters for data processing. It is unclear how to use them on problems such as theorem proving that have a tree structure and where the amount of time spent on any particular task is difficult to predict. Cilk [9] is an extension of the C programming language providing support for nested parallelism with provable efficiency in terms of space, time, and communication. Cilk’s primary parallelization mechanism follows a work-stealing model.


5

Cilk can emulate limited or-parallelism by aborting branches that become irrelevant. But, unlike Banyan, Cilk does ensure a fair exploration of or-branches, does not allow the programmer to specify the relative time share of parallel tasks, and does not allow communication between threads of computation. NESL [10] is a programming language with support for nested data parallelism. One of its more significant features is a method, called a vectorization transformation, for automatically unfolding a recursive computation tree into a flat structure that can easily be executed in parallel. More recently, Data-Parallel Haskell [11] has extended this line of research. However, since vectorization is only useful for and-parallelism, it is of limited use in our problem domain. Parallel Automated Reasoning. There has been much work on parallel SAT solvers. They come in two main varieties: divide and conquer solvers [12] that partition the search space and strategy-parallel solvers [13] which try many strategies in parallel. Both of these approaches are typically enhanced by sharing of information between parallel processors. This work has been successful. However, SAT solving has a rather different structure from the problems we are targeting. SAT is a decidable problem with an accepted fasted algorithm where all the gains come from tuning various parameters. We are interested in a more open-ended exploration of an undecidable problem. There has also been work on Parallel theorem provers. For example, PARTHEO [14] is a parallel prover for first-order logic. It uses work-stealing to keep all processors busy. It does not attempt to schedule branches fairly. Other provers have taken a strategy-parallel approach. RCTHEO [15] lets each processor work on the entire problem, using a different random seed at each processor, and then makes random selection at choice points. Banyan aims to give much more control over exploration.

3

And/Or Branching Structure in Proof Search

Suppose we are trying to prove a formula φ. We come to a point φ where we have two independent subgoals A and B. If both of them must return for us to prove φ, then this is an instance of and and-branching (all subgoals need to be proved). An example is if we want to prove a conjunction A∧B then we need to prove both A B A and B; see Fig. 2. Then the only question is which processor gets to do which work and when. If Tn is the ideal amount of time it takes for φ to be solved on n processors, and P is our actual Fig. 2. And number of processors, then T1 /P and T∞ are lower bounds on branching the time it takes to complete. Cilk provides support primarily for this kind of and-parallelism and provides an (expected) running time of T1 /P + O(T∞ ) [16]. Suppose, on the other hand, that when trying to prove a formula we come to a point with a formula φ where we have two methods of proceeding, A, B, either of which may or may not give us an answer to the problem of deciding φ. The

6


methods A, B could be, for example, two different proof rules that are applicable to the subgoal φ, two different invariants to try, or two different decision procedures. Then this is an instance of or-branching (see Fig. 3), and dealing with it is not so straightforward. Depending on what we know about the performance characteristics of A and B, we have several ways of proceeding. The simplest thing we might do is first to try method A and φ then to try method B. But what if method A does not terminate and we can only succeed by trying B? To deal with that case, or we might try A and B concurrently, for example by spawning a thread for each. Even if these threads are running on the same A B processor, we have gained something, for now we are guaranteed that we will spend a total amount of time no greater than twice the minimum termination time of A and B (assuming fair Fig. 3. Or branching scheduling of the two threads). If we know something about the expected running times of A and B, then we can do even better. Suppose that we know that A succeeds 80% of the time with an expected termination time of tA = 16 minutes, and suppose that B succeeds the other 20% of the time with an expected termination time of tB = 1 minute. Assume that when A (or B) does not succeed it does not terminate, which is a fairly common situation in practical decision procedures for arithmetic, for instance. If a is the proportion of our time we spend on A, + 0.20×1 then the expected number of minutes spent is 0.80×16 a 1−a . In this case it 8 1 is optimal to spend 9 of our time on rule A and 9 on rule B. This results in an expected amount of work of 16.2 minutes. The naive strategy of devoting half of our time to each rule yields an expected amount of work of 26 minutes. Consequently, we observe that it makes sense to consider time shared parallel exploration for theorem proving problems. If we have two processors we can of course do even better again. If no other subproblem is using the processors, we do best by trying A on one processor and B on the other. Then we are done in time min(tA , tB ). In automated theorem proving problems, we will typically have more tasks than processors. Then our goal should be to distribute tasks among processors and to share time on each processor in such a way that 1. We can specify the proportion of processor time being devoted to each subtask. 2. We maintain locality in the proof exploration in order to minimize communication costs and maximize coherence of the resulting proof. These are our primary goals in the design of Banyan. If they can be achieved, we argue that our framework provides a solid basis for theorem provers, because and- and or-branching are fundamental in backwards proof search. Choosing which proof rule to apply is an instance of or-branching. Proving all the premises of that rule is an instance of and-branching.


7

and or

or

and or

and or

or

and or

or

and or

or

or

Fig. 4. Nested And/Or branching structure in proof search problems

4

Automated Theorem Proving for Hybrid Systems

The specific problem that we consider is the verification problem for hybrid systems [1], i.e., systems with interacting discrete dynamics and continuous dynamics along differential equations. This is an important but challenging problem [1], with applications in air traffic control, car control, robotics, and many more domains. We consider a proof calculus for hybrid systems that is a complete axiomatization relative to differential equations [3]. Specifically, we use a backwards proof search procedure [2] for a sequent calculus for a logic for hybrid systems that is called differential dynamic logic dL [3]. Safety properties (and more general properties) of hybrid systems can be stated in dL and proved in the dL calculus. In this paper, we mostly focus on verifying safety properties of hybrid systems. The most crucial parts of the proof search procedure for hybrid systems are the search for invariants to prove properties of loops and the search for a generalization called differential invariants [17] to prove properties of differential equations without having to solve them. In a sense made precise by a relative completeness argument and a relative semidecision procedure [3, 2], this search for (differential) invariants is the “only” theoretically difficult part. In practice, another significant challenge comes from the need to handle real arithmetic for proving properties of the evolution of continuous quantities like positions and velocities in the system dynamics. Real arithmetic is decidable [4] but of high computational complexity [5]. In practice, a considerable percentage of decision procedure [4] calls for real arithmetic does not even terminate after a day or sometimes weeks. Alternatively, decision procedure calls for checking validities can be split at conjunctions and fed into separate decision procedure invocations. For proving the validity of A ∧ B, for instance, the proof search procedure can choose between deciding the validity of A ∧ B or, instead, separately deciding the validity of A and the validity of B: decide(A ∧ B)

versus

decide(A) and decide(B)

This choice causes nondeterminisms in solving real arithmetic. Because decision procedures for real arithmetic work algebraically and do not operate by logical

8


proof principles, this nondeterminism can impact the efficiency of proof search by several orders of magnitude [2]. With these nondeterminisms, the dL proof calculus fits naturally to the proposed and/or branching model for theorem proving. While the branches resulting from one rule application are and-branches (all of them have to close for the proof to succeed), the various rule alternatives are or-branches (only one of the proof search alternatives has to be successful for the proof). For instance, differential equations can either be handled using their solutions (if it is polynomial) or by searching for differential invariants. Most notably, induction rules give rise to or-branches, as there are several possible formulas (infinitely many) that could be used as (differential) invariants, but one successful invariant is enough for closing the proof. For techniques that generate promising candidates for (differential) invariants, we refer to previous work [18]. Some unsuccessful candidates for invariants can be refuted quickly, but others run for days without a result. Hence, practical and scalable hybrid systems verification requires a fair concurrent exploration of all proof options to be successful in advanced applications.

5

Programming Model for Distributed Proving

Our programming model is designed to expose maximally the concurrent nature of the proof search problem. The main unit of control is the computation node. One such node would correspond to, for instance, each and-node and or-node in Fig. 4. Each node (except the root) has a parent and zero or more children, forming a computation tree replacing the traditional control-flow stack. 5.1

Background

Consider the function in Fig. 5 (written in the C programming bool f (formula fm ) { bool r1, r2; language) which attempts to r1 = g(fm); prove a formula fm by using r2 = h(fm); two methods, represented by if( r1 || r2 ) return true; the functions g and h, where h else return false; is a recursive function that calls } itself. A sequential execution of the code might produce a Fig. 5. A function that could be parallelized control-flow graph as in Fig. 6(a). Such an execution would not take advantage of the fact that the calls to g and h are independent. The same function with a few annotations could be parallelized in parallelization frameworks for C, e.g., Cilk. Cilk would produce a control flow graph like the one in Fig. 6(b). The function calls are executed in parallel with the rest of the function. Results are collected at a sync point, at which the calling function blocks until


(a) Sequential execution

9

(b) Parallel execution in Cilk

Fig. 6. Execution models

its spawned function calls return. The primary design goal of Cilk is efficiency in the case when all of these subtasks need to be performed, i.e. the case of and-branching. The red circle signifies what is called an inlet in Cilk terminology. It is a piece of code that is called whenever a function executing in parallel returns. The inlet can cause the other active function calls to abort. For example, if g(fm) returns true, it would be possible to write an inlet that aborts the call to h. Thus, Cilk can emulate or-branching. However, Cilk is not designed for doing or-branching fairly. It uses a work-stealing system to map tasks to processors but does not share multiple tasks on a single processor. If there are more tasks than processors (e.g., 2118 in Fig. 1) and some of them do not terminate then others may starve, which would render the proof search problem incomplete due to fairness issues. Our model makes two major improvements, reflected in Fig. 7. The boxes, which represented stack frames in the original Fig. 6, now represent computation nodes. The first improvement is that the calls to parallel functions are annotated with a number of tickets to pass to the call. These tickets are meant to indicate the proportion of total processor time that should be spent on each branch. The application programmer can assume the existence of a scheduler that will give each node an amount of time approximately proportional to its share of tickets. Delivering on this promise is the Fig. 7. Parallel execution difficult part of making our programming model useful, in Banyan as we detail in the next section. Second, there are message-passing channels, represented by dotted lines, between a parent and its children, allowing local sharing of information. We work under the assumption that nodes can be moved around among different processors

10


Each node has a work function which is where its main functionality will be specified. If the node is a leaf, this function might call a computationally expensive decision procedure. An internal node would compute possible rule applications, or generate (differential) invariants. It might construct new nodes and register them as its children and give them tickets. A node must have childReturned function to deal with a child’s result. E.g. an or-node deals with a child returning with a proof by immediately returning with a proof. Finally, a node must also have handleMessage function, which specifies what to do when a message arrives. A typical implementation might selectively pass a modified version of the message to some of the node’s children.

5.2

Example Proof Strategies

In this model we can easily simulate common sequential search strategies, which we generalize to parallel processing. To perform a depth-first exploration, a node can give all its tickets to its leftmost active child. To perform breadth-first search, a node can evenly split tickets among its active children. We can also use more sophisticated strategies such as what might be termed staged search, in which we explore breadth first for nodes that have more than a certain number of tickets, and we explore depth-first otherwise. This strategy might be desirable to prevent a search from spreading itself out too thin. The message-passing feature allows, for example, sharing of information about successful invariants. It is often the case that a hybrid system will have a differential equations for continuous evolutions inside of a loop. Then, during proof search we will be trying many possible loop invariants, and for each we will be trying many differential invariants. Differential invariants that succeed for one loop invariant might be more likely to succeed for another or may even be guaranteed to work already. By communication through their common ancestor, these branches can explore more promising options first and potentially save time. In practice we have found that having fairly scheduled or-parallelism is very useful. In our logic for hybrid systems, there are two proof rules that work by solving differential equations. One gives a difficult subgoal. The other gives an easier subgoal but is incomplete. It is difficult to know a priori whether the easier one will work. We have found that trying them both in parallel is very effective and can save hours of work.

6

Implementation: Banyan

Banyan is implemented in Scala [19], an object-oriented language built on the Java Virtual Machine, providing extensive support for functional programming. The main logical components of runtime instance of Banyan are a coordinator, a client, and one or more workers.


6.1

11

Worker implementation

Most of the core functionality of Banyan is provided by the workers. During execution there will be one worker for each processor being utilized. The proof nodes will be distributed among the workers. A worker has two threads: a listener and a scheduler. The listener’s job is to wait for messages from other workers, the coordinator, or the client. The scheduler’s job is to loop through the task queue and to cause work to get done by calling the work function on each node that it owns. Scheduling algorithm The scheduler performs weighted round-robin cooperative scheduling on the nodes that it owns. That is, in each round each node gets to work for a timeslice proportional to the number of tickets that it holds. After that time, the scheduler notifies the node and waits for it to yield control. If new nodes were created during the timeslice, they are added at the end of the task queue. If a node needs more than one timeslice to finish some piece of indivisible work, it is allowed to save up timeslices for future use. If a node receives a timeout and has to kill a computation, it might save up twice as many timeslices for the next try. If that still fails, it might save twice as many again. Using this strategy, a node will never work more than three times its minimum required time. Banyan does not provide support for nodes that wish to pause threads or processes between calls of its work function. Allowing a paused thread to persist between timeslices would mean that no longer could any node be shipped off at any time between timeslices, because paused threads are not serializable. This could get in the way of the load balancing. It would be interesting to investigate how allowing nodes to be non-mobile would affect the performance of Banyan. Node transfer strategy Each worker keeps track of the total number of tickets held by nodes that it owns and the target number of tickets that the coordinator has indicated that this worker should hold. The total minus the target is called the ticket surplus. Each worker strives to keeps its surplus close to zero. Periodically (once every two seconds for our experiments), the scheduler pauses between timeslices to assess the ticket situation at this worker. If the surplus is negative, the scheduler sends an update to the coordinator indicating its present total and target. If the surplus is positive, the scheduler is responsible for finding a suitable subtree to try to ship to another worker. To do this, it first computes the total tickets held by and total time spent on each local subtree. It then selects a subtree with number of tickets approximately equal to the surplus and with time-spent above a threshold. (Here we assume that nodes which have required a lot of work in the past are less likely to terminate quickly in the future.) It then communicates with the coordinatoor to find a good worker to send the subtree to. If such a worker exists, the subtree is shipped there. We are still investigating possible metrics for assessing the suitability of a subtree (or possibly set of subtrees) to be shipped to other workers. The present

12


version of Banyan uses a slightly modified version of a “lowest absolute value” metric described. We are also examining what to do when the tickets are not evenly enough distributed, in which case fairly distributing the nodes is impossible. 6.2

Memoization

Banyan provides another feature. The application programmer can specify a key for each node. Before a node is created, Banyan will check the global shared database to see if a node with the same key has already been created. If so, the existing node is used in place of a new one. In this way, the computation tree becomes a DAG. This option is not yet fully operational in our implementation, but there are no fundamental barriers to having it so. We imagine that this feature would be very useful for theorem proving in logics where identical subgoals are often created. 6.3

DLBanyan

On top of the Banyan framework we have built DLBanyan, a theorem prover for hybrid systems. To illustrate a typical Banyan program, here is a slightly simplified code sample from our implementation of and-nodes: def childReturned(child: Int, v: ReturnType) : Unit = v match { case Proved(rl) => numOpenChildren -= 1 if(numOpenChildren returnNode(GaveUp()) } This says that if the node learns that one of its children has returned with a proof, then it will return if all the rest of its children have already returned with a proof. If one of its children returns indicating that it cannot find a proof, then the node returns indicating that it also cannot find a proof.

7

Experimental Results

We present experimental results in Table 1. All of the DLBanyan proofs were run on a Mac Pro with two 2.66 GHz quad core Intel Xeon processors and 16 GB of RAM. We show a visualization of the car control proof search problem in Fig. 8. Nodes are drawn with an area proportional to the amount of time spent on them. Blue nodes returned with a proof, orange nodes returned disproved, and gray nodes were aborted after becoming irrelevant to the main proof search. The darker blue nodes are part of the successful complete proof.


13

Fig. 8. Proof search for car control (full invariant search)

In the car control problem with full invariant search (Fig. 8) the root node has 16 children, corresponding to 16 choices for an invariant in the first step of the proof. DLBanyan is able to parallelize the work on these branches with near linear speedup for up to at least six workers. We also performed the proof search with a good loop invariant preselected. In that case, the search only has two large chunks of work which could be performed in parallel. As Table 1 shows, DLBanyan does indeed exploit that parallelism. Observe that DLBanyan can prove the switching damped oscillator example [2], but KeYmaera cannot prove it automatically. (The 148s cited in Table 1 requires a user interaction.) The reason is an inherent limitation in the sequential implementation of the proof search strategies in KeYmaera: KeYmaera follows a sequential proof strategy and uses timeouts for deciding when to move on to a different choice for the current nondeterminism. With its sequential proof search implementation, KeYmaera does not manage the real concurrent proof structure and cannot revisit earlier proof options in different subgraphs of the proof search that had timed out earlier. In KeYmaera we found it extremely hard to

14


Table 1. Experimental results: time in seconds of verification in KeYmaera compared to Banyan with 1/2/3/4/5/6 workers. The (*) signifies a non-automatic proof. Hybrid systems KeYmaera Banyan workers verification problem Prover 1 2 3 4 5 6 simplified car control 415 3 car control 6110 26 17 19 car control (full invariant search) 137 67 42 44 36 24 switching damped oscillator 148∗ 2

make proof search more complete and automatic without explicitly addressing the parallel structure in proof search as Banyan allows us to do. This is one of the reasons why we are convinced that Banyan is a successful framework for developing automatic proof search procedures. DLBanyan provides less functionality than KeYmaera, but it follows a much better proof exploration strategy. We must admit that some of the difference in running time between KeYmaera and DLBanyan is due to more mundane reasons. In the course of developing DLBanyan, we discovered several improvements to our proof search methods that seem to significantly lower running time. Two things in particular seem to help significantly: performing substitution to eliminate auxiliary variables, and universally closing a formula before sending it to the real arithemtic decision procedure. However, these are steps that are not always clearly beneficial, and one of the reasons we are developing banyan is so that we can specify strategies that try different choices for such options in parallel.

8

Conclusions and Future Work

We have presented a new programming model that fits naturally to the parallel proof exploration in (backwards style) automated theorem proving. Our framework makes it easy to specify advanced fair parallel proof search strategies. Our approach directly adopts and/or proof structures in theorem proving and uses a time sharing primitive we call tickets to control fair exploration dynamically. Our implementation of this framework, Banyan, takes care of the distribution and redistribution of workload to the processors following the ticket guidelines and provides local communication among proof nodes for dynamic re-prioritization and sharing of partial successes and proof hints. In the Banyan framework, we have implemented a theorem prover for hybrid systems called DLBanyan. Experimental results show that our concurrent representation of the proof search problem enables us to find proofs automatically much faster than in a previous sequential implementation. The fact that Banyan directly allows the exploration of concurrent proof options even enables us to prove two cases automatically that previous provers could not. At the same time, DLBanyan still falls short of providing means for interactive user guidance, which we have found to be quite useful in proving more advanced verification problems interactively in KeYmaera. A combination of the


15

parallel proof search framework Banyan with the capabilities of KeYmaera is promising future work. Banyan is available for download at www.cs.cmu.edu/∼renshaw/banyan.

References 1. Henzinger, T.A.: The theory of hybrid automata. In: LICS, IEEE CS (1996) 2. Platzer, A.: Logical Analysis of Hybrid Systems: Proving Theorems for Complex Dynamics. Springer, Heidelberg (2010) 3. Platzer, A.: Differential dynamic logic for hybrid systems. J Autom Reas 41(2) (2008) 143–189 4. Collins, G.E., Hong, H.: Partial cylindrical algebraic decomposition for quantifier elimination. J. Symb. Comput. 12(3) (1991) 299–328 5. Davenport, J.H., Heintz, J.: Real quantifier elimination is doubly exponential. J. Symb. Comput. 5(1/2) (1988) 29–35 6. Platzer, A., Quesel, J.D.: KeYmaera: A hybrid theorem prover for hybrid systems. In Armando, A., Baumgartner, P., Dowek, G., eds.: IJCAR. Volume 5195 of LNCS., Springer (2008) 171–178 7. Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. In: OSDI’04: Proceedings of the 6th conference on Symposium on Operating Systems Design & Implementation, Berkeley, CA, USA, USENIX Association (2004) 8. Isard, M., Budiu, M., Yu, Y., Birrell, A., Fetterly, D.: Dryad: distributed dataparallel programs from sequential building blocks. In: EuroSys, ACM (2007) 59–72 9. Randall, K.H.: Cilk: Efficient Multithreaded Computing. PhD thesis, Department of Electrical Engineering and Computer Science, MIT (1998) 10. Blelloch, G.E., Chatterjee, S., Hardwick, J.C., Sipelstein, J., Zagha, M.: Implementation of a portable nested data-parallel language. Journal of Parallel and Distributed Computing 21(1) (1994) 4–14 11. Jones, S.P., Leshchinskiy, R., Keller, G., Chakravarty, M.M.T.: Harnessing the multicores: Nested data parallelism in haskell. In Hariharan, R., Mukund, M., Vinay, V., eds.: FSTTCS. Volume 2 of LIPIcs., Dagstuhl-Leibniz-Zentrum (2008) 383–414 12. B¨ ohm, M., Speckenmeyer, E.: A fast parallel sat-solver - efficient workload balancing. In: Annals of Mathematics and Artificial Intelligence. (1996) 40 13. Hamadi, Y., Sais, L.: Manysat: a parallel sat solver. Journal on Satisfiability, Boolean Modeling and Computation (JSAT (2009) 14. Schumann, J., Letz, R.: Partheo: A high performance parallel theorem prover. In: CADE, Springer (1990) 40–56 15. Ertel, W.: Or-parallel theorem proving with random competition. In: LPAR ’92: Proceedings of the International Conference on Logic Programming and Automated Reasoning, London, UK, Springer-Verlag (1992) 226–237 16. Frigo, M., Leiserson, C.E., Randall, K.H.: The implementation of the Cilk-5 multithreaded language. In: Proceedings of the ACM SIGPLAN ’98 Conference on Programming Language Design and Implementation. (1998) 212–223 17. Platzer, A.: Differential-algebraic dynamic logic for differential-algebraic programs. J. Log. Comput. 20(1) (2010) 309–352 18. Platzer, A., Clarke, E.M.: Computing differential invariants of hybrid systems as fixedpoints. Form. Methods Syst. Des. 35(1) (2009) 98–120 19. Odersky, M.: Scala. http://www.scala-lang.org (2010)