After the Turing Machine Contents

After the Turing Machine Toni Kazic  [email protected] Institute for Biomedical Computing Washington University St. Louis, MO 63110 February 18, 1999

Abstract

Problems in implementing DNA and other types of molecular computers stem from the inherent physical nature of molecules and their reactions. The current theory of computation makes assumptions that at best are very crude approximations of the physical chemistry. A theory which took into account the physical chemistry would likely be very dierent from what we have now and should help in designing more optimal molecular computing systems. In this paper I describe brie y the discordance between these assumptions and the physical chemistry; indicate some of the properties a more physically realistic model might have; and sketch some of the possibilities for a computing system which exploited the physical chemistry. We are developing a model for networks of biochemical reactions and molecules incorporating treatments of both continuous and discrete aspects of biochemical systems. The formal system of the model provides an example of what a CPC might be if the problems of encoding and design were solved. Moreover the model may be useful in resolving design issues once it is suciently populated with data.

Contents 1 2 3 4 5 6

Introduction The Present Theories Toward a More Comprehensive Theory of Computation Issues Facing a Uni ed Theory Strategies for New Machines Epilogue Bibliography Signi cant Changes in Revision Two A List of Symbols and Abbreviations

2 4 11 19 23 30 31 34 35

 This is a revised and expanded version of a paper from the proceedings of the DNA Computing 4 meeting, Philadelphia, June, 1998 [29], and an IBC technical report [30]. A precis of an earlier version of some of the rst four sections is [24]. This document is partly working notes and so continues to evolve; this is its second major revision. The most important dierences in notation or de nitions between reference [24] and this are marked in the text as superscripted lower case Roman numerals and their explanatory notes collected at the end.

1

1 Introduction Molecular computing is justi ably exciting, not least for the alluring prospect of biologically-inspired machines nicely handling NP-complete problems1. However, current molecular computers are limited in three ways. First, it is technically hard to compute by biochemical means, and matters worsen with problem size, reagent inhomogeneities, and the number of manipulations. Second, each computer is a special-purpose machine tuned for particular problems, data structures, algorithms, and problem sizes, not a general-purpose device. Momentarily setting aside concerns for the validity of proofs that particular molecular systems are universal Turing machines, the strong fusion of computer, data structure, and algorithm achieve compactness in the expression of those notions at the expense of the exibility of general machines. Third, algorithms for combinatoric and number-theoretic problems only have been attempted so far. A count of the NP complete problems in Garey and Johnson [16] reveals several problems drawn from other areas of mathematics, and of course there are many important numerical problems. But molecular architectures have been conceived almost exclusively as discrete, digital designs, and this combined with their special-purpose nature has militated against attempts at continuous problems. Despite the interesting results achieved so far [1, 21, 37], these three observations indicate the current approaches may not lie on a path that would eventually produce machines as exible and as capable as our current electronic devices. Are these limitations simply technical challenges that can be overcome by clever engineering, or do they represent more fundamental problems? The answer depends on how well the theory of computation can predict the outcome of molecular algorithms and the behavior of molecular computers. Here experience is less than encouraging. The outcome of iterating a molecular algorithm ten times, let alone 10,000, is not predicted by the theory, and the properties of iteration vary depending on whether the computer is electronic or biochemical. The computed output is a mixed population of \answer" and \others", not the single unambiguous result the theory presumes. Since it assumes the execution of each step is error-free2 , the theory cannot help us estimate how good our engineering of the biochemical machine must be or the theoretical consequences of error rates of 1 { 50%, reasonable values for many reactions in vitro ; and error analyses are ad hoc . A theory failing such empirical tests would be judged inadequate by a natural scientist. Empiricism properly has little weight in pure mathematics, which of course is the origin and methodology of our present theory of computation. And it is fairly objected the theory was not intended to predict how any physically rei ed computer would behave, only to elucidate the nature of and requirements for computation. After all, in the very earliest days of electronic computing, the components poorly imitated the premises and results of the theory in their error rates and functions. The extent to which contemporary electronic machines now emulate the theory is remarkable, but perhaps this emulation is incidental to the theory's predictive success3 . The claim would be that as long as the abstract properties of the molecular computer match those of the theory, the theory is adequate. If correct, then changing the computing machinery should have no eect on the 1 I use the term \molecular computing" in preference to \DNA computing" to not prejudge which molecules and reactions might be most suitable. In Section 4 I give some reasons why DNA may not be the optimal biochemistry. 2 Yields of less than 100% for a single step by de nition means that other molecular products result from that step. The net yield for a system of reactions R; ri 2 R (or equally, laboratory manipulations as simple as transferring material from one test tube to another) is simply

YR =

YN y(ri);

i=1

(1)

so that the eects of imcomplete yields are propagated through the system. Even for individual, very simple reactions, yields of y(ri ) = 0:8 to 0:9 are considered extremely good, and are often far worse. The present theory of computing of course assumes YR = 1. Indeed, much of the complexity of the biochemical systems of organisms can be viewed as engineering to consistently boost yields. 3 While molecular computers will surely improve, I believe it is extremely optimistic to expect them to improve the many orders of magnitude (103 { 1010 ) required by the current theory of computing, at least until our understanding of molecular engineering sharply improves. Certainly such remarkable technological improvements cannot occur without careful attention to the physical properties intrinsic to molecular systems.

2

theory's ability to predict the outcome of computations implemented on the new machine. Yet this result can be obtained only by assumptions so severe as to contradict the premises of the theory | for example designating just one of the many molecular species which persist at the \end" of a computation as the output. Since the theory of computing is so sensitive to the physical rei cation of the computation in any but the most limited instances, the logical conclusion is that the abstract properties of the molecular system do not match those of the theory of computation. By the standards of mathematics, the present theory of computing is insucient. This result is hardly surprising to those with any acquaintance with physical biochemistry, which accurately and reliably predicts the behavior of molecular systems at the scales and structural complexity found in molecular computers. I suggest these and other diculties re ect deeper problems in reconciling the physical nature of systems of biochemical reactions with the existing theory of computability than have been addressed to date. The fundamental premises of the theories of computation and physical chemistry are so dierent as to be discordant. It is as if the theory of computation was a very severely approximated view of the physical chemistry, at the cost of many deep errors, loss of generality, and weakening of predictive power. Though many approximations are possible, some of which rather resemble the approximations the theory of computation would seek to impose, a theory which included all of them simultaneously would be inadequate when tested against reality; and the grounds for inclusion of so many approximations could well prove quite dicult to derive. Most simply stated, the theory of computing describes digital machines; but systems of reactions are inherently analogue machines. By its very formulation the theory of computing cannot ful ll the requirements of the natural scientist, the designer of a molecular computer, or the theoretician. Indeed, a theory which failed the empirical tests indicated above would be judged by a natural scientist to be inadequate, and that person's response would be to get a dierent theory. One must therefore consider the hypothesis that the present theory is simply a special case of a more comprehensive theory of computing, one which would unify aspects of the theories of computation and physical biochemistry to account equally well for the properties of algorithms when implemented on abstract, electronic, and biochemical computers. This paper considers some of the elements of a uni ed theory and what issues it would need to address to represent an improvement over the present situation. It is not a precis of that theory or its applications: at best it is only a crude sketch of the territory. Are new theories of computation worth the eort of their creation? There are four reasons that justify a serious eort in this direction. First, it is important to determine the conditions under which the present theory of computation holds. If the premises or methods of present proofs are restricted, then their results, including the properties of computations, will be less general than previously thought. Second, a more physically grounded theory would likely stimulate substantial changes in our understanding of computing and the design of computers. After all, this is one of the original, compelling motivations for examining the idea of DNA computers. Third, our present notion of eective computability may be unnecessarily limited. Smith shows that the mathematical descriptions of the behavior of physically extant objects, such as the Hamiltonian dynamics of many-body systems, are far more complicated than NP-complete problems [50]. Demonstrating these descriptions subsumed the class of general recursive functions [32] and performed operations which were isomorphic to them after a reductive transformation would argue that the de nition of computability must be expanded, and supra-Turing computations would become possible. Alternatively these descriptions may not be computations under any useful de nition of the term: there may well be many natural phenomena which do not compute anything. Finally, a theory that helped design better algorithms and molecular computers would distinctly bene t the current situation. Thus the time seems ripe to re-evaluate the theoretical and pragmatic sides of molecular computing and see whether the physical chemistry can oer new insights that will produce both better theory and better molecular computers. Therefore this paper considers four questions. 1. How are the theories of computation and physical biochemistry discordant? 2. What are the fundamental principles of a more comprehensive theory? 3

3. What issues would a more comprehensive theory of computation encounter, and how might these be addressed? 4. What might a molecular computer, constructed along the lines of a more comprehensive theory, be like?

2 The Present Theories To understand why the present theory of computation is insucient for biochemically based systems, I brie y compare the fundamental axioms and basic results of the theories of physical biochemistry and computation. There are a number of excellent references which describe the physics of systems of molecules and reactions, and any good text can serve as a starting point4 [3, 5, 20]. Similarly, references [11,32,41,45,46,52] cover the foundations of modern computational theory.

2.1 The Theory of Physical Biochemistry

Molecular computing relies on the properties of molecules and their reactions to specify, execute, and decode computations. Though present models of molecular computation focus chie y on some idealizations of the molecular species, consideration of the reactions which occur among them is required if a more accurate and predictive theory is to be developed. Our present description of the mechanisms and properties of chemical reactions is a set of ideas and laws which are fundamental to much of modern physics, chemistry, and biology. To the extent that any description of the phenomenological world is merely a model of our understanding of it, then it is certainly true that these laws and their underlying assumptions are models of molecules and their collective behaviors, thus reminding one of philosophical concerns about the relationship of model to phenomena. However the current description works extremely well for reactions at the level of detail relevant to the problem of molecular computing, and it is hard to imagine it changing signi cantly in its most fundamental aspects without prompting a profound revolution in science. We may be as assured as it is possible to be that the theory is correct. I rst summarize informally the major intrinsic physical properties of reactions and molecules. All chemical reactions, including those involving molecules of biological derivation, arise from populations of dierent types of molecules interacting in physical space and time. These populations are large: at standard temperature and pressure a microliter of water contains  3  1019 molecules; if it was also one femtoMolar in an oligonucleotide the total population would rise to  3  1028 molecules (neglecting the oligonucleotide's counterions). The thermodynamic extensional variables which describe the system | for example, concentration5 , free energy, enthalpy, entropy, and other state variables | are averages both over the entire population of molecules and the in nite number of possible quantum states each molecule can assume, and their values range over the real or positive real numbers. The kinetic properties of a system | variables such as reaction rate and ux | are determined by the statistical mechanics of the molecular populations. Some states of molecules are discretely approximated, such as covalent structure, but the quantum mechanical description of those structures is a set of continuous functions; and each molecular specie has characteristic probabilities for dierent types of structural changes, including covalent modi cations, under different environmental conditions. Even though molecules are discrete, the number of molecules is a nite subset of the natural numbers, and many properties of molecular species are discrete (e. g. their quantum states), the variables and equations which describe how a molecular computer and It may be useful to remark that the theory of physical biochemistry incorporates and extends that of physical chemistry, so that the two are indistinguishable except by the size and complexity of the molecular systems considered, and less often by particular questions posed to them (though the methods of choice can dier strikingly). Thus books with \Physical Chemistry" in their titles are as important for our discussion as those with \Physical Biochemistry" (as are those titled \Biophysical Chemistry"); and words whose root is \biochemistry" can be substituted for \chemistry", and vice versa , in nearly all contexts. 5 Strictly, this is the chemical activity of each specie if the mixture is nonideal in a thermodynamic sense. However for most reactions the activities are reasonably well approximated by the concentrations, and I will consistently use concentration in what follows at a slight cost in generality. 4

4

its components behave are as continuous as those for any other system of reactions, independent of any idealization of a molecular specie or reaction6 . In practice, nearly all the laws describing chemical systems are continuous, either because they are inherently so by virtue of their formulation in terms of continuous variables; or because the ranges of the variables of interest are the real numbers; or because the most useful descriptions are continuous formulations over very large ensembles. Equation 1 illustrates all three sources of continuity. This does not mean that approximations are impossible (see Section 4), but it does mean they are exactly that and their quality must be evaluated. I now present a brief rationale for the main points in the preceding summary, either by indicating a derivation or summarizing the physical chemistry. Three axioms are assumed: that matter and energy are conserved; that space and time are continuous variables; and that mass is a discrete variable. The rst is simply the fundamental assumption of thermodynamics. The second accords with the way these observables are treated in the physical theory at these size scales, and with our accumulated experience that space and time can assume any real value7 . The third axiom expresses a fundamental observation of the last few hundred years.

2.1.1 An Example

Consider the initial ligation of two double-stranded oligonucleotides o1 and o2 , the single-stranded region of each the reverse complement of the other, by E. coli DNA ligase. This may be written as two formal reaction equations [35], o1 + o2 o1
o2 (2) + * o1 o2 + 2 AMP + 2 PPi + 2 H : o1
o2 + 2 ATP ???? (3) )ligase ???? Here ATP, AMP, and PPi are deoxyadenosine 50 -triphosphate, deoxyadenosine 50 -monophosphate, and inorganic pyrophosphate (here the i simply designates \inorganic", and is not an index). These equations abstract many simultaneous events occuring in very large populations of molecules, focusing on the relative numbers, and types of molecules, involved in the ligation of two oligonucleotides. The indicate that the reaction can proceed in either the forward (left to right) or reverse (right to left) directions, depending on the current status of the system. To characterize the behavior of reactions chemists measure reactions' kinetics | how fast the reaction goes in both directions | and thermodynamics | which direction is eventually favored by how much. The system of kinetic and thermodynamic equations corresponding to the formal reaction equations isi [o_1 ] = k?2 [o1
o2 ] ? k2 [o1 ][o2 ] (4) [o_2 ] = [o_1 ] (5) 2 2 + 2 2 _ [o1
o2 ] = k2 [o1 ][o2 ] + k?3 [o1 o2 ][AMP] [PPi ] [H ] ? [o1
o2 ](k?2 + k3 [ATP] ) (6) _ = 2k?3 [o1 o2 ][AMP]2 [PPi ]2 [H+ ]2 ? 2k3 [o1
o2 ][ATP]2 [ATP] (7) 6 The only situation in which a discrete treatment of reaction rates has been found helpful is when the total number of molecules is extremely low (0.15 { 100/system (reference 47 and Harley McAdams, personal communication). The very high concentrations of molecules used in essentially all reactions, including those of DNA computing (106 molecules/l to 1020 molecules/l per cell or test tube), keeps us far from that lower bound. For example Adelman's initial ligation was about 6  1014 molecules of oligonucleotide, at a concentration of  10M [1]. 7 Attempts to unify quantum mechanics and gravity suggest the continuity of spacetime breaks down in the range of the Planck units | L  10?33 cm T   10?44 sec However, molecular reactions operate at length and time scales of approximately 10?10 cm and 10?15 sec respectively, 23 and 29 orders of magnitude greater than the Planck units [42]. So irrespective of the outcome of the grand uni cation theories, for biochemical purposes we are squarely planted on continuous spacetime. My thanks to Will Gillett for asking the question, Bill Wise for pointing out the answer, and Wai-Mo Suen for making sure I got it right.

5

[o1_o2 ] = k3 [o1
o2 ][ATP]2 ? k?3 [o1 o2 ][AMP]2 [PPi ]2 [H+ ]2 _ = 2k3 [o1
o2 ][ATP]2 ? 2k?3 [o1 o2 ][AMP]2 [PPi ]2 [H+ ]2 [AMP] _ [PP_ i ] = [AMP] _ [H_+ ] = [AMP]

(8) (9) (10) (11)

dG2!3 = dG2 + dG3 2 [PPi ]2 [H+ ]2 =
G020 +
G030 + RT ln [o1 o2 ][AMP] [o ][o ][ATP]2 ;

(12)

1

2

(13)

where o1
o2 is their appropriately collided and suciently annealed complex; o1 o2 the two oligonucleotides ligated together; ATP, AMP, PPi , and H+ , those molecular species; x and [x] the concentration of specie x; x_ and [x]_ the time derivatives of that concentration, and ki the specieindependent rate constants for reaction ri , where i < 0 for the reverse and i > 0 for the forward directions. In equations 12 and 13, G is the free energy of the reaction (a global measure of the extent to which one side or the other is eventually favored energetically); dGi the dierential free energy for reaction ri under conditions not far from equilibrium (the standard assumption of thermodynamics); dG2!3 is the dierential of the overall free energy for reactions 2 and 3;
G0i 0 the free energy for reaction ri under the thermodynamic standard conditions (a scalar); R the gas constant; and T the temperature of the reaction (both scalars). The ratio of the concentrations of dextralateral to sinistralateral reactants8 at any time t is also called the mass action ratio, Qt . In these equations I have assumed there is sucient ligase present to saturate the system, so that the rates depend only on the other species. This is not always a valid assumption; in such cases the catalyst appears in the rate equations on the same footing as the other reactants.

2.1.2 Every reaction is continuous.

To illustrate the continuous nature of reactions I sketch the derivation of the general versions of the equations in the example. Let the set R be a system of reactions ri ; 1  i  N among a set of molecular species X = fxj g; 1  j  M (called reactants); i, j , N , and M are any positive integer. Each reactant xj may participate in more than one reaction, and every reaction has at least two reactants. Each reaction ri is described by a formal reaction equation of the form Xd;i ki X ??? * ni;d;j xj ni;s;j xj ) ??? k ? i j =1 j =1

Xs;i X

(14)

where the sets of reactants written sinistralaterally (s) and dextralaterally (d) in the formal reaction equation are Xs;i and Xd;i , respectively; the cardinality of any set S is denoted by S, so that Xs;i and Xd;i are the number of reactants in each set; ni;fsjdg;j is the stoichiometry of reactant xj in Xs;i or Xd;i ( j is logical or); and ki and k?i are the forward and reverse specie-independent reaction rate constants, respectively. A specie appearing catalytically in a reaction equation is included on both sides. R is then described by the set of formal reaction equations, one for each ri 2 R. Call the set of reactions in which any particular reactant participates Rj ; Rj  R. If the formal reaction equations generated by equation 14 describe chemically elementary reactions and certain chemical conventions are applied, then one can describe the kinetics of R in terms of the changes in concentration (xj ) for each specie. These changes are simply the derivative of concentration with respect to time (x_ j ) for each molecular specie in R. The system of ordinary 8 The set of reactants occuring on the right hand side and left hand side of any arbitrarily written reaction equation, respectively. The appellations \substrate" and \product" depend on the instantaneous equilibrium, and are not xed roles for a reactant.

6

dierential equations describing the kinetics of R is generated by9

x_ j =

N X i=?N i6=0

(ri ; xj ) nj;i k i

M Y l=1

xnl l;? i

(15)

where the additional variables are

8