Probe. System. Trace. The Infodynamics of Developmental Systems VERSION 1.0 MAY, Y2K © Satindranath Mishtu Banerjee Scientificals Consulting, 309-7297 Moffatt Road Email:
[email protected]
Development is characterized by increasing specification. Vague patterns early in development become distinct categories later in development. To make a measure on a developing system assumes we can first identify the category of what we are about to measure -- and that the category even exists at the point of observation. That is, in the course of development, measurable variables emerge, and the ability to measure emerges. Beginning with considerations of what can be observed in a developing system, I develop a conceptual framework for developmental systems, by proposing an extension of fuzzy set theory to deal with vague sets. Vague sets are represented as curvature in a sets-as-points space, and represent sets whose definitions are incomplete, and become increasingly specified over time. In the course of this program I use data operations on relations to expand beyond the limitations of functions, which are the basis of dynamical systems. The move from a functional to relational calculus, corresponds with the generalization from dynamical to developmental systems, dynamical systems emerging as a special case of temporal and spatial scale invariance in the generating process. KEYWORDS: Developmental systems, functions, relations, fuzzy sets, vague sets
Developmental Systems
Prelude NothingExistsWithoutContext
Imagine a world that looks like ours. With people who seem much as we do. Imagine on this world a society that strives towards perfect mechanism. A society that minimizes all friction, built to last forever, or as long as it can. Imagine a locked room with a single occupant, a simple unspectacular looking individual. He deviates from his function, has become asynchronous with his peers, frictive. The locked room is designed to rehabilitate him from this malaise. On the wall, there is a single ornament, a clock that ticks precisely and regularly. He breathes with the same regularity. When he first entered this room he was given a list of tasks. One task for each duration of ticks on the clock. When he performs these tasks perfectly, he will be allowed to leave, he will be free. From without he is monitored, and when he deviates from the tasks -- either in timing or in degree -- he is given a small shock he can not avoid since both the floor and walls are metal. The lights from the ceiling dim precisely at a particular moment, and brighten precisely at another particular moment. If he does not close his eyes to coincide with the dimming light he will be shocked. If he does not open his eyes to coincide with the return of the light, he will be shocked. The last person he saw before he entered this room, the person who gave him the list of tasks, said only, "You will become a function again, as you should be." One task is particularly impossible. At the middle of the day or period of light, a tray of food is slid in, and also a second tray, with 100 photos of embryos of 50 different species of plants. His task is to first eat his food, and then to sort the embryos into 50 different pairs of photos, each pair from the same species. Every cycle he fails at this task and is shocked through the floor. His former function was as a scientist, specifically a botanical systematist, and the embryos are from a group of species he is the authority on. Still, he fails at the task. His original ordering of species was based on adult specimens, and the features used to distinguish the species, to limb the tree of life at it's branch nodes, have yet to appear in the photos, or exist as stubby mounds of tissue he can not distinguish. This is the only time of the day he is shocked. He can complete every other task successfully, but not this one. Each cycle he fails again, fails to be a function, and is shocked for his lack of synchronicity. 2
Developmental Systems Introduction "And so, I repeat, what is in this book is about biology. It directly addresses the basic question of biology: 'What is Life?' And nothing can be more biological than that." -- Robert Rosen, Life Itself
My task is to explain to you why this "occupant" must fail in his task. That his failure is our failure too, and tells us something fundamental about the nature of our world, and of our scientific experience of that world. My argument, in essence, following Robert Rosen's in his book "Life Itself" (1991), is that biological systems are irreducibly relational and non-mechanical, and to handle them, we will have to get past the mathematical apparatus that has appeared to serve us so well in the physical sciences, the analysis of functions, and its complement dynamical systems theory. Rosen made his arguments deductively, developing general principles, using category theory as his probe to deconstruct the fundamental nature of a mathematical function (a mapping), formalizing the notion of mechanism, and developing a formal model of organization in terms of the biological notion of function (the causal relationship of a component with respect to the whole, its "purpose" or telos), and finally developing a relational model which he asserted was coincident with the state "living" for any system that could be said to correspond to the causal relationships, or entailments that the model illustrated. My arguments will follow a very different path, proceeding inductively out of the nature of observations and data, and ultimately depending upon the data, and data-driven insights of several empirically based research groups that could be loosely said to focus on the relation of information and dynamics in biological systems, or "infodynamics". I proceed inductively from cases, insights from individual data sets, and the nature and limits of data analysis itself, to apprehend some general principles, and some tools for thought. We will not get to "living systems", but to "developmental systems", a generalization of dynamical systems, and hopefully a model suitably robust to provide the substrate for, and include as a sub-class, "the living". In short, we will approach the problem of biology from the other side of the looking glass, through the nature of the data the world provides us. This is an exploration, and here is the landscape we will try to cover: 3
Developmental Systems 1. Why are functions not an appropriate way to develop mathematical models for many biological systems? 2. Can we develop an inductive algorithmic mathematical framework -- I call it developmental systems theory -- as a relational generalization of functions and function based explanations that can be used to model biological (and other) systems, and that will return dynamical systems under special cases? 3. Is there a concrete way to think about information and about interactions in biological systems, so that we can develop models of very complex, hierarchical and network structured systems in biology 4. Finally, can we address the inherent "messiness" of biology? Where does it come from? Are biologists just sloppier than other scientists? Or is there something deeper at work, some fundamental vagueness in biological systems? My view, as a data analyst, is certainly not from the heights, but from the ravines, and the local topography of a field and bench-top biologist. I can't do it Rosen's way, so I won't mimic his arguments in Life Itself (I urge you to read this book, and follow him on his epic journey into the heart of formalizing the living). I didn't even fully understand many of Rosen's key arguments until I had gotten to essentially the same territory by a very different set of trails and leads. So, I will give you the path I took, as clearly and honestly as I can, while still hewing somewhat to the dictates of a scientific narrative. I give you the journey in progress, what others and I have worked out so far, and also what needs to be worked out further as I see it. Specifically, I attempt the following in this paper: 1. Develop a model of the measurement process, that results in "traces" the material patterns and observations that supply the raw empirical data that a scientist works with, and develop some intuition as to what we can and can't compute from traces 2. Use our model to look at data sets, and identify when we can develop a functional (that is dynamical systems) model of our observations and when we can't. 3. Develop a model of developmental systems, that is both capable of being formalized as operations on a particular kind of set, a generalization of fuzzy sets which we will
4
Developmental Systems call Vague Sets, and that can also be used to define real limits in our ability to gather the very observations we need to develop models of complex biological systems. 4. Explore how the ideas developed here, can be used be connected to other research in biology and computer science, and to connect biology and computer science. Obviously, this is vast territory to explore, and so we can only explore it superficially and rather rapidly in this paper, so let's start now.
5
Developmental Systems
Traces: "Thoughts are words. When rhythmic they are poems. Chaotic, they slosh like bilge in the mind keeping it afloat. Reel after reel of ruffed grouse drum on the brain -- Catch them!" -- S.N. Salthe, Flux
We call it sensation. It is a fundamental characteristic of organisms, the primitive basis of an organism's ability to respond to changes external and internal. In essence, it is a process of measurement. Measurement itself requires a material interaction. Material interaction requires establishing a relation between two or more cohesive systems. And so, we will seek a general framework for measurement, whether it be one organism perceiving something about another organism, or more generally one cohesive system measuring a second cohesive system. We will develop a simple model of measurement, and we will use it as a probe. Our model has the following terms, four components of an interaction, measurement, communication, and three artifacts of the same communication, measurement, interaction. For simplicity sake we will name the system of relationship "Measurement", and identify it as a system with components (subsystems) and artifacts (by products of the actions of the system). We could equally well have named it, in less neutral terms, communication, or interaction. The first component is "Probe System". The second component is "Object System" The third component is "Carrier System" The fourth component (and first artifact) is "Trace" (which may or may not be a system, depending on the cohesion of the substrate that is materially traced). The second artifact is "Signature" (which is a symmetry grouping on traces). The third artifact is "State" (which arises from scale differences in the subsystems of Measurement) and is derived from local groupings and orderings of the material trace. The artifacts Signature and State, are thus secondary artifacts of the primary artifact, Trace.
6
Developmental Systems These terms define the system "Measurement" as a meta-system composed of subsystems with a specific set of causal relationships. The notions are functional ("functional" in the biological sense, they identify different roles within the act of measurement), rather than structural. That is, the terms are defined by what an object corresponding to those terms does with respect to the relation "measurement", rather than how it is made. They are also defined by context. Every component term also represents in and of itself a cohesive system. A probe system (the subject system) is capable of measuring another system (the object system) by receiving a sign or signal from it via a carrier. A carrier is the medium via which the sign or signal is transported. To make this less abstract, a visualization might help at this point. Noting that all the subcomponents of measurements are systems, we will now simply call the probe system "probe", and the object system "system". Imagine a young girl playing alone on a beach, entertaining herself by playing a game with an imaginary friend. She is playing a game a bit like GO, using marks of X's and O's in the sand, in a large square board she has drawn in the sand. She draws lines with a small stick. There is a slight wind. As she fills in the cells with X's and (for her imaginary friend who is not good at moving material objects) 0's, the square fills up. But, as the wind picks up, she finds that her X's start looking much like her O's, and so she must play the game quickly, or the marks in the cells become unclear, and indeed the cells of her square themselves fade, so she is no longer sure where the edges are. Viewed from the perspective of the sand, it is the probe. The system being measured is the girl. The carrier is the stick. The trace is the shifting of sand particles. Viewed from the perspective of the girl, she is the probe. The carrier is light that allows her to see the marks made in the sand. She (or her imaginary friend) is also the object. She is playing by herself. She is measuring herself. Sometimes for days, there is little wind, and she can draw many GO-boards in the sand, play them and come back the next day and see the "memory" of her games in the sand. Other days, the GO board is undrawn by wind, faster than she can trace its edges, and she can't play at all.
7
Developmental Systems A simpler example is "I see Matsuno. Matsuno sees me". We alternate as probe systems, with the other as object systems. The carrier is light, tracing firing patterns on rod and cone cells. The scales that are important is the scale of light, first, and secondly the scale of the rods and cones at the back of the retina. The first, scale of the carrier, defines the maximal possible resolutions under conditions of ideal measurement. The second defines the maximum scale of resolution available in the trace. Whichever scale is smaller defines the scale of the trace. As the light dims, I see Matsuno less clearly. "I wave to Matsuno. Matsuno waves to me". Two systems measuring each other, can fire off subsystems, cascades of traces, read by further subsystems, and yet all constrained by the higher level measurement to result in synchronized activity. "I do a jig. Matsuno shakes his head". There is choice in the response on both sides, stimulus and response need not be equal. There is delay, until recognition or cognitive impact, "I throw a ball, it hits Matsuno". There is recognition followed by the possibility of synchronization, "Matsuno throws the ball back, and we play catch". What is a signature? It carries a generalization of the meaning it does in daily life. It is a characteristic pattern to a trace, that is a pattern shared by a group of traces or in other words it is a symmetry grouping against which the traces are invariant (and can thus be recognized as the same signature). Every time you sign a check, it represents a different trace. Your unique signature represents what is common to all such traces. It is an artifact, because it is derived from relationships among the traces, which themselves are artifacts of the measurement relationship. The process of modeling is to develop formal signatures (the mathematical model) that can be used to unify or group traces. The formal signatures supervene on each other if they can be seen to bring more and more traces into a relationship as instances of the same invariant. In the language of software design they are classes. In the language of mathematics, they are categories. What is a trace? Traces are the basis of empirical science, they are, in essence, all the evidence we have both to test deductions, and to stimulate the formation of hypotheses from observed patterns. They are what we pose against our formal signatures, and ask, "how does the trace fit, assuming this is the signature?" A trace is information left in the 8
Developmental Systems wake of the interaction. When our interaction is reflection of ourselves, they are selfinformation. Let us take a more prosaically scientific example. I am measuring a farm field experiment that looks at full-sibling family by cultural regime interactions in Douglas-fir, (an experiment that contrasts the effects of genotype against the effects of a particular earlier growth environment). I measure several variables related to the growth of the trees with a ruler and a caliper. Height of the seedling, ground level diameter, radial crown width, etc. I write the measurements down in a "write-in-the-rain" field-book. I will later transcribe the notes into a database program, and hopefully not make too many transcription errors. My notes on the paper, and the bit locations in the computer while materially different, can be said to share much the same information, the same signature. There is a correspondence they can be brought into that is the invariant. In terms of the situation as I am taking my field notes, I am the probe, light and the markings on a ruler or caliper are the carrier or medium of transmission of information, each seedling measured is the system, and the trace is my markings in the field book (itself representing the trace of patterns in my optical field, that create a trace of neural firings, that cause my "mind to move, that cause my hand to move …. it's traces all the way down!). There are several things to notice about the creation of a trace, 1. Ordering 2. Scale 3. Temporal Duration. 4. All functional parts of the relation are cohesive systems. and the parts may change their functional roles in the relation, as two sets of systems come into synchronization. Furthermore, all parts of the relation may themselves be composed of subparts conducting other measurements, and entering into other relations. First, the trace represents both a temporal and spatial ordering, of sequential markings on some material. This ordering relationship organizes the observations into a distinctive
9
Developmental Systems trace. This ordering also allows groupings on the trace, hence coarse graining and generalization. Secondly, scale is important at three different points. There is the scale at which the system generates a sign. In the tree example, this depends on the processes of primarily cell elongation that lead to growth. There is the scale of the carrier that transmits information. That is, in the case of light/ruler (or carrier) the smallest marking on the ruler given that the wavelength of light is much smaller. There is the scale at which I can receive the carrier --which is dependent on the rod and cone cells in my eyes. If I had bad eyesight, the scale of my observation may be greater than the scale on the ruler, and I may be interpolating and guessing, in which case the trace I write down may not be coincident with the actual trace in my optical field, and the apparent resolution is false. Whichever of the three scales is largest defines the resolution of the trace, the distinctive markings. If the scale is millimeters (mm), then anything within a mm increment is treated as equal, as effectively being in the same state, or you could say, in the same cell of measurement. Note here, that state itself has moved from an absolute and unconditional, to relational status. State represents simply the lowest scale of resolution on a trace, of a dynamicmulti-scale interaction, the limits of measurement. It is therefore dependant on the different scales (temporal and spatial).of the systems that compose the measurement. Lemke (1999) has called such multi-scale interactions "scale-heterogeneity", and distinguishes them from the assumption of "scale-homogeneity" used in many models of physical systems. He asserts that scale-heterogeneity is the norm in our complex world. Scale-heterogeneity implies scale can change during a measurement process (say with dimming light), so the degree of precision represented by a state, can itself be dynamic. In other words, the state, can be constantly re-negotiated by two systems in inter-action. We can use this notion of state, as the minimal precision, to define a finite state space, which in and of itself reflects nothing more than the lowest level of resolution of our observations.
10
Developmental Systems Thirdly, the markings or symbols I place on the page, stand for a whole host of probesystem interactions in the background that allow me to convert a pattern of firings in the optical field of rod and cone cells (the raw trace) to ultimately a set of symbols, the highly derived trace in my field book. That is, I must assume some process of semiosis, to convert raw traces, markings on a material substrate, for the derived traces, where each marking now stands for a grouping on the raw trace (and the intermediate series of traces), represented symbolically. Lemke (1999) has called this process the change from topological information (the raw trace) to typological information (the symbol, or derived trace, essentially a signature), and noted that topological information and typological information can alternate as traces cascade through subcomponents of a cohesive system. Temporal and spatial scale differences between probes and object systems can render them effectively invisible to each other at times. I walking in a bog, am essentially unseen by the bog ecosystem, at its scale of being. My individual footprints leave barely a mark, barely a trace. A group of tourists being marched through the bog daily along the same path, is soon seen by the bog ecosystem, marking it, changing the structure of the peat locally, and by that, the local composition of species along the path. That is, aggregate enough infinitesimal traces in a very short order, and a very significant trace may soon emerge, the microscopic impinges on the macroscopic. Finally, the trace is of limited duration, it is impermanent, and subject to other agencies, interacting with its substrate and creating other traces that revise or obscure the original trace. A trace acts as the memory of an interaction, communication, measurement for only a limited duration, but during that duration it may in its turn be the system measured by other probes, which means that by the time the last trace is being drawn, the initial trace that caused it has vanished from sight, and we have an incomplete causal record. Because, it does indeed appear to be "traces" all the way down, or in other-words scaleheterogenous networks of system interaction, we must draw some boundaries. We do this by encapsulation, and assuming for a given purpose, only one or two hierarchical levels, 11
Developmental Systems which are the focus of our attention. In his extended analysis of hierarchical structures in biology Salthe (1985, 1993) points out that often our analysis of a system is "triadic" in that the analysis breaks down to a "focal level" that is explained both in terms arrangements of its sub-components (i.e. systems one hierarchical level below our focal level) and constraints imposed by the a larger system that our focal level is itself a component of (i.e. the system one hierarchical level above our focal level).The process of scientific observations is to focus on recording something, and not on recording everything. Furthermore, observations in the context of an experiment, are really no fundamentally different than any other experiential sensory perception, and generalize to systems of interaction. If you were to take any system you are very familiar with (whatever your field), and being to make a diagram of all the probe-carrier-trace relations, you would soon end up with a very complex network of networks. This network of networks of interactions is always there, but it is overwhelming, and so we only focus on subcomponents of it, those most strongly interacting with us. That is, the actual raw traces of our measurement process are much richer than the filtered traces that become our objects of scientific study. This is not something uniquely human, a planet does the same thing, reacting most to the largest local gravitational objects, they dominate its dynamics under most conditions. To some extent all systems are connected, but by choosing the level of connections we will focus on, all systems are conceptually isolatable (if not in practice) by making a cut or distinction, to limit the outward chain of implication, and by encapsulation, to limit the inward chain of implication. That is, we only look so many systems out, from our focal system, and we do not look deeply within the subsystems it is composed of, but allow the behaviors of all subsystems to be encapsulated. In short I do not use quantum chromodynamics to explain why absent-mindedly spilling hot tea on myself hurts. While any explanation of complex systems in general must account for this network of relations as a whole, we will focus the rest of this narrative on one artifact of the system, the trace, the recorded series of observations. In the next section, we will build upon the 12
Developmental Systems trace, to develop the mathematical notions of function, and their generalization, relations. The fact that a single word can have very different meanings can be confusing, so I should make some clarifications on the word, "function". In this section, we have used function, in the sense of "What is the function of this?", that is, what is the role of a component with respect to the whole. This is the way that function is used commonly in biological studies. In the next section we will use the word function, in its mathematical sense, which is very different. In that sense, a function represents a kind of map.
13
Developmental Systems
Function and Relations: "My biggest disappointment is that I'm unable to use program-size complexity to make mathematics out of Darwin, to prove that life must evolve, because it's very hard to make my kind of complexity increase." -- G.J. Chaitin, The Unknowable
Now imagine another world, much like ours, but with a different historical order to the development of the sciences. No Newton existed here, no Leibniz, no infinitesimals were posited, no limit theorem was not explicated. Calculus has not been invented yet, real numbers have not been invented either, but computers have, algorithms have. Their mathematics is procedural and operates on numbers of finite precision. They have not yet imagined a number with infinite precision, they have not yet imagined the reals, or the continuum. And in this world within the context of computations, functions exist, as a mapping of states, where a given input state uniquely maps to a single output state, according to either a look-up table or a generating formula. The notion of functions is generalized to be applicable to any set of symbols, that can be used as signs, whether they represent counting numbers, words for emotions, or symbols to represent abstract qualities. A dynamical system is defined as an ordered series of the mapping of an input symbol to an output symbol, irrespective of any meaning applied to that symbol: If (input state) then (output state). They imagine their functions operating in a space that is an infinite number of finitely precise cells. Because infinity can not be measured in the field or the lab-bench, the working scientists usually restrict themselves to finite spaces, with the number of cells constrained by the observed phenomena and leave infinity to the mathematicians who are beyond physical containment. This world whose mathematical imagination is limited by the finite and measurable, are surely no slouches with respect to formal systems, which in their world are represented not by axioms and theorems, but by computer programs, more generally by algorithms. If something can be stated as a set of sequential steps or operations, there is some or other industrious individual who has struggled with the stating of the algorithm for just that something. Rather than doing proofs by contradiction, they do proofs based on the 14
Developmental Systems halting or non-halting of computer programs that implement the algorithms. An open question in the sciences of this world is that while a function uniquely generates a trace, can a trace be compressed back into its generating function, when only the trace exists, and the generating function is unknown? What evidence does the function betray in its trace? A young algorithm designer is given a set of traces, and asked to find a way to compress them, and to report back on their properties. The functions are an ordered series of input/outputs. Depending on your perspective, the input/outputs can be considered, cause/effect or before/after sequences. The prior variable is simply called "X" and the posterior variable is simply called "Y". and X 's role is to be the domain and Y's role is to be the codomain or the consequence of X, where the logical ordering of roles is always, X before Y, or X implies Y, stated in algorithms as "If (X) then (Y). Below are the traces our algorithm designer is given.
15
Developmental Systems
Trace 1 X
Trace 2 Y
X
Trace 3 Y
X
Trace 4 Y
X
Y
110
225
420
352805
2100
8841005
7
312250
120
245
430
369805
2200
9702005
8
532600
130
265
440
387205
2300 10603005
9
853050
140
285
450
405005
2400 11544005
10
1300120
150
305
460
423205
2500 12525005
11
1903450
160
325
470
441805
2600 13546005
12
2695800
170
345
480
460805
2700 14607005
13
3713050
180
365
490
480205
2800 15708005
14
4994200
190
385
500
500005
2900 16849005
15
6581370
200
405
510
520205
3000 18030005
16
8519800
210
425
520
540805
3100 19251005
17 10857850
220
445
530
561805
3200 20512005
18 13647000
230
465
540
583205
3300 21813005
19 16941850
240
485
550
605005
3400 23154005
20 20800120
250
505
560
627205
3500 24535005
21 25282650
260
525
570
649805
3600 25956005
22 30453400
270
545
580
672805
3700 27417005
23 36379450
280
565
590
696205
3800 28918005
24 43131000
290
585
600
720005
3900 30459005
25 50781370
300
605
610
744205
4000 32040005
26 59407000
Table 1: Four Traces for Four Functions There are four traces from apparently different functions. He plots the traces out on graph paper, sees that they are all rising functions. He guesses they may be polynomial functions of some order. But how to get back from the trace to the generating polynomial function? He develops traces from functions he knows well, does some experimentation, and hits upon a simple method. First he linearly orders the X's (ascending or descending order does not matter). If he calculates the differences between two subsequent X's, their subsequent Y's (that is for example, the difference between the first and second X and the
16
Developmental Systems first and second Y), and takes the ratio, he reaches a constant value for a linear equation of the form: [1] Y=bX+C This constant, has a value equal to b, and since it is derived from the trace, we will call it b* to represent that it is an estimate of b. That is, [2] (Interval [Yn - Yn-1])/(Interval [Xn -Xn-1]) = b* This constant value holds, irrespective of the interval he takes, as long as it is uniformly taken (that is the increments between X's are the same). Since his procedure is wholly local, comparing adjacent values along an ordering of the domain X, there is a b value calculated for every interval. That is, if there are N observations, there are N-1 estimations of b*. In the case where the intervals on the ordering of X are identical, all values for b* are identical, and equal to b of the generating function. If there are large differences in the X interval, there is fluctuation in the value of b*, but the modal value along the ordering is the same constant for a given linear equation. This value, b* is equal to the change in the codomain relative to changes in the domain for the linear equation. By multiplying his estimated b* by the original X values, and subtracting that result from the original Y values, he is able to also estimate the constant of the original function, C, and this value is itself constant, C* so that [3] C* = Y - Y*, where Y*=b*X. If he takes a higher order polynomial function, such as [4] Y=bX4 + C
17
Developmental Systems then he finds that he must take several repeated intervals of Y to reach a constant value, along the ordering, each time re-dividing by the original interval of X. That is, he takes the result of his first calculation (a ratio), and uses it as the new Y for effectively the same calculation. The procedure is thus recursive. He labels each interval of [Yn - Yn-1] as R1: IntY, the first time an interval is taken, and as R2: IntY, the second time, where R stands for recursion. Each time, he divides the interval by the original interval of X, which he calls IntX. These are all finite intervals. What he finds, is that for the fourth order polynomial, he reaches a constant at R4:IntY/IntX (the fourth recursion), and that the value of the constant is equal to the factorial of the order of the polynomial multiplied by b. That is, if there is a polynomial of order P, the constant value of the fourth recursion is R4:IntY/IntX (which we will simplify from labeling "R4:IntY/IntX "to simply "R4",) which we will call the "recursive", representing the point where the recursion reaches a constant interval ratio. The recursive for a simple fourth order polynomial has the following form: [5] R4=(b*)(P*!) which recovers for us, P the order of the polynomial, b the coefficient of X. We can recover C by the same method as for the linear equation. For example, for a trace generated from the equation Y=2X4 + 999, b*P! will be 48, and since this is the fourth recursion, P=4. Hence P! = 24 and R4/P*=b*=48/24=2. So, we can recover the equation exactly from the trace, and in general, for a simple polynomial function of order P [6] RP = (bp*P*!) In the more complicated case, of which the generating equation is a sum of polynomials terms, such as
18
Developmental Systems [7] Y=2X2+10X+5, or generally Y = bpXp +bp-1 X p-1 + … +b1X1 + C the algorithm must be extended, in that after the calculation of the first term -- in this case R2 -- estimation of the constant C does not yield C* as a constant along the ordering. Therefore another recursive set of intervals must be taken, which recovers the additive term of the second term of the equation (in this case a linear term). This requires a slight modification of our "R" notation, to keep track of both loops occurring in the procedure. (see appendix 1 for how this is represented as Rij) What is happening is the step-wise calculation of each term in the polynomial, -- from highest to lowest -- and only after the lowest order term in the polynomial has been estimated can the constant, C, be estimated by C*, and found to be invariant across the ordering (the calculations are shown in appendix 1 for a quadratic equation). So one has the same procedure looping at two levels. First we find the highest order term in the polynomial, which is calculated based on recursive calculation of ratios, until a constant or a minimum is reached. Then the estimated values of the function are compared against the original, to estimate the equations constant (or slope intercept), C. If this does not result in a constant along the ordering, the first procedure is repeated again. And so on, until every term of the polynomial has been estimated. Finally, he found the time behavior of his trace compression algorithm to be rather simple, where the running time was related to the order of the polynomial by the following equation, for a polynomial of order P: [8] Running Time ~ P + (P-1) + (P-2) + ….+1 which reduces to Running Time ~ P (P+1)/2
Please see appendix one for a worked out example and you can follow the calculations by hand or by computer. 19
Developmental Systems
By experimenting with various polynomials and their traces, the algorithm designer discovered the following: 1. He could fully recover from its trace any function that was the sum of polynomial terms, without any error, as long as the X intervals were uniform, and regardless of the actual size of the interval. That is the trace was fully compressible, and any local estimate, of b*, P*, C* equaled the global values b, P, C, for each term of the equation. 2. If intervals were irregular, there would be fluctuation around a modal value. Happy with these results, our enthusiastic algorithmizer (probe system) turned to some real data, and went out and measured some irises (object system), for their petal widths and petal lengths. He reasoned that width develops prior to length and thus width could serve as X to length's Y. Here is a sample of the trace he returned with (there were actually 150 samples, but we are only viewing a part of the sample, to develop a feel for the numbers).
20
Developmental Systems
Data X (Petal Width)
Bounded Relations Y (Petal Length)
Y*LowerBound=2X+0.8
Y*UpperBound=1X+2.3
0.2
1.6
1.2
2.5
0.2
1.6
1.2
2.5
0.2
1.7
1.2
2.5
0.2
1.9
1.2
2.5
0.3
1.3
1.4
2.6
0.3
1.3
1.4
2.6
0.3
1.4
1.4
2.6
0.3
1.4
1.4
2.6
0.3
1.4
1.4
2.6
0.3
1.5
1.4
2.6
0.3
1.7
1.4
2.6
0.4
1.3
1.6
2.7
0.4
1.5
1.6
2.7
0.4
1.5
1.6
2.7
0.4
1.5
1.6
2.7
0.4
1.6
1.6
2.7
0.4
1.7
1.6
2.7
0.4
1.9
1.6
2.7
0.5
1.7
1.8
2.8
0.6
1.6
2.0
2.9
Table 2: Raw Data and Estimated Values for Iris Morphology Relations
His data is represented according to the precision he measured with, in this case to 10ths of centimeters with a ruler. He immediately encountered difficulties in applying the trace compression algorithm. First, for a given X value, there were often many Y's. That is, the data represented not a function, but a relation. such that, If (X) then (Y1 or Y2 or Y3). 21
Developmental Systems Every time there was more than one Y, for an X, he would have a branching point. So, he could not reduce the trace down to a single equation. Rather he would have to calculate an equation for each branching point. That is, if the branching points represented alternative "pathways", he would have to calculate an equation for each alternate pathway. The trace was no longer fully compressible. Table 2 gives two of the equations he calculated using the trace compression algorithm on the iris morphology data. The first equation was done by for every unique X value, selecting the minimum value of Y given X. And the second equation was developed by selecting the maximum value of Y given X. That is, the two equations bounded the original relations below and above. These results made him a bit happier. The trace was somewhat compressible, and could be bounded. It could even be compressed into a number of equations, just not a single equation. He decided to engage in some experimental algorithm design, and gathered further data sets, he gathered frogs and measured them, he gathered cones off trees and measured them, he measured the how fast the corn in his neighbor's yard grew on a weekly basis. What he found were the following general trends after much experimentation with traces measured in the field. He made a list of how traces in the field, differed from traces generated from an equation: 1. At the level of precision available to him, the resulting traces were relations not functions. That is, each unique X value maps one or more Y values. 2. The value of the coefficient for each term in the polynomial functions compressed from the trace could vary with the ordering, being close to constant for a sequential series of values on the domain, then changing and being constant for another series of values on the domain. . He took this to mean that a trace was varying in its slope, but not in its polynomial order, or in other words the generating function was changing its slope over the ordering but not its general shape, hence multiple functions of the same general shape needed to be added to compress the trace.
22
Developmental Systems 3. He no longer reached a modal constant after a fixed number of recursions, but a minimum for each point in the ordering. That is the interval ratios decreased over recursions, then began to increase. 4. The number of recursions to reach a minimum value could also vary for different parts of a trace. Along the ordering of X, the shape of the functions the trace could be compressed into was also variable. 5. If he grouped neighboring X values, and neighboring Y values, and took their median, he was often after a number of groupings able to recover a single function. That is, if he rough-grained over the relations, or looked at the data as if it were taken with a lower level of precision than it actually was, a single function would recover. He took this to mean that the compressibility into a function had scale dependence. 6. If he used time as his X and the value of a rolled die as his Y, the resulting trace was not compressible, except in the trivial case where his number of recursions was approaching the sample size. 7. For a given X, with multiple Y's, there was not an equal likelihood for each Y. Thus, if (X) then (0.7Y1 or 0.2Y2 or 0.1Y3). He considered this bias in pathways taken among alternative pathways for a given X as "propensities", and he saw the likelihood of each propensity as the likeliness of the system to take that path. Since he had no precision below that of his state space, as far as he could tell, the system somehow chose among alternatives. For a while he considered looking at a theory of "hidden variables" to represent what may happen below the scale of his measurements, but the effort proved very frustrating, since he could find no way to prove the various hidden variable theories against data. The formal part of him was rather disappointed with the fact that traces in general are not fully compressible. But then he took heart in the idea that degree of compressibility is itself a characteristic of traces. He began to look at himself as a kind of taxonomist of traces. He also investigated other algorithms the help him understand traces. Because he was working with finite precision in his X and Y, he knew that he could calculate the number 23
Developmental Systems of distinguishable states in his space as dependent on the precision of the domain and codomain, in the following way: [9] # Unique States in X = (Max X - Min X)/Precision of X, and # Unique States in Y = (Max Y - Min Y)/Precision of Y This gave him the total number of possible unique cells (combinations of states along two or more dimensions of his space) as [10] (# Unique States in X) (# Unique States in Y) = Distinguishable Cells, and that the number of total paths or mappings from X to Y was [11] (# Unique States in Y)(# Unique States in X) = Total # Possible Mappings from X to Y. He could calculate these values for any original trace, or a trace derived as a grouping on X (coarse graining), and compare it to the actual values from the data as a ratio with the actual count as numerator and the calculated count as denominator. These ratios would give him an idea of the compressibility of a trace. The smaller the ratio of the actual value divided by the calculated value, the more compressible the trace, assuming you have many more data points than cells (when he didn't he could either use techniques to resample data, or coarse grain to a level with fewer distinguishable cells and fewer paths). This allowed him to relate trace compressibility to measures of order, where the trace was more ordered (in terms of states and paths respectively) as the ratios tended towards 0. As the possibilities of trace compression become clearer, our young algorithm designer is overwhelmed with enthusiasm. He sees traces everywhere. He seeks better compression algorithms. Exponential function trace compression algorithms, sine function trace compression algorithms. …. meta-trace controller algorithms that determine which trace algorithm to use. He seeks to run trace algorithms in reverse, treating the trace as a series of intervals, and working backwards … He seeks to understand their behavior in finite 24
Developmental Systems spaces, in terms o f not only the amount of space occupied but in terms of the shape of the occupied space, in terms of the number of branching paths in data, relative to all possible paths given the finite spaces he works in …….. We will leave our young algorithm designer well on his way to developing a recursive algorithmic calculus and an algorithmic information theory based on traces and the possibilities and limits of trace compression. (See Chaitin, 1998, and 1999 for his account of the algorithmic information theory developed in our world; see Fisher, 1936 for his analysis of Edgar Anderson's iris data; see Ulanowicz 1997 and 1999 for a development of the notion of propensities in terms of calculations similar to those we have looked at). We return to our world, our time, and our immediate problem: biology. We truly are "victims of history", when it comes to our development of the biological sciences. Our three century long affair with the function, so successful in the physical realm, allows us to be satisfied in the biological realm with the development of functional models which often explain less that 10 % of the behavior captured in a trace, and yet are "statistically significant" that is, deviate somewhat from the results you would get by just tossing a coin. What other science would accept such poor evidence for a model? We throw away all the information on the true relations, as the error terms in our dysfunctional models. We take rich biological consequence, and turn it into stupefyingly boring simplifications. We do not look too closely at the level of compressibility our models afford the evidence. The error was not in our data but in our choice of models! The development of an algorithm for compressing a trace that I have illustrated here for the case of polynomial functions demonstrates a number of data (trace) based caveats about when a functional approach to trace (data) compression can be used. The lessons we have learned from this extended example can be generalized to properties of functions in general, and where the traces they generate (that is the dynamical systems models concordant with them) fail the traces of our empirical experience.
25
Developmental Systems From our extended numerical example of trace compression we can glean some insights as to the features that distinguish the idealized traces generated from functions (models of dynamical systems) and the actual biological traces found wild in the field: : 1. Wild traces exist in a finite state space, where the resolution of state space depends on the actual measurement precision. Given the development of a trace, as having a fixed precision there area finite number of intervals for a given variable, and a multivariate space can be seen as a finite number of distinguishable cells. These correspond to what we know as "state". But, as presented in the section on traces -- states are artifacts of the scales relevant in the relational system required for measurement. 2. Wild traces reflect processes where form varies. A trace from a single function recurses to a single constant, which holds for the complete ordering. But this need not be so, for a trace generated by a biological system. The single constant, is because the form of the function is invariant with the ordering of X (often time, space, causal factor, etc.). There is a single attractor. A pattern of changing constants in a trace being analyzed using the compression algorithm, would then represent movements across multiple attractors by the system that generated the trace, that is, in wild systems form changes, which would reflect change in the dynamical laws governing the system. 3. Wild traces are scale dependent, ideal traces are scale invariant. If the compression of a trace is found to be reducible to a function after grouping of intervals (grouping adjacent cells in the finite space) then, a functional model of the trace is available at that scale only. The reason ideal functions are scale invariant is that a function is selfsimilar at all scales, that is, they are the limit of the fractal concept, with integer dimensions, continuity, and self-similarity. 4. Wild traces are relational, not functional in nature. For a function, there is a one-toone relationship among the domain and the codomain. For biological data, where a one-to-many relation results, we have a branching point. The branching points can be used to define multiple traces, each of which can be estimated to varying degrees with a function. The greater the number of functions required, and the lower the fit of the function to the original trace, the less compressible the trace overall. This results in 26
Developmental Systems traces being only partially compressible both due to components that could be effectively designated random (error in fit), and components that exhibit regular change, such as movement across attractors. 5. Wild traces have varying levels of compressions; ideal traces are fully compressible. This follows from logically from point 4. Compressibility depends on number of choices (branch points) for a given X. If we group at a higher scale -- that is represent our data at a more coarse grained scale than it was collected -- we may be able to recover a functional relationship. It is at this scale, where we can develop a dynamical model of the system. But the model only applies at that scale. It does not hold in general. 6. Branching points in a trace lead to the notion of propensities. Because at a branching point, some mappings are more likely than others, there is a propensity for the system to take certain directions, but it may take other directions, within the codomain space available to it. Put simply, a cause can have alternate effects rather than a single fully determined, or statistically determined effect. 7. Because there can be one X to many Y relationships, and also one Y to many X relationships, the maps we build will often not commute and will not be invertable. This provides an intrinsic asymmetry to implication, i.e. the mapping of X (cause) on Y (effect) for a time ordering will not be reversible, thus the time dependant behavior of the system is inherently irreversible at the level of sequential mappings on state spaces We now have the tools, and intuitions we need to move into the next section and develop a generalization from fuzzy sets to vague sets. Our key inward, is the realization that a fuzzy set is represented by a function, and that it carries all the assumptions inherent in the notion of functions.
27
Developmental Systems
Fuzzy Sets as Functions, Vague Sets as Relations "The apparition of these faces in a crowd; Petals on a wet, black bough." -- Ezra Pound, In a Station of the Metro
Set theory is composed of sets and elements. It is sort of a ground zero in mathematics, the making of distinctions that forms the basis for all the other roles in the theater of math, the countable numbers, the reals, the imaginary numbers. For a clear introduction to set theory see Klir et al, (1997), and for an authoritative account of the visual representation sets-as-points that I will use later in this section, see Kosko (1992,1997). In this section we will use a metaphor to move from classical, through to fuzzy and finally to vague sets in quick order. Consider set theory in terms of theatre; sets are the roles, elements are potential cast members hoping to fill those roles. Set theory can then be looked at, as the relationship between roles and actors. You are casting a play, say Hamlet. You've read the script, you understand it intimately, you know the roles. You just have to assign the actors. You have a number of roles to fill, and the potential actors come in. The fools, Rosencrantz and Guildenstern. It's obvious, those two jokers in the corner. They're perfect. No one else is even a possibility. We call them elements of the crisp set "Fool" with membership status 1, and everyone else is a 0. Now Hamlet. Everyone wants to be Hamlet. Six fellows are brushing up on being broodily intense and indecisive. Two of the potential Hamlets you consider a bit too old; they would do better as Polonius. Two of them probably could do the job. Two of them look like they could do a really good job. Of these 6 people there are no out-right rejects, and no one who you think is perfect for the job. You can make a list, ordering degrees of suitability for the role of Hamlet as well as your criteria (the rule) for each degree of suitability, and unequivocally assign each potential Hamlet a "fit" value between 1 and 0, where 1 is the perfect Hamlet, and 0 is totally unfit for the role of Hamlet. We call these actors elements of the fuzzy set "Hamlet", with the first two actors having a fit to the role of 0. The second two actors have assignments of 0.4 and 0.5 respectively, and the last two you assign the same fit, 0.8. All the other actors who are not even being considered for 28
Developmental Systems Hamlet, naturally have a fit value of 0. You've got a 10-point scale from 0 to 1 because you think you can discriminate about 10 levels. If you were less familiar with the play, you might discriminate about five levels. Now, consider the case where you do not have a play in hand, the roles are not assigned. You just have the actors. Well initially, you're not even sure what the play is. But you question the actors on their interests. You workshop. Gradually you find out that there are certain categories you could use. You develop proto-roles. The actors try them out, you continue to improvise. The roles grow out of how the individual actors relate to each other. Four of the actors just want to be in a crowd scene. Two actors are seriously interested in a lead role, and are deeply committed to the role-discovery process. One classically trained actor quits because he thinks your whole approach to theatre is insane. Over a period of time, clear roles develop, and at that point, you can treat the situation as you did the play Hamlet. You now have clear roles. The difference is this clarity has emerged through improvisation and interactions. You call your play, "Rosencrantz and Guildenstern Search for Vagueness". Set theory assumes an identifiable set, and clear rules for membership. This is the nominal set. However, if we consider sets to originate out of a process, rather than existing since the beginning of time, we must look for natural sets, groupings that arise out of interactions, or measurement. Crisp sets, assume well defined universal rules for membership, and that every potential element is either a member of the set, or not a member of the set. Crisp sets lead to what has been called the liar's paradox, "I say I am a liar". Have I told the truth or have I lied? Fuzzy sets assume a well defined universal rule for membership, and that membership occurs on a continuum, with either 0 or 1. They assign a truth value of 0.5 to the liar. Vague sets assume that natural groupings emerge out of interactions, and that sets themselves emerge out of the groupings of their members. These groupings represent the development of rules to identify set membership. In a vague set, you do not initially know what a liar is, but your understanding emerges out of interactions with liars and truth tellers, you develop the concept of "liar" and "truth-teller" and gradually are able to assign people you meet into these roles. 29
Developmental Systems
Notice, we have moved from a static definition of sets, with universal rules, to a dynamic definition, where rules locally emerge from interactions. Let us use an example from systematics again. Say we have 10 samples, and we have measured a variable that we believe discriminates among species, and based on that, we have divided up the 10 samples into three piles, representing each of the species, and we make a data chart to keep track of our distinctions. Since the data chart is based on our measured variable, it could also be called the "rule table" for assigning set membership to an element. . Our measured variable, X, is represented in one unit increments, and Y is our assignment of membership for each sample where we distinguish "fit" in 1/10th unit increments. Thus, our data table of measurements and fit values says essentially, if X then Y, and we have a great deal of experience now in handling such if--then statements. Our results for a crisp set would resemble that below:
Species 1 X
Y
Species 2 X
Y
Species 3 X
Y
1
1
1
0
1
0
2
1
2
0
2
0
3
1
3
0
3
0
4
1
4
0
4
0
5
1
5
0
5
0
6
0
6
1
6
0
7
0
7
1
7
0
8
0
8
0
8
1
9
0
9
0
9
1
10
0
10
0
10
1
30
Developmental Systems Table 3: Crisp Sets Each species is directly assign-able to one set. That is, we can divide the samples clearly into three piles. And our measurements explicitly assign each sample to one pile, and membership (or fit) in a pile is binary: either 0 or 1. This is what the data table would look like if a fuzzy assignment could be made Species 1 X
Y
Species 2 X
Y
Species 3 X
Y
1
1.0
1
0.0
1
0.0
2
0.9
2
0.0
2
0.0
3
0.8
3
0.0
3
0.0
4
0.7
4
0.1
4
0.0
5
0.7
5
0.3
5
0.1
6
0.1
6
0.9
6
0.2
7
0.0
7
1.0
7
0.3
8
0.0
8
0.3
8
0.8
9
0.0
9
0.2
9
0.9
10
0.0
10
0.0
10
1.0
Table 4: Fuzzy Sets We still recognize and can group our samples into three distinct piles, which we associate with different species. In this case, not all elements are assign-able to a pile, intermediate forms show up. But they are recognizably intermediate between a pair of species, and so we can assign them a position between the piles, and order them between piles as to degree of fit. We can also "draw a line" through the ordering called an alpha-cut, and scoop up all samples with a value higher than the alpha cut into the nearest pile.
31
Developmental Systems
This is what the data table would look like if vague assignments were made. We represent the same individuals (elements) measured at three different times and their increasing specification, or move towards crisp sets. Species 1 Time 1
X
Y
Species 2 X
Y
Species 3 X
Y
1
---
1
---
1
---
1
1.0
1
0.0
2
0.0
1
0.9
2
0.0
3
0.0
1
0.8
3
0.0
4
0.0
4
0.7
3
0.1
4
0.1
5
0.7
3
0.3
4
0.2
1
1.0
1
0.0
1
0.0
2
1.0
2
0.0
2
0.0
3
1.0
3
0.0
3
0.0
4
0.7
4
0.0
4
0.0
5
0.7
5
0.3
5
0.0
6
0.1
6
0.9
6
0.0
7
0.0
7
1.0
7
0.3
8
0.0
8
0.0
8
0.9
9
0.0
9
0.0
9
1.0
10
0.0
10
0.0
10
1.0
Time 2
Time 3
Table 5: Vague Sets We now look at the data over time; for example if one could make repeated observations on the same individual elements (systems). At time one all the individuals are essentially identical in value, for the measured variable -- and are not assign-able to a membership value in a species. The observer (probe system) has no way to abstract a species concept or species types from the individual samples. There is no grouping above the samples 32
Developmental Systems themselves. They need not be identical, they are simply indistinguishable based on the measurement criteria. At time two, a species concept begins to develop. We can make some distinctions, but there is hesitation, a membership function can not be clearly assigned as a single value, rather there is a range of values, of putative set memberships for each measurement. We can not assign a function, but as groupings on elements begin to emerge, we can delimit a relation. At time three, clear species categories have emerged and we can assign fuzzy membership values to samples, and recognize intermediate forms. Over time the vagueness has been specified and species have emerged. To demonstrate that this problem of vagueness is not unique to the species problem, let us think of another developmental system, the ecosystem whose developmental pattern we call "succession" Think of the categories we apply to ecosystems. Herbivores, detrivores, autotrophs, carnivores. Each of these are roles we use to rough grain across individual organisms, even species. The classes have fixed relations to each other. The autotrophs produce the raw material and harness energy from the sun. The herbivores eat the autotrophs. The carnivores eat the herbivores. The detrivores eat everything. But now, think of a time, before these relations had been established, before a set of species had coevolved for millennia. The roles, the relations between the organisms, the very categories would have been less clear. Now think further back, to a time before organisms, there would simply be various complex molecules, some of which could capture light energy, some of which were capable of harnessing other chemicals; there were even some biomolecules with simple metabolism like abilities. Our categories, "carnivore," "detrivore," "autotroph", emerged out of a long process of co-evolution. Their roles developed, and out of it the categories we assign functional components of an ecosystem. Sets are emergent. This view of sets may seem strange to the mathematician, used to sets as formal objects. My focus is on the primitive concepts that get formalized, and in this case, a set corresponds to the ability to make a distinct grouping on elements. When elements are individuals and not types, then, because the elements themselves change, their set assignments must change. This weakening of the set concept, is empirically based, and 33
Developmental Systems follows from attempting to derive sets based on developing individuals, which unlike types or natural kinds, are not static. I believe such a change does not weaken the set concept, but makes it dynamic and capable of handling the biological phenomena of the individual, and the vagueness inherent in developing systems. Mathematics abstracts from reality, and I suggest this is a useful abstraction, one which may lead to interesting mathematics. Sets originate from experience, set concepts are the groupings we give to experience, and in that sense, they can be seen to have a dynamical nature. In terms of the development of individuals, set concepts are emergent, and as individuals fall into clearly assign-able groupings over the course of their development, set assignment moves from vague, to fuzzy, to ultimately crisp in extreme cases. There is a useful way of viewing fuzzy sets, known as "sets-as-points" that brings out some of the possibilities of vague sets. While originally used primarily as a way of visualizing fuzzy sets, in Kosko's (1992, 1997) development of the sets-as-points representation, it is not simply an interpretative and heuristic device to introduce fuzzy sets, but the fundamental space, the stage upon which fuzzy sets act, and he uses it to define a number of generalizations. Essentially, it is an Euclidean space, as pictured below. 1.0
*
*
0.9 0.8
*
0.7 0.6 0.5
*
0.4
*
0.3 0.2 0.1
*
* 34
Developmental Systems
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Figure 1: Sets as Points Figure 1 presents a sets-as-points representation of a two-element set. While usually presented as a continuous space, I present the sets-as-points representation as a discrete space, following from how it would actually emerge from a trace or data. The sets-as-points representation illustrates the power set, all possible subsets of a given number of elements. For a two-element set, there are 4 possible crisp subsets: (1,1), (1,0), (0,1) and (0,0). These are represented as the vertices (four corners) in the figure above. Fuzzy sets, which are defined by the membership function, define a point in space, and of course, in the continuum there would be an infinity of such fuzzy sets. In the discrete space, I have illustrated, there are 100 distinguishable fuzzy sets. If you had two elements with membership (0.8,0.4) in a set, they would define that point in the X-Y plane, where the horizontal X axis represents the membership value of the first element, and the vertical Y axis represent the membership value of the second element. The complement of the set would be represented by the points (0.4, 0.8). Sets with a greater number of elements would be represented in higher-dimensional spaces. Within this sets-as-points representation, one point is fuzzier than all the rest, that is the midpoint (0.5,0.5) which represents maximal fuzziness, and the liar's paradox. Within this space, Kosko (1998) develops a number of results, essentially by using analytic geometry, he develops measures of the distance between sets, develops entropy measures, and finally develops a measure of information, and an information field concept within the sets as points representation of a fuzzy set. What geometric prospects are opened by vague sets? The set no longer represents a single point in space; rather the "point" is smeared out, into an area, proportional to the vagueness of the set. In the finite representation of sets-as-points (more appropriately 35
Developmental Systems called "sets-as-cells") it takes up several contiguous cells. We can represent this as I have the dark grey area in Figure 1. Because the actual points are smeared out, any measure of the distance between sets would be itself fuzzy. Just as fuzzy sets generalize crisp sets, where crisp sets represent vertices, vague sets generalize fuzzy sets, where fuzzy sets represent localization of the point. In vague sets, the point itself is fuzzified, or rather, its location is. I have offered so far, what may be called an operational definition of vagueness. The essential idea is that vagueness represents an inability to make distinctions, hence a limit to measurement. Individual elements are distinguishable from the background, say individual embryos -- but no grouping is possible. We can not say that any two elements are more similar, or more dissimilar from each other, nor that they are identical. The vagueness is resolved through increasing specification, because the "elements" are individuals, are cohesive systems in development, and their properties are changing. As new features are added, the elements become distinguishable into groupings, categories emerge, clear sets emerge. There is a certain degree of relativism to this view, and in this context I will offer a final image, that builds on the sets-as-points space. The sets-as-points representation assumes an Euclidean space. What if sets-as-smears existed in a non-Euclidean space? Imagine a field of distinguishable points drawn on a flat piece of some transparent extremely thin fabric. Take a single point, and begin curving the fabric into a sphere around it, roll it into a ball (perhaps use a baseball). Keep rolling, points begin to overlay each other. Now take a single point on the fabric, and roll it around a marble. Now roll it around a ball bearing. Soon, as you roll the transparent fabric around more and more miniscule spheres all the points begin to smudge and become indistinguishable; they are "superposed" over each other. Now, imagine the fabric flat again, take another point, and begin rolling both sides of the fabric away from this central point in a "saddle" shape, where both sides curve away from the central point, as the process continues, two separate planes appear, the fabric is now split around the central point. The fabric spread flat and its points represent the sets-as-points representation in a flat Euclidean space or 0 curvature; the first image 36
Developmental Systems of rolling the fabric into spheres, represents a space with positive curvature; rolling the fabric away from a central point or into a hyperbolic shape represents negative curvature. We can view the move from vague to fuzzy sets, as the move from a curved space to a flat space, and further, we could imagine movement past a flat space to a negatively curved space. Positive curvature would represent vagueness. As curvature goes to positive infinity, the system would be maximally vague, all elements superposed and indistinguishable. As curvature approaches 0, classic fuzzy sets would emerge, and at their vertices, classic crisp sets. What happens when curvature goes negative? I can only speculate. Perhaps, it is like the break-up of a power-set (all possible combinations of truth values), so that two elements could no longer be brought into any relation of comparison. At the extreme of negative curvature, perhaps each element becomes its single-element set, and thus its own power set, with every other element excluded. The imagery is powerful, since it represents a continuum from absolute vagueness to absolute distinction, with the world we are familiar with sitting comfortably in the middle ground. Often much of what we measure or observe appears distinct, and natural groupings emerge over time that we may call sets. But even then, there is some degree of indeterminateness; our elements are not quite points in a sets-as-points space, they are a little smudged. I hope in the near future to be able to demonstrate this mathematically, rather than metaphorically. The operationalist definition of a vague set will have to do for the interim. By defining the set as emerging out of the elements, in a process of specification, the generation of distinguishable features over the course of time, we have arrived at a notion of vagueness that has some deep symmetry with the observations of embryologists witnessing the emergence of species specific features in developmental series of embryos. Our sets are not abstract, but abstracted from countless empirical investigations. We have approached vagueness slowly, in stages, working forward from our ability or inability to make distinctions, and relied heavily on visual imagery and some geometric reasoning. Ultimately, if these proposals are convincing, they see the "set" as not a static timeless abstract concept, but here interpreted as a grouping on elements, and so the 37
Developmental Systems concept "element" is primitive (and itself must be distinguished from the background). The uncertainly we are dealing with here, is not probabilistic in nature where uncertainty is simply a ratio in a combinatorial space of binary distinctions, nor is it the same as fuzziness which still requires defined categories, but a primal uncertainty, perhaps better called "indeterminateness", prior to the category. If set theory is ground zero in mathematics, that ground rests upon vagueness. Peirce (1998, p.174) from his analysis across logical systems seems to speak of a similar indeterminateness, or what he called "pure chance": At any time, however, an element of pure chance survives, and will remain until the world becomes an absolutely perfect, rational, and symmetrical system, in which the mind is at last crystallized in the infinitely distant future. That idea has been worked out by me with elaboration. It accounts for the main features of the universe as we know it -- the character of time, space, matter, force, gravitation, electricity, etc. It predicts many more things that new observations alone can bring to the test. May some future student go over this ground again, and have the leisure to give his results to the world.
38
Developmental Systems
Developmental Systems "He who cannot draw on three thousand years is living from hand to mouth." -- Goethe
This section relies heavily on Stan Salthe's "Development and Evolution" (1993) for its central image of a systems progress from immature to mature to senescent whereby the system moves materially from the non-mechanical to the mechanical to the dead nonliving material. Salthe's goals are significantly wider than my own. He seeks to explore "evolution" and "development" in contra-distinction to each other, and how they have been yin-and-yang themes in our construction of a theoretical biology. My goal is to simply understand the logical consequences of the patterns I observe as a developmental and field biologist, and to identify the components of a model sufficient to generate such patterns, which I cal the measurement relation, Probe-System-Trace. Since my choice of terminology and notation differs from Stan's, I will add some tie-ins to his terminology where appropriate, and also provide some links to other literature of interest, but beyond the scope of this essay. We have had very difficult road to hew so far. The mathematician is justified in questioning why I have dawdled so long in mere numerics; the developmental biologist is equally astute to ask what this all has to do with her. The answer is, I had to look at the mathematical tools we take for granted, from the perspective of the data a biologist gathers, to make clear the holes in those tools. All of classical statistics, is based on functions, the mean and the variance, are derivatives of functions. We have to understand the nature of traces that can not compress to simple functions, to fully understand the challenge of the relational data we collect in biological studies, and to begin to look beyond dynamics for a generalization sufficient to encompass developmental data specifically; data from complex relational, scaleheterogeneous systems more generally. Based on our analysis of traces, we can reasonably deduce from the individual cases and examples some general themes, which are hardly revelations to any developmental 39
Developmental Systems biologist: the data we gather, the observations we make, seem to be scale dependant; forms change over time; things do not fit neatly into categories or sets; history is marked into the system (into chromosomes, into tree rings, into bone) irreversibly. In effect developmental systems write back into themselves, thus becoming closed causal loops to some degree. I propose the following: A developmental system is1: 1. Relational. For a given X there is a restricted codomain of Ys, and a propensity for each X to the various Ys. When a system is so tightly constrained that for every X, a single Y is all that is available as the consequent, the relation reduces to the special case, a function. 2. Scale dependent in its dynamics, and requires causal explanation simultaneously at multiple scales. This scale dependence is referred to as "the scalar hierarchy" in Salthe's terminology, and denoted by the symbol "Hsc", and results in long range temporal and spatial correlations among the components of a developing system. 3. Increasingly specified as a function of time, but inherently vague at certain levels. The movement from vagueness to specification is equivalent to "the specification hierarchy" in Salthe's terminology, and denoted by the symbol "Hsp". The result of specification is a hierarchy of natural laws governing a system, with more general laws representing constraints that occurred earlier in a system's development, and more specific laws representing constraints that occurred later in development. 4. Encapsulated to various levels, depending on the functional relations it is capable of entering into in terms of the Probe-System-Trace model.
1
In reviewing an earlier draft of this essay, Stan Salthe has pointed out to me that the terms "developmental
system" and "developmental systems theory" have previously appeared in the literature, centred around a philosphical critique and deconstruction of genetic determinism in neodarwinian theory. Oyama's book "The Ontogeny of Information" (2000) is the seminal work, and has been recently reissued and updated. My use of developmental system in this essay has a different origin, and extends the notion of dynamical systems to deal with the patterns found in developmental studies, hence "developmental systems". I include some preliminary thoughts on her very interesting work in the postscript to this essay.
40
Developmental Systems 5. Has multiple attractors, and moves across attractors with time. Form changes. 6. Self-acting. Systems write into themselves in various ways, that is they create and modify traces internally. Systems act upon themselves, have causal closure, selfentail. The formal consequences of this are developed in much more detail in Rosen (1985, 1991), and the biological consequences of this are discussed in Ulanowicz (1997) in terms of a simple model of this "self-acting" as autocatalytic cycles which he uses to ground his model of ecosystem development. 7. Local in its interactions. The measurement relation, both applied externally, and applied internally, creates the requirement for a process of recognition, measurement, and synchronization. Local interactions give rise to global dynamics. The local process of recognition and synchronization among parts of a system has been labeled "internal time" by Matsuno (1997, 1998), and will be dealt with in more detail later in this essay. This is an abstract conceptual model, anything that hews to this model -- a mathematical model, a linguistic theory, a data analysis algorithm -- can be called a "Developmental System Model" in the context of this program. It abstracts features we observe in real developing systems. This model recovers dynamical systems under certain conditions: one-to one relations only, scale invariance, single attractors. The model is a way of exploring the patterns we find in traces, which may or may not correspond to the model to varying degrees. Just as lack of correspondence between wild traces and traces from functions led me to identify limits to the conceptual model underlying a dynamical system, lack of correspondence to the model developed here, moves us to look for deeper generalizations. In some sense, it is not only traces all the way down, it is also models much of the way down, until a model smashes into a trace it can not encompass, and is delimited by this failure. The irritation caused by such failure is the ultimate source of new models in science. A fundamental article of faith common to scientists and artists is that the myriad details of the observed natural world are compressible into forms, whether the form is an equation, or a line drawing of the profile of a face. What remains in doubt is the level of 41
Developmental Systems compression attainable. At the level of full compressibility, a natural system is reducible to a set of equations, and formal analysis of those equations is sufficient to specify the complete behavior. At the extreme of incompressibility, the "trace" as it were, stands on its own. We can make no compression greater than the observed results. It is in the intermediate zone that developmental systems exist, their features are regular, but not completely specified. Their behaviors are open, the traces they leave, often surprising. Developmental systems are highly constrained, they are also, in some sense, relatively free, They are not fully determined, and thus admit the possibility of choice as existing between the extremes of absolute order and absolute chaos. There's a lot of room between order and chaos, and thus developmental systems have many ways to individuate. Following Salthe in Development and Evolution (1993:29-32), I make a distinction between development and evolution. Both involve irreversible changes with time. However, the underlying model for evolution is statistical mechanical, and the system is in principle reversible at the microscopic level (level of sub-components of a system), though such reversal is improbable. At the level of "state", the maps from state to state of a model of an evolving system are reversible. If one actually knew the microstates, one could in theory run the system backwards. Furthermore, the system has no memory, collisions between molecules wipe out long term histories, which is what allows us to treat all particles in the system as "identical" -- they are not marked The irreversibility in an evolutionary model comes from the idea that for a given macrostate (state of the whole system), there are a number of compatible microstates (states of the parts). That is, there is a one-to-many relationship on microstate groupings relative to the macrostate. For a developmental system, the irreversibility is fundamental, and rises from several sources: 1. At the level of mathematical representation, from the fact that the relations we are positing between an object at subsequent points in time will not commute, hence we can not reverse the order, and recover the original system. 2. At the level of the phenomena, the system, first marks itself, and thus has a memory (recall that "particles" in statistical mechanics have no memory). Secondly, each 42
Developmental Systems subsequent stage in a developmental sequence depends on the organization of the prior stage, but the movement into the next change, entails a change in the organization. Return to the previous state is lost. And development individuates the components. "Particles" if they could be called such, can no longer be treated as identical; rather they are now individuals. Two examples should help make the distinction clear. The "grey" colour of the moth Biston betularia (the hardest working moth neodarwinian theory) can revert to white, when environmental conditions change. Indeed the colour changes from grey-to-white could fluctuate over periods of time, tracking the environment. "The Beak of the Finch: A Story of Evolution in Our Time" by Jonathon Weiner (1994) is a beautifully written popularization of the work of Peter and Rosemary Grant, that illustrates such reversible fluctuations, as the beaks of Darwin's finches track boom and bust cycles in their sustenance following wet/dry cycles in the environment. Given that the core models and the data collected in evolutionary studies is reversible, where does irreversibility come in? Irreversibility of evolution is gained by positing that all the relevant environmental values selecting on genotypes would not revert to a previous state simultaneously. Consider now a classic developmental example, cellular differentiation. Once a cell in the cambium of a tree has changed to either a xylem or phloem, environmental changes can not cause it to revert back the undifferentiated cambial state, or to xylem. The development is irreversible. Unfortunately, I can't currently think of a beautifully written popularization of development -- and that's part of the problem -- development has not been deeply incorporated into the neodarwinian synthesis, or even the neo-neodarwinian synthesis, and is not deeply understood as a general phenomena, of which ontogeny is the biological example. The problem is waiting for its moment. By the distinctions made here between evolution and development, population genetics, the study of changes in gene frequencies, is an evolutionary theory; but phylogenetic systematics, the study of irreversible change in species lineages, is a developmental
43
Developmental Systems theory. That is, the origin of new species, is a developmental problem. Change in species, is an evolutionary problem. The interplay of specification and scale can be seen to result in the multiple attractors in development, as a system moves through the various stages from immaturity to senescence. Consider the development of a temperate forest, in an area after fire. Initially, the vegetation interacts across small scales, and the dynamics of an individual plant largely depend on its microsite environment, which includes its immediately neighboring vegetation. As the tree layer reaches past the herbaceous vegetation, a continuous canopy is created. This canopy provides a "boundary" between the forest and what is outside the forest (and I should note, that the root system below creates a second canopy and boundary, but one we know much less about). The canopy can be said to act as a cohesive element in creating a forest out of a group of trees. A boundary layer is created that buffers what is beneath the canopy from the environment without. Light, moisture, nutrients are modulated. The presence of a canopy, now acts as a substrate which "affords" (Gibson, 1986) new habitats for other organisms. That is, at the moment a canopy emerges, the system moves to a new attractor, from that prior to canopy closure. New combinatorial possibilities are available to the system. As the forest moves towards maturity, it is increasingly sub-structured into functional components. The soil is modified by leachates from the vegetation, a community of microorganisms develops, tree roots are linked into fungal systems, animal migratory pathways are marked into vegetation, new functional components are created as a forest soil and litter layer develop. Nutrient cycling begins to be confined largely to this upper soil layer, making the forest both more efficient and also increasingly dependant on the continued health of the litter and modified soil layers for the health of the forest as a whole. Because of interactions among processes at different time scales (e.g. canopy dynamics vs. soil dynamics, vs. soil microfauna dynamics) which impinge upon and constrain each other (the soil dynamics alters substantially following establishment of a canopy, which in turn modifies microfauna dynamics, which in turn modifies soil dynamics, which in turn modifies the nutrient conditions for the forest understory, which in turn ……) leads to the formation of long range temporal correlations between processes, which may seem to act 44
Developmental Systems independent of each other in terms of their short scale dynamics. Such long range across scale dynamics when viewed in physical systems (usually as spatial correlations) are called "macroscopic correlations", and will occur in developing systems both temporally and spatially, and represent the cohesive synchronization of system components. As the forest ages, more and more of it is caught in fixed biomass, and less of it, in flows of material. It is this latter stage that is called senescence. The standing biomass does provide further habitat for certain animals and insects, but the level of growth and change in the forest has leveled off. Finally, we have the breakdown of the canopy, which gave cohesion to the forest as a whole. The process begins again with the next fire. Through the life of a forest, we begin to see that as it individuates into a particular forest, it also begins to have processes developing in it at multiple scales, and as each process develops, it begins to act as a substrate for new processes, and combines with existing processes in new forms. Furthermore, it is the links across scales, that provide much of the cohesion, the fact that the leachate from trees that follows a rainstorm, ultimately modifies the soil, which provides a habitat for specific soil organisms, depending on the nature of the leached material (an acid soil in the case of conifers, a more basic soil in the case of hardwoods). What I have described is a generic forest, and only one developmental possibility for a forest. But it is a good example of the process from immaturity to senescence at a level higher than the level of the individual organism, where, as individuals ourselves, we are intimately aware of this movement from immaturity to maturity to senescence. The attractors are not created by "external forces" but by the emergence, development, and ultimate loss of cohesive properties in the system, that allow components of the system to communicate at multiple scales. As a truly wonderful example of cohesiveness, consider a forest stand that is also an individual -- a clonal aspen forest, where one can mark the movement across one individual and into another by such subtle changes as foliage colour in the fall. In this case, neighboring patches of a forest stand can be seen to individuate, because of differences in the clonal individual, in terms of resistance to pathogens, mycorrhizal associations, palatability of foliage to herbivores, etc., that change the nature of the 45
Developmental Systems affordances, and hence, the ultimate developmental trajectory. Whenever you wonder why something in biology seems so complex, "surface area" is often a good answer, as is "boundaries". Development results in new surfaces, new boundaries, new attractors as a system moves from immature, to mature to senescent. Often, it is the creation of new cohesive structures in development that "marks time" in the system, creating a notion of internal time, based on the cohesive relationships within a developing system. A good example of this is the formation of basal rosettes in many plant species. A common species that does this is Arabadopsis thaliana. One can vary the "external time" to formation of a basal rosette, by controlling temperature, nutrients, light. But, only after a basal rosette is formed, does the plant send up a main shoot, and develop reproductive parts. Hence, the internal sequence of stages, the internal time, is a better predictor of the future events, than any externally imposed time, and the regular behavior of the plant can be seen as more a function of internal time, than an externally imposed clock time of hours, days, weeks, months (which is just the internal time of our solar system). And indeed, if you look across species, you will note species that are very distantly related, e.g. Arabadopsis thaliana (a member of the mustard family) and Balsamorhiza hookeri (a member of the aster family), which have very different growth dynamics, Balsamorhiza taking years to do what Arabadopsis does in weeks, with isometric patterns, when everything is scaled to measures of internal time. This concept, that one stage in development, creates the enabling conditions (the "affordances") for a subsequent stage has some interesting consequences. First of all, as cohesive entities emerge in a developing system, new properties are introduced, and new law like rules come to govern system dynamics. However, before the new properties come into being, we can not predict their existence from the previous properties, because our models are necessarily incomplete with respect to the new properties, and the system itself has not specified the properties. The properties arise out of interactions among the system components, and the interactions themselves may be occurring at different scales; the interactions to create a forest canopy occurring in say 20 years, the interactions to create a stable surface soil structure in say a 100 years. Returning to embryo 46
Developmental Systems development, we know what happens after the tetrad or blastula or gastrula stages in embryo development, not through prediction based on the measurable properties of an embryo at that stage, but because we have seen the process before. Thus, as new cohesive levels are created in a developing system, they create both new law-like behaviors, and also create greater diversity of behaviors. Prior to a cohesive level forming, and the source of cohesion achieving closure (e.g. canopy closure in case of a forest), the system must remain vague with respect to these new properties -- they have yet to be specified. There is a consequence of a developmental system as a network of traces being read at different temporal and spatial scales of the subcomponents, with the readings themselves creating the coordination and cohesiveness of the system. Lemke (1991) cited in Salthe (1993) expresses this consequence beautifully, this "rhythmic entrainment" by which a system's identity exists not in the frozen snapshot, but in the motion picture: The system as an individual entity, cannot be defined at one moment in time because the dynamics which maintain it in being must occur over time. In each instant the system is dead; only over time is it alive. So much is true for any dissipative structure. … Only the system extended in time along its complete developmental trajectory, from formation to disintegration, from conception to decomposition, is a properly defined theoretical entity. …. the caterpillar-pupabutterfly is one individual developing system …" We should look at what this notion of developmental systems means in terms of the classic statement "Ontogeny recapitulates phylogeny". This can be taken to mean, that the ontogenetic sequence is a-priori-determined by phylogeny, that is it is genetically predetermined, and an ontogeny is simply an organism running its program (which we associate with DNA). However, we know that there is not a one-to-one mapping between genotype specification and developmental specification. One need only to look at the different fates of identical twins. Another example is cases where an individual is developmentally female, but genetically male. Such cases are rare, but what they indicate is that development is a contingent activity. Genetic information, the traces recorded physically in the system, are its memory, and the system must continuously interpret that memory in the context of local conditions. While we do not expect a person who is
47
Developmental Systems genetically human to spontaneously revert to a common ancestor, we must recall that we share over 90% of our genome with our nearest common ancestors the primates (and share over 98% genetic similarity with chimpanzees). Another situation to consider is inter-specific hybridization, between fairly distantly related species. Here the organism, in development must interpret two sets of blueprints or scores, and its ability to survive, depends on whether it can do so (Datta et al., 1960). These latter thoughts are speculative, but they are a natural consequence of the relational set-ups we have developed between probes, object systems and traces. If we go to the medical literature of developmental abnormalities, we will indeed find some support for the idea that what we call genotype, acts merely as a trace (actually, many traces), that must be dynamically interpreted by the organism in situ, and that act of interpretation itself can act as the trace for the next act of interpretation, and so on. An organism in development can wander away from its genotype, in a manner similar to a musician wandering away from the score. What science creates out of the factual, art transcreates into the mythical. Based on the concepts developed in this essay, what would a developmental mythology look like? How would we render developmental systems through a poet's narration rather than a scientist's methods and materials section? The notion of vagueness as a non-Euclidean geometry appeals to me, first because it provides a scheme for synthesis of developmental progression from immature to mature to senescent in terms of geometric transformations, and secondly because it leads to lovely imagery. When one tries to visualize a system moving from immature-vague to mature-fuzzy to senescent-crisp and beyond into the hyperbolic, one is taken to the edge of the imagination. Perhaps a developmental mythology would look like this: In the beginning was Vagueness. No inside, no outside, a perfect symmetry of nothingness and everything, all potentiality. Then symmetry broke, and existence was cut into is and is-not. Is and Is-Not fractured into This and That, which fractured into ….. well very soon you had a whole family of original distinctions, with varying relations between themselves. Out of distinction information came into being, and when two 48
Developmental Systems distinctions met or collided, information passed between them giving birth to further distinctions, the transfer of energy and generalized field theories. Symmetry broke and broke and broke, creating the fundamental laws, the original particles, the enumerable and non-enumerable things. Each time symmetry broke, matter got more and more complex, more particular, more local. Little bits of existence began to differentiate from other bits of existence The early symmetry shatterings becoming cosmological constants, the latter symmetry breakages becoming synampomorphies setting the pattern of bones in your right hand. At some point, the life that would become you arose on the distinct little place that is your world. In time, the living matter grew sentient, asked questions, wondered if things would keep getting more complex and complex, more fine grained and local. Would existence stabilize, so that the order of today was the order of tomorrow, the order of an eternity of tomorrows? Would existence reverse itself, un-symmetry breaking, the pattern of bones in your right hand, unworking their way back to cartilage, and finally fusing with the pattern in a distant common ancestor. Or would the symmetry breaking continue unabated until some day everything would get so distinct and local, that nothing in existence could touch anything else, that the connectives in logic would cease to exist, the physical laws that connect things from one time to the next, one place to the next would cease to exist, and existence itself would grow senescent, isolated, perfectly frozen, every last distinction done? Little bits of distinction would wonder such things, sometimes musingly, sometimes self-aggrandizingly, sometimes worriedly. Distinctions would look out at the stars at night, and wonder if other distinctions from somewhere in the great unknown were looking back? They would look at the bones in their right hand, curl and uncurl the fingers, imagine the same hand before it was a hand, when they were embryos, and they would wonder what happened to the vagueness.
49
Developmental Systems
Complexly Organized Dynamical Systems MatterFlowsThroughForm
Having developed a notion of a developmental system, my next task is to provide further context. I need to first ground an abstract notion of developmental systems and the ProbeSystem-Trace relational model within the constraints we observe in real developing systems, and I need further to identify where such systems should apply. To do so, I turn to the work of Collier, Hooker and Christensen, and their development of an account of "Complexly Organized Dynamical Systems" (Christensen et al.1997; Collier and Hooker, 1999; Collier, 1999). It is within the context provided by their account, that I will introduce some further restrictions on developmental systems, place developmental systems in the wider frame of complexly organized dynamical systems, and examine how developmental systems as a general notion apply to living systems as a specific manifestation. Having spent the greater part of this essay pounding our way through the underbrush of traces, ascending slowly and steadily, we now arrive at steeper slope, preceding a clearing. Once we have ascended this section we can more clearly see the landscape we have just passed through, and also the landscape that lies ahead of us. At the end of this section, please feel free to pause, take some sustenance, and have a breath of fresh air. To ground developmental systems so that they descend from the abstract to the concrete, I need to summarize two concepts developed by Collier and Co. in the context of developing a framework for analyzing complexly organized dynamical systems. The first is the notion of "cohesion", which we have been implicitly using throughout this essay. The second is a heuristic device that places real world constraints on any physical system (i.e. a "real" versus a "formal" system), that is called the Negentropy Principle of Information (NPI). It is within the confines of NPI that I will introduce "information", which like cohesion has remained implicit in this essay thus far. Both concepts -cohesion and NPI -- are needed to bridge an abstract notion of developmental systems, that could support a number of alternative mathematical formulations, including those 50
Developmental Systems providing unphysical results, to a concrete formulation of developmental systems that will not lead to unphysical results, and hence is applicable to the real developmental systems we can empirically observe, measure, study. The fundamental problem that cohesion addresses is the problem of unity, that is to say, "these parts, are parts of a unified whole". Rosen, citing Rashevsky in "Life Itself" (1991, p.111) emphatically brings into focus the problem of unity that must be addressed by any account of complex systems: But by 1950 Rashevsky was growing uneasy. He had asked himself the basic question "What is life?" and approached it from a viewpoint tacitly as reductionistic as any of today's molecular biologists. The trouble was that, by dealing with individual functions of organisms, and capturing these aspects in separate models and formalisms, he had somehow lost the organisms themselves and could not get them back. As he himself said, some years later: It is important to know how pressure waves are reflected in blood vessels. It is important to know that diffusion drag forces may produce cell division. It is important to have a mathematical theory of complicated neural networks. But nothing so far in these theories indicates that the proper functioning of arteries and veins is essential for the normal course of the intra-cellular processes; nor does anything in those theories indicate that a complex phenomenon in the central nervous system ….. (is) tied up with metabolic processes of other cells in the organism. … And yet this integrated activity of the organism is probably the most essential manifestation of life. So far as the theories mentioned above are concerned, we may just as well treat, indeed do treat, the effects of diffusion and drag forces as a peculiar diffusion problem in a rather specialized physical system, and we do treat the problems of circulation as special hydrodynamic problems. The fundamental manifestation of life mentioned above drop out from all our theories in mathematical biology. … As we have seen, a direct application of the physical principles used in mathematical models of biological phenomena, for the purpose of building a theory of life … is not likely to be fruitful. We must look for a principle which connects the different physical phenomena involved and expresses the biological unity of the organism and of the organic world as a whole.
I have italicized the last sentence from Rashevsky, to emphasize his intuition that the "missing principle" concerned a unity relation. Rosen's approach to moving forward into 51
Developmental Systems a relational biology was to identify the "component" and its function as the atomic unit of organization with respect to a biological whole. Rosen introduces his formal notion of a component thusly (1991:pg 120; emphasis Rosen's): The component may be thought of as a particle of function; it plays the same kind of role in relational modeling that particles play in reductionistic or Newtonian modeling. Just as in the case of particles, components for us will be the basic analytical units into which natural systems are resolved. With a further elucidation of its properties (1991, p. 121, emphasis Rosen's) Thus what we call a component must be endowed with the following properties: (1) it must possess enough "identity" to be considered a thing in itself, and (2) there must be enough room for it to acquire properties from larger systems to which it may belong. That is: the formal description of a component in itself as a thing; the other part must be contingent on such a larger system. It is this latter part that specifically pertains to the function of the component. Rosen's definition raises an obvious question: how does one identify a component, and what is the level of interaction between a system and its component, that makes it a component of that system, and no other system? To answer this, we must make explicit something implicit in Rosen's notion of the component, and that is cohesion. We will find that cohesion, like a ghost in our thoughts, has been with us throughout our conception of "systems", appearing in various forms. Collier (1988, p.210) introduces cohesion as part of an argument as to why the properties of whole biological systems can not be usefully reduced to the properties of their parts. Collier's conception of cohesion looks at system organization through its origin in dynamical interactions:
52
Developmental Systems Roughly, a system is cohesive if there are causal interactions among its parts that make it insensitive to fluctuations in the properties of its lower level components. When a system is cohesive, these fluctuations are irrelevant to its state description, and it is both pointless and misleading to describe the system in terms of the properties of its lower level components. Christensen, Collier and Hooker (1997, p. 10-11) sharpen the notion of cohesion and its dynamical basis: Systems are collections of dynamically related elements of some kind. To individuate dynamical systems we need to specify the dynamical relations between components of a system that make them components of the same system, and distinguish them from other systems and components of other systems (cf. organs vis-à-vis component cells and super-system bodies). There are two aspects that need to be kept in interrelated balance: an individual is something more than merely a stable pattern, but neither are all individuals closed, statically stable systems. The first aspect requires providing a sense of belonging together, an interactive "gluing" that binds a system together as an individual unity, while the second aspect requires a sense of interactive organization that allows systems to be open and adaptive, exchanging energy and materials with their environments and being altered by those interactions, yet remaining an individual system They then relate cohesion to other system level properties (1997, p. 15): The main role of cohesion in the dynamical characterization of systems is that it is the basic principle of division for dynamical systems and their dynamical properties. Systems stability, identity, differentiation and evolution are to be understood in terms of making and breaking cohesion. Furthermore, the dynamical interpretation of functional properties will be constrained by cohesive closure.
53
Developmental Systems Cohesion, is essentially a relation that produces closure in a system, such that it is identifiably separable from all other systems it could be seen to be part of. As such, it is both multi-dimensional (different dynamical variables may contribute to cohesion) and also a matter of degree (there may be different interaction strengths). The nature of the closure, is system dependent. But the effect is uniform. Essentially cohesion encapsulates a system, and it is the encapsulated system that can play the role of "functional component" in an organized whole (a system with various subsystems, the components). Cohesion, also establishes a scale of interaction between one cohesive system, and other cohesive systems (because the "component" may be a system in its own right); it allows the system to act in a concerted manner to the effect of forces (originating both externally, and internally), and to average the forces impinging on it. Cohesion serves thus to individuate a system, and is "the glue that separates," as it has been called by Cliff Hooker. In individuating a system, and allowing it to operate on a scale different from its subcomponents, cohesion also results in what is called "emergence", the establishment of new system level properties not contained in the components themselves, nor directly deducible from a dynamical description of the components. Cohesion is essentially an organizational concept, but one with implications that are both material and dynamical. Effectively, cohesion is required for a probe to recognize a system, and cohesion is required for a carrier to mark a probe with a trace. Without cohesion, the Probe-System-Trace relation could not exist. In this sense, it is at the very heart of the measurement relation identified with Probe-System-Trace. Only cohesive systems can be marked, and retain a mark of sufficient duration to create a trace that can be read by other (probe) systems. Without cohesion, there is neither recognition, nor is there memory of an interaction. An example of cohesion, unifying a system of diverse components are the long range correlations that I described as coming into place in the development of a forest ecosystem. Canopy closure, which changes the dynamics of a stand of trees, is not the property of any single tree, but of the stand at the moment a canopy bounds the stand by providing a boundary layer. Following canopy closure new stand level properties emerge, 54
Developmental Systems because the boundary layer of the canopy buffers the system so that it averages external forces (such as wind), and internally re-organizes the environment now encapsulated by the stand: biota, soil, etc. I hope that pragmatic example helps to place cohesion in an intuitive context. Cohesion is a subtle concept, because we use it innately. The most difficult concepts to think about, are the ones we can use without consciously thinking about them -- the concepts that are part of our unconscious repertoire. I believe cohesion to be such a concept. Think about cohesion the next time your recognize something, as that thing. How did you do it, what makes it that thing? Cohesion is the unity between an immature embryo system, its senescent finale, and all points between. It makes it the same system. And it makes it, that specific individual. Now, we turn to the second concept that has been left implicit in this essay, "information". The Probe-Trace-System model, is essentially a model of information transfer. Information can be treated as a purely formal quantity. A mathematical bookkeeping device applied to symbols, irrespective of their meaning. This formal notion of information, does not help us to ground developmental systems in the physical world we work in. Rather, I am interested in the notion of physical information (Collier, 1986), as the constraints upon a system that allow it to do useful work, by giving it internal structure. The Probe-System-Trace model, is a set of roles, looking for actors. One actor, the Probe, gains information about a second actor, the System via the agency of a Carrier, resulting in a Trace, which is physically marked on a substrate, and has duration long enough to be read (once at least) by the Probe. Those actors, must for empirical purposes be real systems, and therefore we need to have a way to conceptualize their actual energetic and informational constraints operating on real physical systems. The "Negentropy Principle of Information" (NPI) is a way to do this. I will summarize John Collier's account of NPI from the paper, "Causation is the Transfer of Information" (1999, p.11-18) in which it is most thoroughly developed to date. Then I will place the concepts into the framework of the Probe-Trace-System model of measurement, so we may gain some intuitive depth in our use of this model. 55
Developmental Systems
Collier's account of the Negentropy Principle of Information (NPI): To connect information theory to physical causation, it is useful to define the notions of order and disorder in a system in terms of informational complexity. The concept of disorder is connected to the concept of entropy, which has its origins in thermodynamics, but is now largely explained via statistical mechanics. The statistical notion of entropy has allowed the extension of the concept in a number of directions, directions that don't always sit happily with each other. In particular, the entropy in mathematical communications theory (Shannon and Weaver, 1949), identified with information, should not be confused with physical entropy (though they are not completely unrelated). Incompatibilities between formal mathematical conceptions of entropy and the thermodynamic entropy of physics have the potential to cause much confusion over what applications of the concepts of entropy and information are proper (e.g. Wicken, 1987; Brooks et al., 1986). To prevent such problems I adopt the interpretive heuristic known as NPI, according to which the information in a specific state of a physical system is a measure of the capacity of the system in that state to do work (Schrodinger, 1944; Brillouin, 1962, p. 153), where work is defined as the application of a force in a specific direction, through a specific distance. Work capacity is the ability to control some physical process, and thus closely connected conceptually to causation, but it is a state variable of a system, and involves no external relations, especially to effects, so the concept of work capacity is not explicitly causal (though work is). Through the connection with work, NPI ties information, and hence complexity and order, to dynamics. NPI implies that physical information (Brillouin, 1962) has the opposite sign to physical entropy, and represents the difference between the maximal possible entropy of the system (its entropy at equilibrium with its environment, assumed otherwise unchanged, after all constraints internal to the system have been removed, and the actual entropy, i.e. 56
Developmental Systems
NPI
Ip = HMAX constant E,constant C - HACT constant E, constant C
Where E is the environment of the system, and C is the set of constraints on the system. The actual entropy, HACT, is a specific physical value that can in principle be measured directly (Atkins, 1994), while the maximal entropy, HMAX, of the system is also unique, since it is a fundamental theorem of thermodynamics that the order of removal of constraints does not affect the value of the state variables at equilibrium (Kestin, 1968). This implies that the equilibrium state contains no trace of the history of the system, but is determined entirely by synchronic boundary conditions. Physical information, then, is a unique measure of the amount of form, order or regularity in a physical system. Its value is non-zero only if the system is not at equilibrium. It is important to remember that NPI is a heuristic physical principle, not a formal or operational definition, and given the current proliferation of formalisms for entropy and information, it needs to be interpreted as appropriate for a given formalism and for a given physical system and its environment. Essentially, physical information is the degree of constraint in a system. And that constraint represents the structure of the system that allows it to do work. In terms of the Probe-System-Trace model, a probe must be structured so that it can make a measurement on an object system. To make the measurement, the probe must expend energy. Thus information is gained via the degradation of energy. This can only happen, if the probe is in a far-from equilibrium state. At equilibrium, a probe can not measure. For a trace to be marked, a difference must occur, which is not averaged out, hence, a trace also requires non-equilibrium conditions to be informative. Collier goes on to break down the physical information, Ip, into two components, which he calls "intropy" and "enformation":
57
Developmental Systems The relations between information and energetic work capacity are somewhat subtle, since they involve the correct application of NPI, which is not yet a canonical part of physics. The physical information in a given system state, its capacity to do work, breaks into two components, one of which is not constrained by the state of the system, and one which is. The former, called intropy, i, is defined by ∆i= ∆(exergy)/T, so that σT∆i measures the available energy to do work, while the latter, called enformation, e, measures the structural constraints internal to the system that can guide energy to do work (Collier, 1990a). Enformation determines the additional energy that would be obtained in a system S if all cohesive constraints on S were released. Intropy measures the ordered energy, which is not controlled by cohesive system, processes, i.e. by system laws, it is unconstrained and hence free to do work. For this reason, though ordered, both intropy and exergy are system statistical properties in this sense: their condition cannot be computed from the cohesive or constrained system state, the cohesive state information determines the micro state underlying intropy only up to an ensemble of intropy-equivalent micro-states. There is another system statistical quantity, entropy, S, but it is completely disordered or random, it cannot be finitely computed from any finite system information. Entropy is expressed by equiprobable states, hence appears as heat and has no capacity to do work; ∆S = ∆Q/T, where Q is heat, and σT∆S measures heat. Enformation is required for work to be accomplished, since unguided energy cannot do work. Intropy is required in dissipative systems, to balance dissipation (S production). I will briefly define some terms that might not be familiar. Exergy represents free energy, i.e. energy available to do work. Microstates represent arrangements of a systems subcomponents. Macrostates represent a measure made on a system -- temperature, pressure, etc. for classic thermodynamic systems. For a given macrostate (say temperature), there are many compatible microstates (i.e. arrangements of the component particles with the same temperature for the system as a whole). Thus, there is a many-to-one relationship between microstates and macrostates. For a given macrostate that we can measure -- we often have no way of distinguishing which of the microstates the system is in, that 58
Developmental Systems actually lead to it. The macrostate that has the largest number of compatible microstates is called the equilibrium state. Now, we usually think of these ideas in terms of physical particles, but there is no reason why we can not think of microstate and macrostate, in more general terms for any system where system and components can be distinguished. Such a viewpoint is particularly useful when developing models for hierarchically structured systems in biology. The important concept is that there is a grouping principle, to identify many microstates, with particular macrostates. The above statement from Collier's article represents an exceptionally compressed account of non-equilibrium thermodynamics. For those who are a bit dizzy after reading that passage, followed by my own fairly compact description of microstate/macrostate, let us put these concepts to work, in terms of the Probe-System-Trace model, and gain some intuition. A Probe reads a Trace. To do so, the probe must have structure that allows it to interact with the trace. This is enformation. The probe has a capacity to change its internal state, based on the trace. This is intropy. It assumes a stable and out of equilibrium internal structure, so that changes in structure can be distinguished. To read a trace, a probe must use up energy. In doing so, some heat is generated. Let us take a more specific example of reading: consider the eye, registering a trace. We'll restrict ourselves to the layer of rod and cone cells, and discuss monochromatic vision only. In this case, enformation represents the actual structure, and arrangements of the rod and cone cells, and the internal structure of the cells (organelle's, etc.) that support visual activity. All this structure represents enformation. A photon of light hits a rod cell, and the pigment rhodopsin is bleached to produce retinine and opsin. In the chemical reaction energy is released and that energy eventually leads to the transmission of an impulse over nerves. The rhodopsin is reconstituted from retinine and opsin under the influence of retinene reductase and vitamin A, and requires energy to be available for the reaction. Thus the intropy is the degree to which the rhodopsin is maintained in the rod cell, under nonequilibrium conditions, so that work can be done when photons of light hit rod cells. At equilibrium, you are effectively blind, and there is no chemical gradient in the eye capable of reacting to light.
59
Developmental Systems Collier uses, a more technological example to illustrate the terms intropy and enformation, which I will use as a jumping off point to discuss the notion of states, with respect to enformation, and its biological significance. A steam engine has an intropy determined by the thermodynamic potential generated in its steam generator, due to the temperature and pressure differences between the generator and the condenser. Unless exergy is applied to the generator, the intropy drops as the engine does work, and the generator and condenser temperatures and pressures gradually equilibrate with each other. The enformation of the engine is its structural design, which guides the steam and the piston the steam pushes to do work. The design confines the steam in a regular way over time and place. If the engine rusts into unrecoverable waste, its enformation is completely gone (as is its intropy, which can no longer be contained), and it has become one with its supersystem, i.e. its surroundings. Such is life. You will notice that though we are speaking of very different systems, there is correspondence of analogy between my discussion of vision, and Collier's discussion of a steam engine. Let us, now radically simplify the image, and consider gas in a cylinder at equilibrium, i.e. an isolated system at equilibrium. Nothing interesting happens. Now, consider the ends of the cylinder to be moveable, so that we have only been looking at gas contained in a small part of the cylinder. The two ends slide back very rapidly. The cylinder ends slide back at a rate faster than the average velocity of the particles in the gas. A non-equilibrium condition ensues. It should be clear, from this example, that the cylinder with sliding ends is actually part of the enformation. Equilibrium assumes essentially constant enformation. As the enformation changes, non-equilibrium conditions are created, and hence intropy. What we have is processes occurring at different scales, and at different rates. This is the common occurrence in developmental systems. Furthermore, while the "microstates" are defined while enformation is constant (and this is what is assumed in statistical mechanical models), they lose definition if enformation is changing. This returns us to my initial statement, in describing the Probe60
Developmental Systems System-Trace model, that "state" is actually a scale dependant concept. Look at your hand. It appears fairly solid and stable. In actuality, you are the bounded locus of a range of processes, of various speeds, and multiple scales, all interacting coherently. Your hand sits there, perfectly still and composed. Variously coupled processes are spinning at a dizzying rate, to keep that still hand in place. The stillness is an illusion, because you are a cohesive system, and everything beneath your surface is effectively encapsulated. We can now sharpen our concepts. A physically realized developmental system is: 1. Cohesively separable from other systems, and composed of parts (components) that are cohesively separable from each other. 2. Satisfies the "Negentropy Principle of Information": a probe can not gain information without expending energy; a trace can not be made without expending energy; information can not be transferred without a transfer of energy. 3. Relational. The relations reflect propensities. Information theory provides a useful bookkeeping calculus to track propensities. 4. Scale dependant in its dynamics 5. Increasingly specified over time, but inherently vague at certain scales. 6. Encapsulated to various levels, depending on the functional relations it is capable of entering into in terms of the Probe-System-Trace model. 7. Has multiple attractors, and moves across attractors with time, i.e. form changes. The multiple attractors are a consequence of features 1,3,4 and 5. 8. "Self-acting", and hence self-entailing or causally looping. 9. Capable of developing internal time, out of cohesive interactions among subsystems. Globally time dependant behavior of the system originates from a local process of recognition and synchronization of the sub-components. This process of synchronization creates internal clocks within a system. Interaction with other systems often requires synchrony of internal clocks of the systems in interaction. Fortified by two new concepts -- cohesion and NPI --, we are now ready to survey the landscape we have crossed, and the landscape we must cross into. To describe this
61
Developmental Systems landscape, we will use a heuristic classification of systems developed in Collier and Hooker (1999). They begin by distinguishing complex systems from simple systems and organized systems from disorganized systems into four classes of system: simple-organized, simple disorganized, complex-disorganized, and complex-organized. This classification is expressed in the table below:
Organized
Class 3:
Class 4:
Constrained
Developmental Systems
multiparticle
Ascendent Systems
Wolfram Class II:
(Ulanowicz's window of
Periodic attractors
vitality) Non-Turing-computable
The Edge-Corners Of Chaos (Wolfram Class IV: universal computation)
Class 2: Statistically specified
Class 1:
Disorganized
Single particle
near or at equilibrium
Conservative,
Gases, fluids
decomposable,
Wolfram Class III:
multiparticle
Chaotic attractors
Wolfram Class I: Point attractors
Simple
Complex
Table 6. Classification of Systems
62
Developmental Systems While the classification is essentially heuristic, it captures some major properties of system, in terms of two dimensions -- complexity and organization -- where any particular system could be said to be a point on this "complexity-organization plane" (not to be confused with the complex plane of real and imaginary numbers) in a manner similar to our sets-as-points visualization utilized earlier. The Simple-to-Complex dimension can be considered a gradient that relates to the number of independent pieces of information required specifying a system. The more information required the more complex. The Disorganized-to-Organized dimension can be considered a gradient that relates the number, strength, and pattern of connections between system components. A disorganized system is one where all system sub-components are effectively independent. For an organized system, there are diverse interactions among components -- and hence organization could be seen as the degree of nonlinearity or symmetry breakage in a system. Three of these "quadrants of complexity" represent systems that were considered mathematically tractable, and hence well studied, in the physical sciences. One quadrant represents systems we have very little experience with, in terms of mathematical analysis, and hence these systems have not been well studied in the physical sciences. Indeed, these systems have classically been considered part of the biological sciences (itself, the bane of quantification, until the latter part of the last century and the development of the methodology of biometrics as a consequence of efforts to quantify Darwin's theory of evolution by natural selection). In Collier and Hooker's classification Simple-Disorganized systems are labeled Class 1. Such systems are exemplified by single particle, constrained multi-particle systems. They are specifiable in one or a few deterministic functions. Complex-Disorganized systems are labeled Class 2. They represent statistically specified equilibrium (or near equilibrium) systems such as gases and fluids. Such systems are specified on functions derived from probability distributions. Simple-Organized systems represent constrained multi-particle systems such as crystals. Again, such systems are specifiable by deterministic functions. These three systems could be considered the "standard operating 63
Developmental Systems range" for physical models of systems. They represent the subject area of mechanics and thermodynamics in physics (and I will argue below, also encompass "chaos" and "complexity" in terms of the nonlinear dynamical models of mechanical systems.). There is a fourth class of systems, which are Complex-Organized. Collier and Hooker assert that only class four systems have the capability of producing new organization through time, and have the capability of generating new internal structure. They further assert, that this capability is necessary to ground autonomy, adaptiveness and anticipation, which are the signature of the living. Physical examples of such systems are hurricanes, stream eddies, solar systems. But the exemplars of such systems are, of course, living systems. Obviously, developmental systems and the Probe-System-Trace model fall into Class 4: complexly organized dynamical systems. Let us consider the systems in terms of traces, and trace compressibility. Class 1 and Class 3 represent systems that lead to fully compressible traces, show scale invariance, and have identical subcomponents. Class 2 represents systems that lead to incompressible traces, show scale invariance, and have identical subcomponents. The scale invariance, and identical subcomponents are part of the dynamical systems models that are used to analyze such systems. Class 4 systems, represent those our enthusiastic algorithmizer confronted as "wild traces", systems which have heterogeneous components, and scale dependency. To put it simply, our mathematical model of dynamical systems excludes a large body of dynamical behavior. It is towards this large unexplained body of behavior that the notion of developmental systems is addressed, and towards it that we need to frame new mathematical models, with greater explanatory capability. To gain further insight into what Class 4 systems are, and are not, it is useful to overlay upon Collier and Hooker's classification, an earlier classification developed for discrete nonlinear dynamical systems by Wolfram (1994) and further investigated by Langton (1991). A few words about nonlinear dynamical systems models first. These are systems modeled by nonlinear functions, such that for "every change in X", there is not a "proportional change in Y". Such functions are the bread and butter of the physicist. Over the centuries, a large body of mathematics has developed to solve them. However, we can 64
Developmental Systems usually only analytically solve "well behaved" nonlinear equations. It was the study of "ill behaved" nonlinear equations that led to the development of fractal geometry, chaos theory, studies of finite automata, etc. Though such functions are not analytically tractable, they are computer-simulable (you can program the equations into a computer, and let it run, i.e. you can turn the equation into an algorithm). The advent of the computer has allowed us to explore such systems, and hence the outer edges of our mechanical models, leading to many new insights into physical systems; in particular that a system can be both deterministic and unpredictable. Using computer simulation to study physical systems has led to a very productive cross-fertilization of concepts between the physical and computer sciences. With respect to the biological sciences, the study of such functions, has led to two new fields, one called "chaos" and the other, closely related, called "complexity" or "complexly adapted systems". Essentially these works study complexity in the context of mechanical automata that can be used for computer simulation, and so the complexity referred to could be called "mechanical complexity", and relates to the general properties of such automata. Examples of such nonlinear function based models are nonlinear difference equations (May, 1976) such as the logistic equation, long used to model population growth; boolean automata (Kauffman, 1993), models of on-off switches, which can also be seen as simple models of genetic regulation; and finally cellular automata (Wolfram, 1994, Langton, 1991discrete rule based models of physical systems as finite state machines, which could be considered ") "programmable matter". Wolfram, used cellular automata to examine discrete dynamical systems in general, and identified four distinct "universality classes", that is classes with qualitatively similar behavior, independent of the construction details of a particular automaton. Cellular automata can be visualized as a checkerboard. You have a rule table, that tells you "for every square on the board", look at the neighbors, and depending on the patterns of neighbors (are they occupied, are they empty), change the state of that cell. All cells on the board are updated in parallel. Wolfram, simplified the problem by studying onedimensional cellular automata (think of one row of the checkerboard), and examined different rule-sets, and different starting patterns systematically. He found a set of 65
Developmental Systems common long-run patterns; the "universality classes" that closely matched those mathematical systems traditionally studied in physics. In Table 6, I have overlaid his classification upon Collier-Hooker's. In my interpretation Wolfram's four "universality classes", correspond to only three quadrants of the complexity-organization plane. Wolfram's four classes were as follows. Class I, systems that move to a single fixed point, and thus represent point attractors. Class II, systems that move into a cycle over time, periodic attractors. Class III, systems that have essentially statistically random patterns, and hence represent chaotic attractors. Class IV, systems that produce complex persistent patterns, that do not appear to coincide with other physical system models. Wolfram noted that Class IV systems, because they could produce persistent structures, could be designed to function as a universal computer; that is they could mimic algorithms, where different starting conditions would represent different algorithms. Langton (1991) further studied Wolfram's universality classes, extended it beyond 1 dimensional cellular automata, and devised a parameter, "lambda" to explore the various rule-spaces. Essentially, the lambda parameter represents the transition probability to a "quiescent state"; which for a binary rule function would be equal to "empty", and simply a chosen state for a multi-state rule function. It could be considered to represent a "birth rate" of sorts. Based on the lambda parameter, Langton was able to characterize the cellular automata rule space in terms of changes in the associated parameter values. He found that varying the Lambda parameter from 0 to the maximum allowable, corresponded to moving through universality classes in the following order I-II-IV-III, with by far the largest set of cellular automata exhibiting either class II (periodic) or Class III (chaotic) behavior. Class IV, represented a very small region of the parameter space between Classes II and III. He was also able to show, that measures of mutual information between a cell's present state, and its future state had the greatest variability within this class. These results led Langton to several closely linked and inter-twined conjectures. The first conjecture was that the pattern of changes from class I to II to IV to III was analogous to changes of state in matter, or phase shifts, with class IV existing at a phase transition from liquid to gaseous. The second conjecture was that the periodic regime (II) and the chaotic regime (III) were analogous to halting and non-halting computations respectively, and that the small region in-between (IV) represented 66
Developmental Systems undecideable computations (that is, if we programmed the computations into a computer, we don't know if the program would halt or not). The third conjecture, dependant on the previous two, were that the diversity and complexity of information patterns in class IV, were characteristic of living systems, and hence, living systems could be seen as analogous to undecideable computations and phase transitions; that life existed in a small region "at the edge of chaos". Kauffman (1993) later confirmed Langton's "edge of chaos" phenomenology for cellular automata in terms of the closely related boolean automata. The results then can be seen to be a general property of mechanical automata. The key metaphorical extension that was made, was to equate life with computation. This is all very suggestive. But there are several problems. First, and foremost, there is no cohesion. The system patterns follow deterministically from a rule table, and the previous state. Enter the same initial pattern, and the system will re-run itself, indeed must re-run itself, if it is to be a universal computer. That is, the patterns that appear, are epiphenomena (independent common cause), rather than due to causal interactions among components. The patterns are wholly reducible to the underlying dynamics of the algorithm the "structures", patterns on the grid, do not interact with each other in any causal sense; they simply follow the local rules of an update table. There are no higher level interactions, or emergent properties. There is no higher level. Knowing the state of the system at time t, you can exactly calculate its state at time t+1. The non-predictability comes in saying what will happen many t steps forward. The system is functional, not relational. The patterns generated in a cellular automaton are no more cohesive, than the figures of actors on your television. As everyone child eventually learns the actors are really not there, inside the television. Rosen (1991, chapter 7) examines simulation versus models in detail, and decouples them, comparing simulation to clockworks such as an orrery. The motion of a set of clockwork planets may simulate the effect of gravity, but it is actually the effect of the underlying gears, and the orbits do not causally reflect gravity at all. Our computational simulations of life (sometimes called "artificial life") are fascinating (indeed, the worlds most famous cellular automata is "Conway's game of life"), but they are far cries from life itself, or even from being appropriate models for
67
Developmental Systems life. A clockwork orange may look sweet, until you attempt to inquire too deeply, and bite into it. Ulanowicz (1997a,b ), in a very interesting synthesis of empirical data, asked whether ecosystems actually exist "on the edge of chaos", by examining how real ecosystems actually line up, respective to a theoretical calculation that delimits the phase transition between stable (like Wolfram's Class II) and unstable (like Class III) network connectances (the number of connections between an ecosystem component, and other ecosystem components). He found that the living does not cluster around the edge of chaos, but in a broad region which he called "the window of vitality". That is, he was describing systems that corresponded to Collier and Hooker's Class 4, and not to Wolfram (and Langton's) Class IV. All four of Wolfram's universality classes can be said to be mechanical, in the sense that they can be reduced to a mechanical procedure to calculate their dynamics, an algorithm or program for a computer simulation. They share this feature, with the mathematically tractable physical systems Class 1,2,3 whose mathematical models can be stated as algorithms. These systems have generating functions or algorithms that fully determine their behavior, and that is what distinguishes them from Class 4 systems. Ulanowicz (1997) has noted that only senescent living systems tend towards determinate mechanical behavior. Class 4 systems, by contrast, can change in form with time. Such change in form occurs when their underlying dynamics change. Finally, such systems are relational rather than functional, so not reducible to a state based description. As I have noted in Table 6, I believe, Class 4 systems, represent a large body of physically realized systems (likely the largest) that have multi-scale dynamics, and are hierarchically structured. The class of nonlinear discrete dynamical functions that exhibit universal computation, is like a small mechanical window, looking out into a much larger non-mechanical landscape. In this larger non-mechanical world, relations, not functions rule, and the state is determined by cohesive interactions among components, rather than a-priori assigned. Life is much less like a phase transition in matter, and much more like 68
Developmental Systems the colloidal state of matter. This is not to belittle insights from investigations of chaos and complexity in the context of mechanical systems. These studies, take us to the edge of the mechanical world-view, and give us initial insight to what lies beyond. To actually explore what lies beyond, one must however venture past mechanical simulations and investigate biological systems directly. Wolfram, Langton and Kauffman's work on automata date from the late 80's and early 90's, and at the time, I was much inspired by their work (Banerjee, 1990), and believed that living systems could be understood deeply in terms of nonlinear dynamical systems. The last decade of data analysis, and work on large ecosystem scale phenomena, changed my mind, and led to my current position, that developmental systems can not be reduced to dynamical systems as they have been classically understood. However we will see later in this essay that Langton's insights based on cellular automata studies are a mechanical prequel to the new non-mechanical landscape we must explore in developmental systems or in complexly organized dynamical systems generally. The road to understanding complexly organized dynamical systems has been something of a process of elimination. First we eliminated linear mechanical systems. Then we eliminated analytically solvable nonlinear mechanical systems. Now, we have eliminated computer simulable nonlinear mechanical systems. The quadrant that is left to us -- Class 4 systems -- however, is wide enough to encompass the majority of real systems in the universe. What are the characteristics unique to living systems? They are systems characterized by fairly weak molecular bonds, thus unlike rocks, they must actively work to maintain cohesion. They are fairly easily impinged upon by external forces, and so must be able to adapt to and anticipate such forces; they must actively attain a degree of autonomy from their environments. The ephemeral nature of complexly organized dynamical systems, the constant work they must do to maintain their internal structure, their organization and infodynamical nature is a new subject of study as we develop a science of self-organized systems and a science of complexity, but the phenomena itself, the fluid nature of the living is a centuries old theme in philosophy and literature. I end this section with a metaphorical illustration of the basic phenomena of living as existing in the interzone
69
Developmental Systems between the liquid and the solid -- the viscous and colloidal -- a matter of degrees of freedom: "Water is verbs: ripple, flow, fall. But immobilized in ice it turns noun -icicle, floe, berg -in its trapped sounds and memories of cascading river, swirling whirlpool, rapids tumbling over rocks -perpetual motion immured in crystal stillness. And we, too, made of water are verbs in our flowing, living, liquid in our bleeding, crying, until death nouns us and the verbs that make us antic and kinetic are stilled." -- Jean Mallinson, "Water is Verbs", from "Quintet"
This section, has been the most difficult passage so far. However, from its vantage, we can see clearer how the notion of developmental system has split away from that underlying dynamical systems or mechanics. From this high point, we will now descend rather steeply back into data. We return to the study of traces, and look for a signature.
70
Developmental Systems
Connections "What distinguishes this system from the others? The relation to permanent traces (Dauerspuren) and to remainders of memory (Errinerungreste)? In all systems the most intense and tenacious of these traces or remainders come from processes which have never reached consciousness. There cannot be permanent traces in the system Perception-Consciousness, for if there were, this system soon would be limited in its receptive capacity. Therefore, processes of excitation must leave no trace in it. If there are traces, they must be inscribed elsewhere, in another system." -- Jaques Derrida, The Post Card
In this section, we will descend rather steeply through several subjects, rapidly gaining some breadth, but sacrificing a certain amount of depth. I will try and highlight enough topography that you can revisit any area that interests you and explore more deeply. We are heading down out of the mountains, towards the sea, and I want to get us there by sunset. FURTHER TRACES OF TRACES So where have we gotten to? We've first developed a model of measurement in terms of the Probe-System-Trace relation and used it as our probe to dig into the limits of traces, to develop a notion of vagueness, and to identify some properties of developmental systems as a generalization of dynamical systems. We've used the development of "complexly organized dynamical systems" to identify the landscape in which developmental systems play, connecting it to the search for a model of autonomy, adaptivity, and anticipation: the triple-A signature of the living. What other connections are there? Well, first of all, to the empirical programs that provided the raw traces from which the Probe-System-Trace relation was abstracted. These are, moving up the biological hierarchy, the research groups centered around K. Matsuno, (development of coordinated activity in biomolecules) J. Maze (development of the individual), R.E. Ulanowicz (development of ecosystems), and D. R. Brooks (development of biota) and their students and collaborators.
71
Developmental Systems The idea of developmental systems originates out of a fifteen year mediation on the nature of evidence biological data provides us. I am primarily a field biologist, and a data analyst. It was in the context of data-analysis, that I began to doubt the mathematical tools we as biologists commonly used to analyze our data, and the mathematical models that were developed. These models have moved in the last century through a progression from deterministic functions, to probabilistic functions, to most recently, chaotic functions. It is the nature of a data analyst to be somewhat at odds with the theorist, and when presented with a model of elegance and simplicity (or even elegance and complexity) to ask simply: what is the evidence to support this. The role of the data analyst is to bring theory and evidence into confrontation (Hilborn and Mangel, 1997), given the proviso that no data analysis can itself be totally theory free. The consequence, is often a profound level of doubt as to the adequacy of any theory, including one's own. However, within this general landscape of doubt, what a data analyst attempts to do is to look at the degree of symmetry between a theory of processes and the patterns it produces. Do the patterns produced by the theories represent signatures, which can be used to group traces found in monitoring, experimental and observational studies? It was in this confrontation between trace and putative signature that the notion of developmental systems developed. In particular, my ideas are a synthesis of the concepts developed and the data collected by the four research programs above, and follow from consideration of their published and unpublished work as well as my own analysis of traces from morphology and ecosystem level studies. This section contains some biologically specific jargon, used in the empirical research, so please feel free to skip over unfamiliar words, and follow the "sense" of the narrative instead. I would like to summarize this confrontation between trace and putative signature in terms of these four research groups. In doing so, we will finally unearth an invariant moving up the biological hierarchy. My summary is in narrative form, and truncates the actual process of discovery, which was unique to each group. Some key references for each group are listed, as a gateway to the empirical research: Matsuno: 1989, 1992,1997, 1998; Honda et al., 1995. Maze: 1983, 1998; Kam and Maze, 1974; Maze et al. 1972, 2000; Robson et al., 1993; Banerjee et al. 1990, Brulisauer et al., 1996. Ulanowicz: 72
Developmental Systems 1997a & b; Ulanowicz and Norden, 1990; Ulanowicz et al. 1997. Brooks: 1997, 1998; Brooks and Wiley, 1988; Brooks and McLennan, 1991. I should note, the groups are not homogenous in their views, nor should they be -- each group has approached the developmental thema on its own terms. What is held in common is the general conclusions that have been reached across the various systems worked upon which unify the different areas of specialization. By identifying each research group with a single individual, I am simply providing a figurehead to encapsulate all the interesting individuals, their interactions, jostlings, discussions, and insights that collectively create a true research group. At the level of biomolecules, many of our theoretical structures are based on reaction diffusion, kinetic and hydrodynamic models, often modified from models in physics and physical chemistry. These models deal with rates of change in material location, flow gradients, etc. Matsuno's confrontation between putative signature and trace came out of the realization that such flow equations must have a starting point, and before that point there is no flow; there is no system that can be coherently modeled as a set of differential or partial differential equations. He realized, in essence, that for coordinated activity such as flow along an energy gradient: a biomolecule must first recognize the source of energy, an interaction must occur to transfer the energy, and then this transfer must finally become synchronized across a series of biomolecules acting in coordination in a spatial direction. Only when all this occurs is there a flow, and the ability to model flows in terms of applied forces. The global flows actually occurred through a series of local events, and the local events did not exist anywhere in the equations that modeled flow. Recognition and coordination was simply assumed. Thus, once an energy gradient was clearly set up, the equations were sufficient to describe the time variant behavior; but failed to describe the conditions under which the energy gradient exists in the first place. The realization that all dynamical accounts assume coordinated activity, or a synchronized global time, led Matsuno and his group to develop a "molecule's eye view" or internalist account of the origins of coordinated biomolecular activity, and develop a theory of how highly local interactions between molecules can lead to global behavior, 73
Developmental Systems flow. Matsuno's empirical results are taken up in more detail later in this essay with respect to a discussion of internal time The theoretical concept of internal time is intimately tied to empirical details of many-body interactions as they are observed by biologists. At the level of the individual, our process theories say that the characteristics of the individual are a function of genetics and environment, and that if environment is held stable, genetics should emerge (see Oyama, 2000 for a critique of this standard viewpoint). The confrontation for Maze's group (of which I am a part) occurred through an analysis of individuals at different stages of development, and in the context of different levels of the biological hierarchy: individuals of related species, individuals from different populations, individuals from different crosses, even individuals (as in embryos) collected repeatedly off the same individual. Over a twenty-year series of botanical studies, it was found that the bulk of variation existed among individuals, and was not well explained by the use of genetic background (family structure), environmental background (sites) and population background (allopatric populations) as covariates. Only at the level of species background as a covariate, did fundamental groupings on the data emerge. Finally, in the process of collecting and analyzing parts (organs) relative to a whole (individual), it became clear that differentiation of the individual was the primary source of variation, and the other levels constrained it to differing degrees. This led to a focus on the emergence of variation within the individual, i.e. the coordinated activities of parts relative to a whole. In a series of studies looking at the concept of emergence, Maze developed an analytical protocol to detect emergence in hierarchically organized data sets. Degree of emergence was assessed as the difference in the correlation structure of the lower levels, relative to the structure of the higher levels; i.e. the higher levels had analytical properties that could not be reduced to the properties of the lower levels. The low/high level pairings were comparisons within/among individuals, populations and species. In a recent study (Maze et al. 2000) degree of emergence was addressed in both ontogenetic and phylogenetic time, by comparing ovule development in related grass species, and looking at changes in emergence between different stages of ovule development. Degree of emergence was found to increase over 74
Developmental Systems ontogenetic time, and also to reflect the higher phylogenetic level. What resulted was a mixture of constraint and variation. Individuals differentiated from each other over developmental time, but were constrained by their phylogenetic repertoire. In an interesting analogy to Matsuno's work, Maze identified the source of variation as occurring in terms of local developmental contingencies, while the source of constraint was in global shared phyletic history. At the level of ecosystems, Ulanowicz began with looking at the nature of coordination among functional components of an ecosystem. His viewpoint was coarse grained above the level of the individual, and focused upon the level of assemblages of individuals (even from different species) who could be said to have well defined roles in the ecosystem, falling into groupings such as autotrophs, herbivores, carnivores, detrivores, etceteras. He characterized the relationships among these ecological roles or components using information based measures of flows among ecosystem components, and tracking material exchanges among the components (being eaten). His focus on material exchanges (such as carbon flow or nitrogen flow) which could be assigned an energy value allowed him to relate his work to thermodynamic principles, but more important were the patterns that developed, and were reflected in the information accounting procedures he developed. With time, an ecosystem tended towards both greater processing of energy (throughput), and greater internal orderliness and organization (mutual information) in terms of flows across functional components of the ecosystem. This product of throughput and mutual information, he called "Ascendency", and this ecosystem ascendency was stable to all but major perturbations. Ascendency was an interesting measure, since the throughput component of it, represented overall ecosystem activity, and could be seen as an extensive (size related) variable, while the mutual information component could be seen as an intensive (size independent) component that represented the internal structuring of ecosystem activity The patterns he found were macroscopic, and represented continual change and succession with time. The confrontation he faced was a unique one -- ecology had no strong theoretical candidates for a dynamical explanation of this phenomenon. Rather, functional ecology was focussed on single level models, such as when predator and prey would reach an 75
Developmental Systems equilibrium, or in the more recent theories would never reach equilibrium but would have boom and bust cycles characterizable by chaotic equations. There were also stochastic ecological models, which saw ecological assemblages as being due to historical accidents or initial conditions. But there were no models to explain the both the regularity and contingency inherent in ecosystem structure. Theoretical ecology had effectively ignored the key pattern evident in his traces. What he calls "The Ascendent Perspective" has been his effort to eliminate that lacuna in theory and develop a comprehensive model of the growth and development of ecosystems that has both regular and stochastic aspects. His model provided a common framework to compare different ecosystems, and within the ascendency model, he was able to develop measures of ecosystem health, stability, and redundancy. He was also able to demonstrate the effect of certain flows as being rate determining steps for the ecosystem as a whole, which linked his ideas back to cybernetics, the idea that certain flows could "steer" the ecosystem as a whole. While his "information accounting" procedures were developed in the context of his ecosystem work, they are actually quite general, and could be applied to any situation with material flows. It is only a short jump from his accounting procedures for ecosystems, to accounting procedures to look at economies. The patterns he observes also do not depend on the information accounting procedures themselves. Brulisauer et al. (1996, p. 1780 fig. 9) looked at a successional series of forest ecosystems using multivariate techniques similar to those used by Maze's group to look at morphology and development of individuals, and demonstrated patterns concordant with ascendency: with age-class, the variability of the ecosystems increased, while their internal organization also increased. Strong traces, like strong signals, are easily cross-validated. Since Ulanowicz's model encompassed both ecosystem regularities, and random events, it again expressed the theme of constraint and variation, both increasing in a coordinated fashion. Brooks group focuses on phylogenetic systematics (also called cladistics) and coevolution of biota at the level of the upper levels of the biological hierarchy, family, genera and species groupings. In particular, his group has focussed on parasites, that is organisms whose environment is another organism. Several aspects of the traces he collected confronted extant theories of biological change. First of all, the methods by 76
Developmental Systems which phylogenetic systematics produced its classifications often identified traits usable for distinguishing species that had no known adaptive consequences. Traits with known adaptive consequences were often useless for distinguishing species and determining ancestral relationships among species. Secondly, while he found a pattern of increasing diversity with time, a biological truism know as "Dollo's Law", what struck him when he began to experiment with information based measures was that the diversity was severely constrained from what it could be. This surprising result led him to ask effectively, "why is their constantly increasing diversity in biology, and why is it also always less than it could be." He particularly compared the patterns derived from his data to various models of speciation (see Brooks and Wily, 234-249). Again, he began with the use of information measures, as essentially an accounting system for tracking speciation events, and found a pattern of change that lacked theoretical explanation. The third line of evidence was from the use of the traits of the parasites he studied, to predict the phylogenies of their hosts. In effect, the parasite had become another character of the host. This large scale consilience between host and parasite, as both evolved was never even addressed in microscopic process theories of co-evolution which treated hostparasite relations on the same scale in terms of competitive models (battle of the reaction diffusion equations), again predator prey type models which were the various stable and chaotic successors of the Lotka-Volterra type logistic equations. Effectively speciation was occurring at different rates along different parts of the phylogenetic tree over geological time periods across different environments. Brooks concluded there was something about the nature of the organisms that was biasing change. It was these trace based investigations that led him to postulate that species change was historically constrained, and that organisms effectively buffered themselves from their environment (and hence selection pressures) by maintaining cohesive internal information over phylogenetic lineages. In this view, natural selection had a role essentially equivalent to a rate determining constraint, but one among many constraints, the primary one being the cohesion of the information system itself over time. There were conditions under which natural selection could dominate species dynamics, but they were highly particular, and operated through affecting species cohesion. The pattern over time, reflected in the information measures was for the system (across species) to increase irreversibly in 77
Developmental Systems diversity, and to do so with bias. The key insight was that if species were looked at as physical information systems, their dynamic pattern across time was effectively entropic, diversity increased. However, the increase in diversity was constrained in such a way that organization also increased. Brooks and Wiley summarized their findings in four general principles, which collectively represented their core hypothesis. First was the principle of irreversibility, which noted that living systems increase in physical information at all hierarchical levels that have been empirically investigated. The second was the principle of individuality, which notes that specific instances of any hierarchical level take on unique characteristics. That is, for example, ecosystems and species can not be modeled as classes, but must be modeled as individuals, each particular case becomes uniquely marked or traced upon. The third principle was that of intrinsic constraints. This refers to the need for system cohesion, whether it be in an individual, an ecosystem, a species. Principle three (intrinsic constraints) placed global constraints on local mechanisms that increased a system's physical information. Thus principles one and three jointly led to the pattern of both increasing variation, and increasing organization in biological systems. The last principle was that of compensatory change. This principle referred to a system's ability to re-structure itself internally to respond to external changes. This last principle is quite interesting, because it provides a way for biological systems to actively generate information internally (through restructuring) in response to information originating externally. The principle of compensatory change thus creates a way for an internal trace to be created in response to an external trace. We will explore this feature in a novel context later in this essay. It was the lack of correspondence between traces and existing process theories for different levels of the biological hierarchy that led each group to diverge from the mainstream of its field. Three of the groups, those of Brooks (Brooks and Wiley, 1988), Ulanowicz (1997), and Matsuno (1989), have produced book length explication of their alternative theories, based on their dissent with the existing process theories. All four groups have develop research programmes for their particular scale of the biological hierarchy that are unified in looking at the information patterns in biology. This focus on
78
Developmental Systems patterns of change in physical information systems has been labeled by Salthe (1993) as "infodynamics". Brooks work actually encompassed the complete biological hierarchy and strongly influenced both Maze and Collier. The development of the notion of complexly organized dynamical systems comes in part out of Colliers efforts to put infodynamics on a firm ontological basis. Brook's group, Ulanowicz's group and Maze's group, each developed hierarchical information measures based on the particulars of the kind of data they collected. What is most interesting is that when an information based account is given for each of these different levels of the biological hierarchy, the pattern that emerges from the information statistics on the original data traces, share a common form. That common form, is summarized in the figure below (redrawn and modified from respectively, Salthe 1993, p. 183 fig. 4.8, Ulanowicz, 1997, p. 87, fig.4.9; and Brooks and Wiley, 1988 p.43 fig. 2.5, Frautschi, 1988, p. 18). I suggest this figure is a signature for developmental systems, or Class 4 systems in general2.
2
See Salthe (2000) for a related argument that figure 2 is the signature of developmental systems.
79
Developmental Systems
Figure 2: A Signature for Developmental Systems This diagram requires some explanation (the labeling is adapted from that used in Sibbald, et. al 1989). As I have said, it can be taken as a signature, an abstract grouping across traces, that shows what is common or invariant among the traces. The horizontal or X-axis is "time" in an external account, or age or stage of development in an internal account of a developmental system. The vertical or Y-axis represents system complexity. Maximal, but disorganized complexity is reached when there are no constraints on a system given its parts, and this is reflected in the line C0. Constraints on the actual distribution of parts of a system leads to a decrease in complexity, C1. The difference between C0 and C1, represents the constraints, and this is what Brooks' group has interpreted as historical constraint. Below C1, is another line, C2. The difference between C1 and C2 is what is called in information theory the "mutual information", and it represents the constraints on Y given X, in the terms of our earlier trace analyses. It is the basis of the "ascendency" measure developed by Ulanowicz, and it is also the basis of the
80
Developmental Systems multivariate measures of order presented in Banerjee et al. (1990)3. Mutual information provides a strong link between information theory, multivariate statistics used in morphological studies, and chaotic dynamics (Banerjee et al., 1990), and can be seen as representing a deviation from symmetry in any physically embodied network of relations (Collier 1986, 1996). It thus provides a handy analytical technique for applying physical information concepts in empirical studies. Physical information arising from symmetry breakage in physical systems also provides a deep unifying principle between systems considered the proper study of physics and systems considered the proper study of biology. A wonderful derivation of mutual information, given from the perspective of a student struggling to grasp information theory is given in Renyi (1984). Collier, (1990) uses mutual information to banish that neat fingered spectre in thermodynamics mythology, Maxwell's Demon. In addition to the first two informations, there can be further constraints on a system, which will lower the visible complexity of a system, which are labeled C3 … Cn in figure 2. One could consider them interactions further back in time, or in a causal sequence. The imposition of such constraints on the complexity of the system would result in higher order measures of information which are labelled,I3…Ip in figure 2. Effectively, each level of a system can have such an account, in terms of hierarchically ordered information measures, that is the diagram relates to a focal level in a hierarchy. In any study across several levels of the biological hierarchy, you could almost think of them as a set of nested diagrams, much like a set of nested boxes. The higher order information measures can also be seen as sums over histories. Where the data is temporally ordered, the mutual information can be seen as looking essentially one step back in history, and the higher information measures looking farther back in time. In that sense, it bears some similarity to other sum-over-history methods, such as the visual notation developed by Feynman for quantum electrodynamics (Mattuck, 1976). The
3
I have labeled it "ascendency" simply to show the relation of ascendency to historical constraint;
Ulanowicz's formulation of acendency is properly the product of mutual information and throughput, i.e. scaling mutual information by system activity.
81
Developmental Systems histories in this case, are derived from the temporal series of measurements, rather than from theory. And yes, at this level of grouping, where we are talking about the behavior of the system as a whole, we have returned a function (modelers rejoice!), but a function that is the macroscopic reflection of the relations characterizing the underlying systems. It is in essence, a function that speaks about how relations in developmental systems change with time. What is truly remarkable, is that this signature first entered biology when Brooks and Wiley (1988, Chapter 2) adapted it from cosmological models developed independently by several physicists (Landsberg, Frautschi, Layzer) that allowed order and entropy to increase together in the development of the universe. This was a rather brave transposition of the cosmological to the biological, and if it holds, then cosmology may be seen as another developmental phenomenon. Some day physicists may speak of the embryology of the universe, in addition to speculating on its birth, heat death, and the existence of sister universes. Can a phylogenetic tree of the fundamental forces and particles be far behind? With respect to developmental systems, this signature has been repeatedly confirmed at the level of development of an individual, development of ecosystems, development of species lineages. As such it has the force of a phenomenological law, specifically what Brooks and Wiley (1988, p.) called the "the principle of irreversibility" whereby "the components of living systems relevant to biological evolution are therefore those which exhibit irreversible (i.e. entropy producing) behavior." Independently, Ulanowicz (1997, p. 75) framed his principle of ascendency around this phenomenon of increasing diversity and internal structuring, whereby "in the absence of overwhelming external disturbances, living systems exhibit a natural propensity to increase in ascendency". Ulanowicz (1997, p. 75) notes several provisos: There are three significant words pertaining to the statement just made. Two are noteworthy by their presence, the third by its absence. The word overwhelming qualifying disturbances implies that living systems are always subject to 82
Developmental Systems disturbances. In fact, I argue below that some minimal disturbance is required in order for a system to continue developing. Second, that the statement is statistical in nature is revealed by the use of the word propensity, sensu Popper. Disturbances may temporarily decrease system ascendency, but the underlying and unremitting restorative tendency is in the direction of increasing ascendency. The only disturbances excluded from the hypotheses are those that overwhelm the autocatalytic configuration itself, i.e., "kill" the system. Missing from the central postulate is the word "maximize". Of course most measures that rise but remain finite, do eventually approach a maximum. That is not to say, however, that such a maximum exists at the outset as a goal or an objective towards which the system is driven, as can in fact be said of many physical systems. Essentially both Brooks and Wiley, and Ulanowicz are describing a phenomenological law. Using the resources developed in the section of complexly organized dynamical systems, and Collier's account of physical information and the negentropy principle of information, we can restate that law as follows. The first law of infodynamics. A physically realized developmental system will have a propensity to increase physical information, in the absence of disturbances large enough to destroy system cohesion. . Each of these groups, working on very different data sets, and aspects of the biological hierarchy, have come up with the same patterns, of systems spontaneously increasing in variety locally, but within constraints that act globally. Both the constraint and variety increase with time, age or stage. Thus at the level of biomolecules, we have the phenomena of a system, such as actin muscle fibres, becoming synchronized to produce a coherent contraction. We have, at the level of development of the individual, the expression of variability in the parts of an individual, and this variability changes in organization with time, so that individuals become both more variable and unique with 83
Developmental Systems time while also undergoing regular changes through time. At the level of ecosystems, we have a pattern of both increasing throughput of total matter, with increasing orderliness and strengthening of connections between the ecosystems components (ascendency). At the level of species themselves, we have a pattern of increasing diversity with time, often regardless of what the environment does. In each of these cases, we have patterns that are not functional but relational in nature. That is, for every X there are many possible Y's. But some Y's are more likely than others, and the pattern of interdependency between X and Y changes with time. The changes in pattern with time, where the change seems to be internally written into the system for future times, constraining a system, but also leave it free to explore within those constraints, is seen as synchronization at the molecular level, individuation at the organismal level, increasing organization at the ecosystem level, and both as increasing diversity at the phylogenetic level, as well as convergence when distantly related groups are free to explore the similar developmental pathways inherited from more distant ancestors. Each group came to its particular representation of this diagram, essentially by a process of trial and error and working closely with data as the guide. Jonathon Smith (1998, 1999), intrigued by the mathematical implications of Brooks and Wiley's (1998) version of this diagram, went on to develop a rigorous mathematical model of hierarchically organized information systems that can be used to analyze the development of complex systems functioning at more than one hierarchical level. Essentially he created a statistical mechanics applicable to biological hierarchies that can act as the formal image of this empirically derived signature. While figure 2, can be seen as a signature of developmental systems, and thus an aid to unification of biological phenomena across hierarchical levels, the nature of its derivation obscures certain subtleties, such as the relationship between order and information. The nature of the diagram -- two dimensional -- does not allow us to distinguish constraints that represent the orderliness of a system, and the actual pattern of organization of the system, and often people have treated order and organization as interchangeable terms -this is incorrect. The inter-relation of order and organization is a subtle concept, for 84
Developmental Systems organization depends on their being a certain level of orderliness in a system, but for a given level of orderliness, there is a one-to-many relationship as to the actual pattern of constraints, or organization of the system. One way to think about it is this in terms of the following visualization: Say you are an artist, and you have created a super stupendous mobile. It consists of a set of wooden branches, which have holes drilled on each end and are bound together with rope lashed through the holes in each end. The tightness of the ropes defines the flexibility of the mobile and the shapes it can take. We will look at order and organization in terms of the shapes this mobile can take as a standing mobile. When you tie it all so that the ropes are slack, the structure can be bent into any shape. It has no apparent order, nor does it have organization, as a mobile (having tied the pieces together, has given it some inherent order, which we'll ignore for the purposes of this metaphor). As you tighten each of the ropes in turn, it begins to stand up. If you tighten each rope the same amount, the degree of movement decreases in the mobile, but its essential shape remains roughly the same. If you tie each rope as tight as it can be, the mobile has essentially no movement left. This is the progress from disorder, to order. Now consider that you change which branches are connected to which, this is a change of organization. You can keep tightening, and for this new organization, you will go through the range from disorder to order. Now consider that you have tied the mobile together in a particular way, tightened it up so it stands. And now, you begin at this point to re-tie some of the connections, but maintain the level of tightness. This would be a case of order staying constant, but organization changing. Now imagine further that you do not do anything so obvious as reconnecting any of the branches, rather what you do is tighten some ropes more, and other ropes less, while maintaining the overall average tension on all ropes. This is a case of very subtly changing the organization, by changing the dynamic range of certain subcomponents and compensating with balancing changes in other subcomponents, where any two branches tied by a rope is a "subcomponent". Note firstly, that the greatest range for creating distinguishable 85
Developmental Systems organizations is when the ropes are tied tightly enough that the mobile has some structure, but not so tightly that parts can not be re-tied in other ways: intermediate order is the substrate for the greatest number of allowable changes in organization. Note secondly, that as the mobile's tied ends have less room for movement, they "knit together the system" and moving one part of the mobile, can cause movement at another part of the mobile, quite distant from where the force is applied, i.e. apparent non-local effects, due to the system now acting as a cohesive whole, and averaging any applied force over its structure. At the point, where almost no movement is allowed, the mobile looks much like a large crystallike structure, and acts as a single rigid body to externally applied forces. Now finally, imagine that there is only a single rope that is used to lash the mobile together; try and see the consequences of this. Draw it out. Our putative signature expressed in figure 2 misses these subtleties, and obscures the details that biologists revel in. What it does do, is allow us a common framework, a language if you will, to talk about different systems on a common basis, and that language concerns physical information.
SYNCHRONIZATION OR INTERNAL TIME
Time, motion, and interaction are intertwined, both materially, and conceptually. We measure time in terms of interactions at various scales, leading to relative motion. Our year, is the motion of the earth relative to the sun. Our day is the motion of the earth's rotation bringing different parts to face the sun or face away. Our month is the relative motion of the moon with respect to the earth. Our minute is roughly 60 heartbeats. . A "molecular clock" measures time relative to the likelihood of a mutation. We understand time, by looking at how one thing moves relative to another. If the two things were independent, there would be no pattern, hence no time. If we had no way of assessing relative motion, there would be no time. Thus, while we may treat time as an axis or a
86
Developmental Systems variable in our equations, and identify other variables as "functions of time", we know that time can not exist independent of other material bodies interacting Time then, is an ordering on relative motions, and the relative motions arise out of interaction (otherwise their relativity would be spurious). Our focus on measurement systems, our Probe-System-Trace description brings up the obvious question of first contact: what happens when two systems encounter each other, and how do they come into coordination where we might say system A depends on system B, but it may just as well be that systems A and B covary, are equal partners in a dance. This problem of the synchronization required for coordinated activity could be called the problem of "internal time". With respect to the signature curve (Figure 2) for developmental systems, such coordination must occur in systems prior to our ability to measure and track their changes. There must be enough internal coordination among parts for there to be recognizable trophic levels in an ecosystem, homologies in a phylogenetic lineage, distinguishable tissues in a developing individual. In that sense, all the activities tracked and aggregated into the signature depend on this stage of initiation of coordinated interactions. Internal time has been approached both empirically and philosophically by Koichiro Matsuno, and through mathematical models of interactive systems by Jonathon D.H. Smith. I will briefly describe their work, and then put it into the context of the Probe-System-Trace relation. Matsuno's work begins by taking a "molecule's eye view" of the origin of biomolecular kinetics, in systems such as the actomyosin complex in muscles (Matsuno1989, 1992, 1997, 1998; Matsuno and Honda, 1991; Honda et al., 1995). His group's work looks at the origin of coordinated kinetic behavior, arising from uncoordinated behavior in terms of interactions. Matsuno notes that for dynamical behavior in a bio-molecular system there must be: 1. Recognition (detection) of molecules. 2. Local interactions.
87
Developmental Systems 3. Transfer of local interactions to create coordinated behavior, that is transfer of information, or realization of dynamics. Dynamical models essentially assume all three activities occur simultaneously. What Matsuno notes is that in observations of bio-molecular interactions these activities actually occur sequentially and with delays. Rather than varying smoothly and globally, the interactions occur locally and discretely. There is a delay between the availability of free energy supplied from an ATP molecule associating with myosin, and the actual hydrolysis of that molecule, and there is a delay between the availability of free energy and the coordinated hydrolysis of ATP by sequentially located actomyosin, to create a wave of contraction along myofibrils. The interaction between an individual myosin molecule and ATP lasts longer than the duration in which the free energy obtained would be dissipated by thermal fluctuations. This imposes a fundamental asymmetry to the extremely local molecular interactions that must be coordinated to exhibit a global behavior, explainable in terms of kinetics or dynamics. That is, for every "action" Matsuno sees a delay before "reaction", rather than action/reaction being simultaneous (as is assumed by Newton's 3rd law). Essentially, he does not take the derivative of a function of time for granted, but asks, "how does it originate." The essential question then becomes, how is locally asymmetric internal time resolved with dynamic and global external time (the "reversible" time in Newtonian mechanics). Matsuno's work, because it links information and energetics, is a good framework for application of NPI. If you take a section of muscle capable of contraction as the system, then the enformation is the actual structuring of the fibres, which provides the framework by which myosin and ATP can come into contact. The free-energy provided by ATP then allows the actomyosin complex to maintain a far from equilibrium state, i.e. be intropic, and be able to do useful work, carrying a wave of contraction along muscle. A key requirement for the maintenance of this intropy is that the ATP-myosin interaction that transfers energy is of longer duration than the inverse of the thermal frequency. A heat engine is thus created, and without it, there is no information flow.
88
Developmental Systems Jonathon Smith arrives at internal time, by a different, but related route. In "Competition and the Canonical Ensemble (1996, see also 1998b), he uses the mathematical model underlying statistical mechanics to look at a populations of competing species each of which have different growth rates. He treats growth rate in a species as analogous to velocity in a particle, which allows him to derive a macroscopic equivalent of temperature, a statistical aggregate. However, the units of this macroscopic parameter are in terms of time (growth rates), and he calls this "historical time". Time, then emerges in this model as a result of interactions. It should be noted that by using the canonical ensemble approach, the model assumes instantaneous and non-local recognition, an increase in the number of one species must be immediately compensated by the number of another species. These assumptions arise from the canonical ensemble model itself, which circumvents the difficulty of explicitly calculating all combinatorial possibilities of microstates, by assuming the system of interest is in equilibrium with a thermal reservoir at constant temperature, and the total energy is held constant. These assumptions allow simplifications that make calculation tractable (Callen, 1985, Van Ness, 1969). In the case of the competing species model, the analogous equivalent to total energy is total population size. Given that much of the biological phenomena of interest to us with respect to developmental systems occurs far from equilibrium, it is worth considering whether Smith's models of interaction could be "opened up" and allow for situations of net imports and exports in a non-equilibrium system. If this were possible, his models of species interaction could serve as a formal image of the species interactions that Ulanowicz tracks using his ascendency calculations. The resulting models however, would no longer be equivalent to a canonical ensemble formulation. What they lose in mathematical tractability, they would gain in application to well documented empirical studies of species interactions in ecosystems. More recently, Smith (1998a, 1999) has extended his model to deal with demography, i.e. spatially distributed species, and looks at the relationship between internal time of the mother and calendar (external) time, noting that the scale ratio between these two times is constantly changing. Common to both Matsuno's and Smith's work is the focus on the relationship between internal time arising out of interactions, and the global time it leaves in its wake, the trace. We will now look at this relationship in more detail. 89
Developmental Systems
The Probe-System-Trace relation, actually originated out of my attempts to understand Matsuno's empirical and philosophical accounts of internal time, so it is somewhat tautological for me to explain internal time in terms of the Probe-System-Trace relation. However, like all good tautologies in logic, I hope it brings knowledge already possessed to the surface. The visualization reflects my childhood affection for action/adventure westerns. . Each biomolecule is a probe. We are riding bareback on the probes, and hence part of the probe. There are various other biomolecules, each with their individual riders. I can see the other riders on their steeds, they can see me. We are riding through a valley, and thus try and find the easiest route through the landscape, the route that uses least energy. Every time two riders get close enough to signal each other, they start riding together, acting as object system to each other's probe, and engaging in mutual measurement leading to coordination. The molecule riders have to be very close to see each other's signals (just like hydrophobic and hydrophilic molecules or molecules with weak positive and negative charges and molecules with specific receptor sites), and so ride about effectively randomly at first, until initial contacts and hook-ups are made. As small groups come near each other, they signal. Soon there is a posse arising out of the signaling, and the motion of the posse as a whole can be tracked through the valley, as a single wave. The cavalry is on its way to the rescue! A few probes and their riders do not join the posse, but wander through the landscape on other paths. Because their activity is not coordinated, no trace is left of their passage, and their journey is lost to history. If we view internal and external time as causally causally related but separated and logically distinct, then we can see the resolution of internal time of local interactions leaving a global time as a problem akin of the proper composition of relations. The problem of resolving internal time with external time can be summarized in the following sketch. In this sketch the "arrows" imply entailment, or "this leads to that", and each arrow represents a relation or a function. A double arrow represents a relation. A single arrow represents a function. For each probe (P), internal time is related to interactions with other probes. Let us label the relation "interaction" as "i", and say it leads to or entails internal time (Ti). Then for a system with only 2 probes Px and Py: [PxPy] 90
Developmental Systems Ti. Now let us say that each instance of internal time can contribute to a trace (Π), via a grouping relation "g". Then [PxPy]
Ti
Tr. Now an ordering relation (partial
ordering) on the trace, "o" defines the global time, Tg. Then [PxPy]
Ti
Tr
Tg. Finally, we will say that the time -Tg - external to the probes Px and Py can be used to define a unique state (S) of a system that exists at a higher level than the probes Px and Py (they are subcomponents of that system, by), by a proper function "f" which admits for each instance of global time a unique state. Then [PxPy] Tg
Ti
Π
S The sketch then is of how the interactions in a relational system of probes can
generate, at a higher system level a functional time. The final component, Tg
S
recovers state as a function of time. While the final mapping is invertible by definition, the sequence of mappings as a whole will not be, except under very special circumstances, and hence the composition of relations is irreversible. The likelihood of the sequence of entailments being completed is relative to the strength of the initial interactions; where they are weak, the system will be vague (two probes recognizing each other, being analogous to the problem of a scientist recognizing embryos). To view only that final entailment - Tg
S - is to ignore the composition of relations that preceded it.
Of course, to understand how we go from the interaction to a "state" we need to define the relations, i,g,o, and the function, f. As in many things, the demon is in the details.
OF TRACES AND TURING MACHINES
A Turing machine is the mathematical signature of a computer. It is a model, designed by Alan Turing, of a devise that can enact any algorithm, that is a set of sequential steps or rules. A universal Turing machine (let us call it UTM for short) is a generalization that can enact any algorithm. If there is something that can be computed, a universal Turing machine can do it. If something can't be computed, it is called non-Turing computable. In this final section, I return to mirror an argument first developed in depth by Robert Rosen in Life Itself, the signature of life, is different than the signature of machine. In a
91
Developmental Systems book whose theme is the living, it might seem odd, that his penultimate chapter prior to getting to "Life Itself" consists of developing a relational theory of machines, one of the most insightful deconstructions of the nature of computation that I have ever encountered. His argument is breathtakingly simple. In a UTM, entailments flow in one direction only, that is a chain of efficient cause (call it instructions from hardware to software); a computer does not rewrite its own hardware. Life does effectively rewrite its hardware, rewrite itself, change, develop, evolve. The latter is an over-simplification of Rosen's argument in Chapter 9 of "Life Itself", and rather than repeating his arguments (which take the previous 8 chapters of his book to set up), I will again approach the same conclusions while orienteering from a different angle. I believe we will arrive at the same place, and if you read both this account and Rosen's account, you will have the benefit of being able to imagine the same thing two (at least) different ways. Two other accounts of Turing machines that tackle computability and non-computability head on are Roger Penrose's "The Emperor's New Mind" (1989 see also Penrose, et al., 1997), and Chaitin's books, "The Limits of Mathematics" (1998) and "The Unknowable". Penrose's work argues that minds and consciousness can not be Turing machines based on problems with quantum mechanics. I can't say I really understand that part of his argument as to why fixing quantum mechanics is necessary to know what consciousness is (we won't be invoking quantum mechanics in the argument I propose). Chaitin's books are all about trace compression and Turing non-computability, er, Algorithmic Information Theory; how to build a Turing machine, and how to build a program that the gets the Turing machine to say, "I can't compute this". Chaitin's work in particular inspired my development of the polynomial trace compression algorithm, and accompanying tale of the enthusiastic algorithmizer. For a literate scientific biography of the algorithm and its formalization into the Turing machine, I can recommend David Berlinski's "The Advent of the Algorithm" (2000). For a concise description of a Turing machine, and its significance as a mathematical object, I recommend "Mathematics and Logic" by Kac and Ulam (1969). Each of these works, places the Turing machine in a slightly different context, and much understanding of the conceptual and mythical power of a Turing machine comes from reading across such accounts. I will briefly sketch an account in
92
Developmental Systems terms of the Probe-System-Trace relation, to clarify the limitations inherent in a Turing machine. Rosen's argument, was essentially to limit what a Turing machine can do. My approach, is to ask (1) can a developmental system be a Turing machine under certain conditions and (2) can it do things a Turing machine can not. Can a developmental system possibly do non-Turing computable computations? What additional capabilities might a non-Turing -- we will not call it machine -- system have? Kac and Ulam (1969, 161) provide a concise account of a Turing machine and its relationship to the notion of an algorithm: An algorithm is a set of precise instructions telling how to perform a certain task. It is easy to conceive of an automaton that will perform according to an algorithm without human intervention. Is there however a universal automaton that could be programmed to execute any algorithm? An affirmative answer to the question was given by the English mathematician Alan Turing, and his work became the theoretical basis of modern, all-purpose digital computers. Turing's universal machine is actually quite simple. It consists of an infinite tape divided into equal squares and a finite set of symbols (alphabet) which for the sake of simplicity may be taken as consisting of only one symbol, a dash
(If one
wants to be fussy one should say that there are actually two symbols, the other symbol being the blank). There is also a moveable scanning square that allows one to scan the tape square by square. Finally the machine can perform the following operations: (l): move scanning square one unit to the left (r): move scanning square one unit to the right 93
Developmental Systems (R): replace the symbol in the scanning square by any other symbol of the alphabet (in our case this means either erasing a vertical dash or printing a vertical dash in a blank square) (h): halt the procedure. A program is a set of instructions of the following form: "if the symbol in the scanned square is ______ perform ______ and look up instruction ______" So -- a Turing machine is an automaton that can enact an algorithm. This defines the notion of machine, and a mechanical procedure: something that can be enacted by an automaton. The cellular and Boolean automata we described earlier fall into this class. To put this in terms of the modern digital computers we are more familiar with, consider the following. The "scanning square" is part of a read-write head. The read/write head is guided by an internal program, which could be called the "machine language", which essentially defines a finite state space, in terms of "If … Then" rules, which we are familiar with as a generalization of functions. Since there is a limited alphabet, the state space must be finite. Together, the read/write head and the finite-state space of rules could be said to constitute the "hardware". The tape, in a UTM, represents data (and can also represent the program). So, given a unit of data and its present internal state, the action of the read/write head, and the next internal state the system moves to is fully determined. An interesting way of looking at this, is to suppose that the internal states of the Turing Machine, are themselves listed on a tape within the Read/Write head, that functions as a "look-up" table. Then, we could simply say, the Turing Machine has a tape inside, and a tape outside. But what is unique, is that the Tape Inside, is never marked upon. Only the Tape Outside is marked upon. Some other assumptions in a Turing machine are worth commenting on. It is assumed that each symbol is perfectly preserved and that the tape lasts with integrity throughout 94
Developmental Systems the session. This suggests a way of linking the Probe-System-Trace relation to a Turing machine. Coarse-graining on a dynamic interaction can result in a trace that looks somewhat like a tape. The creation of a Turing machine then consists of (1) coarse graining the trace into a tape, finding a probe to read the tape, and giving the probe a set of internal states. If all of this is achieved, you have a Turing machine that can imitate any formal system. However, most traces will only partially correspond to any formal system. Any trace that fully corresponds to a formal system; will have no emergent information, other than that contained in the formal system. So, to simulate a Turing machine within the Probe-System-Trace relation we need to make a list of replacements, to bring the Turing machine context into relation with the Probe-System-Trace model. 1. Let us consider the read/write head now to be a Probe (P) . 2. Let us consider the set of internal states to be a Trace within the probe, an internal trace (Tinternal). 3. Let us consider the Tape to be an external Trace (Texternal). Since it is the object of measurement, it is also the Object System. That is Osystem= Texternal. 4. The Probe reads Texternal by moving along it. It is able to move one unit forward or backwards. It is able to refer to its own internal trace, Tinternal. It is able to mark the trace Texternal With these replacements, the Probe-System-Trace model is brought into analogy with a Turing machine. Now let us consider several ways to move the Probe-System-Trace we have just designated into new capabilities. To do so, we have to look a bit deeper at assumptions in the Turing machine that may be weakened to provide new behaviors. Assumption 1: The internal states of the Probe are pre-determined and immutable. The implication is that for every Turing machine, there must be a Turing to design and fix 95
Developmental Systems those internal states. Let us weaken this by arguing for both mutation ands differentiation so that the internal trace itself undergoes change. The result is, that , given an identical external tape, the Probe will no longer respond identically. The question then becomes, how much change can an internal trace undergo, and still produce effective computations. If we assume that the likelihood of change to the Tinternal is stochastic, then we have change through mutation. If we assume that the change is due to reading the external tape, we have change through differentiation. In this latter case, the information "read" by the Probe does not identify a move to a different location on the Tinternal, it also can change the internal rule set. Four types of changes are possible: 1. Add/delete to alphabet. A new symbol could be added (or removed) 2. Add/delete to states. A new state could be added (or removed) 3. Add/delete to existing state transitions. A new transition could be added (a new rule), or an existing transition deleted 4. Redirect existing state transitions. An existing transition could be changed (change of rule). In a Turing machine, the structure of the finite state space is static, and the dynamics of the system consist of transitions from state to state. The actual basic transition functions (the internal tape) never changes. The modifications we have added, lead to a finite state space that is now dynamic, whose topological organization, whose alphabet, and whose extensive size can change.
Assumption 2: The Probe can uniquely distinguish the symbols of the alphabet. That is, it can recognize markings on the external trace such as 111111 as equivalently "1" and 000000 as equivalently "0". The symbols, can be seen as actually signatures. And this raises the question, what is the scaling requirement between the Probe and Texternal to allow such a reading? So -- the Probe-System-Trace relation can function as a Turing machine as long as the internal states are constrained to simulate a static finite state space, and as long as scale 96
Developmental Systems heterogeneity is constrained to simulate precisely distinguishable symbols on the Texternal. If the constraints are completely eliminated, the system would not be capable of producing any effective computations. However, it the constraints are relaxed, the system may well be able to produce effective computations, but also now have new properties. In particular, its internal states can change, which allows the repertoire of behaviors to change and grow. Secondly, scale-heterogeneity between Probe and Texternal, allows for multiple readings of the same tape -- several calculations can occur simultaneously, and be different calculations. These changes allow for capabilities that are beyond a Turing machine, while also guaranteeing that under sufficient onstraints, the system can effectively imitate or simulate a Turing machine. The converse is not true; because the Turing machine represents the Probe-System-Trace relation under tight constraints, it can not continue to simulate a Probe-System-Trace relation once those constraints are relaxed, because the resulting behaviors are beyond the repertoire of the Turing machine, which must assume a static internal state space, and scale-homogeneity. As for the Probe-System-Trace relation, as constraints are loosened, and while the system maintains cohesion, it also becomes capable of doing calculations that are non-Turing-computable. The system thus becomes relational. It may not ever produce exactly the same calculation whenever it reads a tape, but the slightly different calculations may be effectively equivalent at a particular scale. That is, at a higher level, the relation can be reduced to a function. Now, let us imagine further that the negentropy principle of information (NPI) comes into play, and we have to ask: Where is the system getting energy to do all these calculations? Let us assume that the energy is itself supplied by Texternal. Let us now generalize the ability of the Probe to move forward or backward, and to say it can move "over" Texternal in discrete steps. Texternal now constitutes a "landscape" over which the probe can move, and from which it gains the resources to maintain itself. Texternal, can now be seen as analogous to an environment that the Probe can wander through, and from which the probe gains the energy required to do work (keep 97
Developmental Systems wandering). If we allow regularities in the environment to have a higher likelihood of creating a change on the Tinternal (internal tape), then we have a system in place, where the Probe can internally model regularities in its environment. As the environment changes, the regularities change, the Probe can rewrite the internal tape to adjust. This is an effectively non-algorithmic procedure, because it requires continuous change of the algorithm. There is no algorithm over large time scales, but there can appear to be an algorithm over short time scales. The probe can function as long as its ability to model its environment is sufficient to provide for its energy needs. It does not have to be an exact model. It does not need to be an optimal model. It just needs to be adequate. The effectiveness of the procedure is now predicated not on the ability for a calculation that halts, but on the ability to obtain enough energy and resources to maintain the systems non-equilibrium state: that is to continue to be able to do work. The "computation" only halts when the system can no longer obtain enough energy to maintain itself. At that point it looses coherence, and is no longer capable of any computation. Scale heterogeneity can be important for "tuning" how such a system responds to regularities in its environment. If it observes its environment at a lower scale of precision than that at which the environment changes, it will only model changes above a particular scale level. Such a method can allow the probe to buffer itself from having to respond to every change in its environment, but only changes that occur above a particular scale. At this point we have arrived at an anticipatory system, that can change internally to compensate for changes in its external conditions. It is also a developmental system. And this leads me to the following conjecture, again under the assumption of NPI. An anticipatory system is capable of non-Turing-computable computations, as long as it can maintain internal cohesion. The unique feature of biology is that the systems of study appear to be teleological. They appear to engage in goal directed behavior, and hence are anticipatory. They can internally reorganize themselves in the face of the unexpected (Collier, 2000), and hence can buffer themselves from unpredictable fluctuations in the environment up to a limit ( a fluctuation so large it wipes the system out, like a random bullet through the heart). Great 98
Developmental Systems circumlocutions occur when biologists attempt to remove all vestiges of teleology from biology, and talk only in terms of structures and dynamics. Such circumlocutions do great violence to the idea that explanations should be parsimonious. However, teleological behavior would arise naturally for a system capable of modeling its environment, capable of using energy obtained from the environment and capable of maintaining its cohesion. The possibility of anticipatory systems was both the motivation of Collier, Hooker, Christensen's development of complexly organized dynamical systems, and also the work underlying "Life Itself" (indeed, an earlier work of Rosen's is titled "Anticipatory Systems"). It is closely related to the concept of "compensatory change" developed by Brooks and Wiley (1988), which focuses on the ability of biological systems to reorganize themselves internally to compensate for external fluctuations. In both cases, what is emphasized is internal change: the ability of a developmental system to "write" regularities in the environment into itself, and to be able to use just those regularities for prediction (anticipation); the ability of a developmental system to reorganize itself in response to environmental fluctuations (compensatory change) to preserve its cohesion. Both anticipation, and compensatory change require the system to be able to do work internally, hence depend on intropy. Through anticipation and compensatory change the system gains "telos" or behaviour towards a final cause, which it proceeds towards even in the face of environmental fluctuations, as long as such fluctuations do not destroy the cohesion of the system (kill it). Such a system, is obviously very different from a computer, from any machine we know today. Indeed, it is not a machine at all. Equally obvious should be the fact that any organism must be an anticipatory system capable of compensatory change. If it were not, it could not actively maintain it's cohesion, and hence could not maintain a degree of autonomy from its environment. If a system's internal state space increases, if it's alphabet can increase, and become more complicated, if the state transitions can be re-defined, then the system essentially has states whose potentiality is not realized, until transition relations among those states are defined. Prior to the transition relations being constrained, the system is vague with respect to those states. As transition relations develop, and move from the relational to the functional, system behaviors move from vague to fuzzy to crisp.
99
Developmental Systems Perhaps we will someday be able to fabricate such a system, and create synthetic rather than organic anticipatory systems. If so, that would be a true "bio-technology". How would we control such systems? Through boundary constraints. However, regardless of the level of boundary constraints we place, there will always be the possibility that such systems will choose to do something else. Their useful work may not be the useful work we intended. In that sense, such systems bear a relation to us, much as our children do. Control would not be absolute, and control would have ethical ramifications. As it always has. There is an account, within mathematics of such an anticipatory system being constrained, to act as an algorithm. Where Turing placed a reading head, and I have placed a probe, Emil Post placed a human computer or "worker". His activities were highly constrained, so that he functioned as an effective Turing machine. However, the constraints are on his external activities, what he must do after reading each square of the tape. A Turing machine is built so constrained, that it only has enough internal degrees of freedom available to it to perform a calculation,, as directed by an external tape and its internal state space. Anyone who has used a real physical computer and had a crash, realizes that it has greater freedom than the idealized Turing machine. A human being of course has even greater internal degrees of freedom. There are internal states that are not being used to do the "effective computation" and hence are free to do other things, such as dream. We must wonder what Post's human computer was thinking; what other computations he was engaged in internally, while externally appearing to be a dutiful and efficient machine. David Berlinski's book "The Advent of the Algorithm" (2000) provides a moving chapter on Post's accomplishments which were eclipsed and shadowed by Turing's, in the chapter "Postscript". Post's "worker" that anticipated Turing's "machine", is even earlier anticipated caught in a non-halting computation in the myth of Sisyphus; a myth the philosopher Albert Camus used to discuss the nature of free will. This is the place I get to, and the place we will stop. Once I get here, I realize I have seen it before, in a post card from Robert Rosen. "Life Itself" is that post card. In it, he uses a branch of mathematics, category theory to develop a relational theory of machines -- that 100
Developmental Systems is Turing machines. Then he moves beyond the relational theory of machines, to model organisms as a specific type of relation between the functions "metabolism", "repair" and "reproduction", such that the relationship is "closed to efficient cause", which means that when you ask "who did the work" the explanation does not refer to mechanical forces outside the organism, but refers back inside the organism. In a metaphorical fashion you could say that when you ask "who", the organism responds "I". As Rosen develops his relational model of an organism, he proceeds to use category theory, but moves out of using it in a sense that corresponds to a relational model of a machine. The final relations Rosen mathematically develops in "Life Itself", are not themselves reducible to a Turing machine, or an algorithm. As such, they are non-Turing-computable mathematical expressions. It took me a while to realize that this is what Rosen was doing. I have tried to show you the path I have taken to gain that understanding. Depending on who you are, the path I have taken (data first) or the path Rosen took (model first) may be more to your liking. Or, perhaps, you see another path altogether, are wondering how we could have missed it, and are even now planning your journey through it. I hope it is a grand adventure! ----------In this essay, I have tried to share the path taken by myself and my colleagues, drawing diverse research programs together into a single linear narrative. To do so, I developed the notion of developmental systems as a generalization of dynamical systems, and the Probe-System-Trace relation, as a way of thinking about developmental systems. However, as in Chaucer's "Canterbury Tales", the true depth is not in the narrative bridge, but in the stories told by the other characters that meet and converse upon the road. In this case, they are Maze's Story, Brooks' Story, Matsuno's Story, Ulanowicz's Story, Smith's Story, Collier's Story, Salthe's Story. There are many other stories, I have not told, or am unaware of, that also occur on the road. Banerjee's Story, as it were, serves as merely an introduction to a much richer body of empirical and theoretical work. Last, but not least, there is Rosen's Story. I have used the work of Robert Rosen, as a foil. His work is an analysis of mathematical models, which deconstructs the mathematical model of a mechanism underlying dynamical systems -- so profoundly useful in the 101
Developmental Systems development of the physical sciences -- and recognizes that it is too restricted to serve biology. He goes on to develop the notion of relational models based on the idea of a component, whose function depends on the context of the system it is part of. My work has been the exploration and synthesis of data, leading to the recognition that biological patterns require a richer mathematical model than that offered by the current notions of functions and sets, and I have tried to sketch parts of that model here. In essence, I have been looking for a few extra degrees of freedom to explain biology. We meet in the middle. Ultimately we build scientific practice upon data and upon a progression of models that help us to organize and abstract data into theories. Darwin said, famously, "All observation is for or against some theory". The progress of models (theories), is the progress of science. What exists in the world is prior, and independent of our ability or inability to explain. Data, is our guide, to the effectiveness of models. New data provides evidence for and against our models. New models often provide hints of where to look for data; create new technologies that extend our senses, and hence the data we can gather. Where is it that we meet? Here. It is sunset. I would like to think Robert Rosen and I meet upon a vast beach at the end of this journey, having come here from different tangents to arrive at a relational biology. I never met him in life, so I would like to think I meet him here. All you who have read this far with me are here too. A few will have turned back. Behind us are rocks, before us water; where we stand now, sand. Our eyes are attracted to a sculpture someone has built just above the high tide level, a piling of heavy rocks into the rough shape of a cylinder, a little higher than a tall woman. We gather around the sculpture. At roughly eye level, an inscription has been engraved, perhaps instructions for a lighthouse, "Lay the Foundations Deep and Massive, Peirce, 98". Salt-breeze has marked the inscription, and obscured the edges, but the writing is still clear. We are all sweat stained and weary. The dusk air chills. We see this vast sea before us, and know we will have to cross it soon. We may sail over it, we may explore its 102
Developmental Systems depths, we may fly above it, we may find land within it. But, we must go forward into new terrain. We will need to build a lighthouse. Right now, we are in the inter-tidal zone. Our feet are a bit wet, we see the sea before us, the land behind, we see the pale fucus washed up on the shore, the clamshells, an occasional snail. We know there is a great deal more we can not see, things stranger than we can guess. We prepare ourselves to go deeper. Each of us may take a different direction forward from here. We can send each other postcards of our travels and explorations. But for now, the water is warm and there is sand in our shoes. We remove our shoes, wade into the shallows and cleanse ourselves. The 21st century stretches ahead of us; vast, salty, warmer than the air.
103
Developmental Systems
Epilogue: "Derived shadow cannot exist without primary shadow. This is proved by the first of this which says: Darkness is the total absence of light, and shadow is the alleviation of darkness and of light, and it is more or less dark or light in proportion as the darkness is modified by the light." -- Leanardo da Vinci, Third Book on Light and Shade
Each cycle he attempts to sort the photos of embryos, fails, is shocked from the floor. In the early cycles he was optimistic -- he knew these species, HE had classified these species, surely he would see the distinctions needed. Later, he grew despondent; it began to dawn on him that he was caught in an infinite loop he could not escape, a cycle that would only halt with his death. Finally, he took comfort in the nature of the failure itself. The precise classifications he had produced, were based on adult specimens that had undergone a process from which emerged the features he used to classify . They emerged from just the messiness and vagueness he saw before him. It was beautiful that such clear distinctions could emerge from such apparent chaos. He marveled. Except for his failure at classification, he could be said to be in every way a model occupant. He completed every other task fully and in perfect synchronicity with the clock. He no longer rebelled against this synchronicity -- his outer actions, the record for the cameras that monitored him showed absolute obedience. He pondered the embryos continuously, their ability to evade all attempts at classification; and in that insight he began to comprehend that whatever happened to his body, whatever external order was imposed short of his demise, his identity was irreducible to a function. He had more degrees of freedom than he had ever imagined possible. One day the door opens. A man is on the other side. He recognizes the man as a much older version of the same individual who handed him the slip defining his function countless cycles ago. He wonders how much he has aged himself, how many cycles have actually passed. The man from outside says to him, "You have closely approximated a function, we have decided to allow you to leave." The occupant, slumps, gathers himself, whispers "Wir mussen wissen, wir werden wissen", then replies simply, "Nothing exists without context", and returns to his cell. 104
Developmental Systems
Acknowledgements
The ideas contained in this paper owe more than I can express to a group of scientists who have been willing to mentor me over a 15-year period. Like any vague set, their specification and membership has flowed with time, and the traces have been identified variously as the "Ames Group", "OCA discussion list", and lately the "Infodynamics Group" I can draw no clear cuts between the ideas I have developed here, and borrowings from a community of scientists willing to share their insights, doubts, faith, and all consuming curiosity about the natural world.
105
Developmental Systems
Postscript: Science and Art, Metaphor and Mythology Or Sitting on a Bench by a Pond Watching the Ducks Go by "It is true that few unscientific people have this particular religious experience. Our poets do not write about it, our artists do not try to portray this remarkable thing. I don't know why. Is no one inspired by our present picture of the universe? This value of science remains unsung by singers: you are reduced to hearing not a song or poem, but an evening lecture about it. This is not yet a scientific age." -- Richard Feynman, The Value of Science
Some things your are born with and can't really explain. Other things you learn. As you live your life, what is innate and what is imprinted become entangled, and you are the alloy of your essential self and what happened to you. This is a poetic way of putting it. A religious person might say what you are born with are gifts from god. A staunch neodarwinian might attempt to explain what you are born with as the consequence of your genes. When science touches upon human nature, it comes into contact with art. The separation of church and state -- science and art -- becomes particularly problematic when one touches upon the essence of human. I wrote this essay primarily for myself. Also for my friends. Finally, for strangers who may find something that sparks their imagination. In this essay, I tell the story of my mentors, the people I learned science from. I have wanted this story told for a very long time, but always assumed someone other than I would tell it So -- I was surprised to see myself telling their story in this essay. I had planned to write a technical paper -- but this essay wanted to come out instead. We are not always in control of our creations. Or at least our conscious minds are not. My position in this essay is odd; I am both reporting on the creations of my teachers, and creating concepts, models, methods, based on their creations. Writing this felt like playing jazz with an ensemble of 20 years of ideas that had suddenly appeared on the stage with me. Reading " Rosencrantz and Guildenstern Guildenstern Are Dead," one sees Tom Stoppard in our time, playing jazz with Shakespeare in his time. The Russian poet, Anna 106
Developmental Systems Akhmatova, wrote, "all right then, get thee to a nunnery", as Ophelia would have heard it from Hamlet's mouth. It seems to be the nature of human art to build upon human art. Poetry is a very old craft, predating the written word. Science, is a much more recent craft, only a few hundred years old in its modern form. Each craft has different rules, require skills to be deployed in divergent ways; but there are also commonalties: both create metaphors, both lead to mythologies. Einstein's boy racing a lightbeam. Laplace's all knowing demon. Maxwell's deft fingered sorting demon. Spencer's summary of Darwin's life work as "nature red in tooth and claw". Lovelock's metaphor of feedback as mother earth, Gaia. Newton's clockwork universe. Wright's shifting balance theory as genes wandering over a landscape full of mountain peaks, valleys, and corridors. Aristotle's homunculus, the little man inside the sperm. Strong metaphors, whether in science or art, seep out of their context into the world at large. They have their own lives irrespective of the thoughts, stratagems, or desires of their discoverers. Often, they seem to hang in the air, waiting to be plucked by the prepared mind, as calculus was plucked by Newton and Leibniz, as evolution by natural selection was plucked by Darwin and Wallace, as the algorithm was plucked nearly simultaneously by Church, Turing and Post. This essay is an odd mix of science and art, as am I. I was born a poet. Or so I believe. Technique was simply the polishing of a jewel that was always there; one that I can't explain, but seem to be able to access at will, and without will. My parents are scientists. So I was raised by, and further trained as a scientist. Poetry is innate to me. Science is my learned art. At first unnatural, it has become a habit, the old familiar bathrobe. It is no longer possible for me to separate the two. The writing of poetry has surely shaped the structure of my thoughts as has the analysis of data, has traced patterns perhaps even at the level of neurological anatomy. Are my natural gifts and my learned skills fractionable, decomposable, separable? To what degree has innateness and circumstance mixed in the development of the pattern that is myself? I am thinking about this innateness and this learnedness that we carry because I am reading Susan Oyama's "The Ontogeny of Information". I am reading about a developmental systems theory; one both similar and unlike the one I have imagined in these pages. 107
Developmental Systems
I am sitting on a park bench by a small duck pond. Reading. Scrawling notes on the back of a draft of this essay. Light and darkness play upon the pages as the weather moves intermittently from sun-warm to cloud-cool. If I sit still, I can feel a slight breeze like a tickle. It is very pleasant to sit here, reading Oyama's book, stopping reading to scribble notes, stopping scribbling to look up and about at the mallards and Canada geese, at the new ducklings and goslings wandering behind their mothers. One particular duckling seems to have a propensity to wander away. I watched it a day ago when it got separated from its siblings and mother, trapped amidst some large rocks while being eyed by some rather large fish. It kept swimming in circles, gradually appearing increasingly panicky. However, it all ended happily. Its mother found it and steered it past the rocks; the large fish ducked back under water. I watched the family's progress, and the same adventurous individual was again wandering away ten minutes later. Was it improperly imprinted? Does it have a defective gene? Or is it simply one curious duckling? The bench I am sitting on has become familiar. The wood is weather-beaten and greyed. Douglas-fir There is a new plaque, though: "Millie Brown. July 9, 1928 to November 19, 1995. Her Table Always Had Space and Her Heart Always Had Room. Rest Peacefully Sweet Millie. We Love You." I try to imagine what kind of person this Millie Brown might have been. It seems an honour to sit on a bench with a plaque dedicated with such love. It has been a month and a half since I started this essay (for which I'd originally scheduled a weekend, to make some quick technical points and write a small paper). Much of that time has been spent sitting on this bench, looking at the external world of ducks, young couples, senior citizens, teenagers on skateboards, overfed squirrels, falling cherry blossoms. Simultaneously I have been lost in the internal world of this essay, moving from concept to concept, skirting a few yawning chasms, struggling for clarity, building bridges, studying Rosen and Peirce, looking for clues within literature and within my imagination. One day I was walking around the pond and found I had lost all external references. When I looked around me, I no longer knew where I was. I could not 108
Developmental Systems place myself in the landscape. That was a curious feeling! It took me a while to gain my bearings again. Today, that creative intensity has decreased. I am mildly exhausted, but can see the end of this essay. For now, I sit and read Oyama's "The Ontogeny of Information", and some related work downloaded from a web site devoted a new book by her and her colleagues. When I began developing the ideas for this essay, my friend Stan Salthe suggested I look at her work. I ignored the suggestion -- what did I care about some philosophical critique of the nature/nurture debate in biology. It didn't seem to have any relation to what I was interested in, a synthesis of ideas drawn from data analysis. As I sent Stan the early drafts of my conceptualization of a developmental system, he became more insistent, "You really should take a look at Oyama's work." So here I am, reading. What I am reading is a careful, closely argued deconstruction of the metaphors biologists use to explain their work to themselves and to others. "The Ontogeny of Information" looks at the way biologists identify the "innate" with genes and genes with our "nature", and the "learned" with environment and environment with our "nurture", and attempt to apportion biological outcomes to one or to the other: nature or nurture. The assumption behind such apportionment is that biological phenomena, and biological entities are clearly fractionable with respect to cause. When such concepts are applied to humans rather than lab rats or fruit flies, metaphor and mythology come into play, and people speak of the genes for criminal behavior, or the role of fetal alcohol syndrome in criminal behavior. In the extreme view, development is seen as the result of a "genetic program", and all causality is attributed to the gene. Or behavior is seen as conditioning, and all causality is attributed to conditioning. "Nature" and "Nurture" become force like constructs, deflecting the trajectory of an individual. In either case, the individual loses several degrees of freedom, begins to be reduced to a function. Development becomes merely the execution of an algorithm. As does individual action. It is this extreme viewpoint that Oyama deconstructs, developing her notion of developmental system as an alternative that focuses on what she calls constructivist-interaction, which seems to mean "context dependant interactions" It's difficult to tell, Oyama is a cagey writer, and I am 109
Developmental Systems finding her constructivist interactions difficult to pin down. A summary of her thesis is given early: (Oyama, 2000, p. 4, italics in original text): It is my contention that developmental information does "develop," not by special creation from nothingness, but always from the conditional transformation of prior structure -- that is, by ontogenetic processes. Depending on the level of analyses, such transformations can be described as resulting from interactions among entities, such as cells or organisms, or from interactions within an entity, such as an embryo or a family. Since sets of interactants can be at least partially nested, a change in focus is sufficient to shift from one perspective to the next. This idea that "information" has a developmental history, doesn't arise spontaneously from nothing, but is itself prepared by empirical and conceptual developments on many fronts: various "interactionisms" in psychology, aspects of systems theories and of Piagetian genetic epistemology, discoveries in molecular and developmental biology (though not much of the most general level of interpretation of these discoveries), cognitive psychology and ethology. It reframes some intractable problems in these fields and offers an alternative way of conceptualizing form and causation, change and variability, normality and necessity. The notion of the ontogenesis of information in what I call developmental systems is implicit in much that is said and done in these sciences today, but because it usually remains implicit, it coexists all to easily with its own contradiction. It is my intent to make it fully explicit, but to do so it is necessary also to render explicit the assumptions supporting the nature-nurture complex. Once the assumptions are clear it becomes easier to see the problems that attend them. There is little here I would disagree with. Her approach begins with critique, but ends with some suggestions for an alternative biology. My approach begins with detailed suggestions for an alternative biology, and leaves the critique of current biology implicit (with the exception of a few one liners that irrepressibly slipped through). In the essay, I attempt to demonstrate exactly why a developing system can not be reduced to a 110
Developmental Systems program, and define within the Probe-System-Trace model a notion of interaction that is extremely constructivist. Her work critiques neo-Darwinian literature. I synthesize 20 years of literature from empirical research programmes that have made a break with rigid neodarwinism (though by no means a break with Darwin). The relation between Oyama's critique of neodarwinian genetic determinism resulting in developmental systems and my construction of developmental systems as a generalization of dynamical systems is unclear at the moment; clarification will come through dialogue, communication, constructive interactions. In reading through Oyama's work and that of her colleagues I am mildly surprised that none of the research programmes I ground this essay, and my construction of developmental systems upon, ever appears. This is particularly surprising given that one of the admitted criticisms of their critique is "where's the beef" as in where's the empirical data to support the notion of developmental systems. Oyama (2000, p.211) neatly side-steps this issue, saying that her work does not make quite the empirical bets that such critics require: One should not, however, seek to articulate the developmental systems approach with models of evolutionary processes with too great a degree of empirical specificity, any more than one should interpret it as a claim, say, about the role of particular environmental factors in this or that developmental process. In the case of the ideas in this essay, and the research programmes that I have described, they all do place rather large empirical wagers. Detailed molecular, morphological, ecological, and phylogenetic field and lab studies take years, and so not only time, but also entire careers are wagered. Collier, Hooker and Christensen's formulation of complexly organized dynamical systems as well as their application of the negentropy principle of information, places large empirical wagers, within a philosophically motivated research programme. What drives people to take such risks? Passion. Curiosity. Ambition. Delight. Stubborness. More curiosity. The joy of finding new questions to explore.
111
Developmental Systems Oyama's work, is part of a growing body of philosophically based critique where the techniques of literary analysis are applied to scientific literature, and where further, scientific metaphors are traced through popular literature4. This makes many scientists rather uncomfortable -- when scientific literature is treated as literature. Science as literature brings science back down (or back up) to the human scale. The extreme view is that science is a social construction, and nothing more. A more moderate view is that every science depends on the ideas and culture of its time. It can not be removed from its times, and treated as a purely objective intellectual pursuit existing without reference to geography, time, culture, circumstance. Ronald Fisher did want to breed better human beings. Sewall Wright did not. Mathematically the formulae they derived in population genetics usually agreed (Provine, 1986) but their interpretations and applications did not. Which brings us back to the human element, and individual context. Science is an exploration of the unknown. As is art. I can't say quite what science is, or what art is. I don't think anyone can make a definitive statement. The difference, I believe, is: A scientific construct requires internal consistency, and an external referent to measureables, a way of encoding and decoding observations into models, what Rosen (1991) has called "the modeling relation". Whereas, art requires internal consistency, and the external referent is the reaction of another person; an appreciation, the tingling of hairs on the palm, a scathing critique -- the transfer of a sensation, thought, feeling, information. To bootstrap ourselves into the unknown, we use metaphors from what we know. We build sandcastles upon metaphors. Deep theories that unify diverse phenomena, easily become mythologies. They help us understand our origins, whether from an eternal garden, the primordial ocean, or a quantum fluctuation in a vacuum. Occasionally the metaphors and mythologies come tumbling back down. Perhaps a few of the metaphors I have used in this essay will tumble down upon me, and give me a sore forehead. Oh well. I am much more interested in exploration at the moment. How to develop vague sets, non-Turing non-machines, the relationship of the Probe-System4
See for example Gillian Beer's study of Darwin's influence on nineteenth century fiction in '"Darwin's
Plots", 1983; see Norris, 1982 p. 18-22, for a capsule summary of the application of literary analysis to philosophy, which is easily extended to scientific literature.
112
Developmental Systems Trace relation to semiotics, the analysis of new data sets with new trace compression algorithms. What I am unable to develop, someone else will, if it is truly worth developing. I am often wondering whether my exploration of something is better expressed as science, or as art. In this essay, I chose to make no distinction. I sit on Millie's bench. Watch ducks, and the seagulls that have just joined them. Feel the breeze that is beginning to chill. Make notes of this moment. When I forget the moment itself, these notes will become the only memory I have; a trace of the original. Many years ago, while running beside another pond in the rain, I imagined Boltzmann's gas equations, the particles suddenly changing, no longer identical ensembles, but marked by their passage, taking on colours, extending into shape, spinning and coalescing about each other, traced. I had been reading works written by Boltzmann in the last year of his life. In trying to understand his equations, allowed myself to be suffused with his spirit. A proud and desperate figure who referred to his equations as gifts brought from the darkness. That moment, so long ago, was the beginning of this essay, which has been part of an unwinding of that particular moment when wet and cold on the outside, I stepped into the inner stillness of Maxwell's poised demon, and watched the gas equations explode into life and transform into something else. The scientific unpacking of that image has taken many years and continues in a slow crabwise fashion, but the poetic unpacking compressed itself into a few hours of marking paper with shaking hands, trying to write before I forgot. And the poem has become my memory of that moment in the rain. Dear readers who have walked with me this far, I leave you the poem behind the science. Thank you. -------
113
Developmental Systems
Shadow on the Wall
I
"Who, where I to scream would hear me there among the angelic hordes." -- R.M. Rilke: Duino Elegies
Demon, have you known beauty; that arc of terror soldering nerve to bone; that pulsing wavefront forever falling into itself. I have read Schiller, weeping as my soul groaned to wake. And Maxwell's equations moved me to wonder: "Was it a God who wrote these signs." Demon, knowing the motion the position, the spin of every least molecule; holding all past and possibility in your pocket like loose change -Demon, have you known beauty? I have.
In moments moments past.
Now, shadows haunt me as I haunt this wreckage of memory. Duino, this banked castle upon a rocky headland along the sea. I Ludwig Boltzmann. O Demon what is to become of me? II
Demon, I was born in twilight as the dying embers of the dance stilled towards Ash Wednesday. 114
Developmental Systems And, I have walked always in dust and twilight joy and despair, each moulded of the other's absence. Now, I am an old man, my blood runs slow, and no heat moulds me. I dwell among abstractions, am misunderstood, have become a name that others rally to or rail against without conception. Yes, these shambling threads still move, still maintain homeostasis or in their age run erect for a passing girl. This posits nothing more than mere physiology. There was a time I could pluck atoms out of the ether. Now I barely recall the equations. III
Demon, Are you there behind my shadow on that wall or are you the edge of light that etches the dark? Demon, are you alive? A living knowledge claims all passions. A living knowledge becomes the hum that outlasts death. I have written books and papers and equations -many, many equations; Have tried to write poetry, but all that came 115
Developmental Systems were sobs and howls that left me shamed. So, I hid my terrors among the numbers. And behind the numbers were the atoms. I watched them dance and listened listened minutely, until my mind rang with their rhythms. This I think -- were all else to be found errors and wrecks that would not cohere -this I think will last. A simple monument: "Ludwig Boltzmann, born, died. S = K log W" This I think may live. IV
Demon, I have known beauty -not the abstract but the particular: the imperfect insect-mimic manifest corrolla of an orchid; a luminescent sea ebbing against a black moonless night; the fine irregular mesh of theory. Demon, pray forgive me the curiosity that faltered. I am left only this final mystery -a little light, a little rain. The rope twists. Darkness opens, I pray I need not return again.
116
Developmental Systems
References Atkins, P.W. (1994). The Second Law: Energy, Chaos, Form. New York: Scientific American Library. Banerjee, S., Sibbald, P.R. and Maze, J. (1990). "Quantifying the Dynamics of Order and Organization in Biological Systems," J. Theor. Biol. 143:91-111. Beer, G. (1983). Darwin's Plots. Evolutionary Narrative in Darwin, George Eliot and Nineteenth-Century Fiction. London: Routledge & Kegan Paul. Berlinski, D. (2000). The Advent of the Algorithm. The Idea That Rules the World. New York: Harcourt, Inc. Brillouin, L. (1962). Science and Information Theory. New York: Academic Press. Brooks, D.R. (1997). "Biological Evolution as a Microcosm of Cosmological Evolution," Bridges 4:9-35. Brooks, D.R. (1998). "The Unified Theory of Evolution and Selection Processes," in Evolutionary Systems: Biological and Epistemological Perspectives on Selection and Self-Organization, (G. van de Vijver, S.N. Salthe and M. Delpos, eds.) Dordrecht: Kluwer Academic Publishers. Brooks, D.R., Collier, J. and Wiley, E.O. (1986). "Definition of Terms and the Essence of Theories: A Reply to J.S. Wicken," Systematic Zoology 35:640-647. Brooks, D.R. and McLennan, D.A. (1991). Phylogeny, Ecology and Behavior: A Research Program in Comparative Biology. Chicago: Univ. Chicago Press. Brooks, D.R. and Wiley, E.O. (1988). Evolution as Entropy: Toward a Unified Theory of Biology. 2nd ed. Chicago: University of Chicago Press. Brulisauer, A.R., Bradfield, G.E. and Maze, J. (1996). "Quantifying Organizational Change After Fire in Lodgepole Pine Forest Understorey," Can. J. Bot. 74:1773-1782. Callen, H.B. (1985). Thermodynamics and an Introduction to Thermostatistics. 2nd Edition. New York: John Wiley and Sons. Chaitin, G.J. (1998). The Limits of Mathematics. Singapore:Springer-Verlag Singapore Pte. Ltd. Chaitin, G.J. (1999). The Unknowable. Singapore: Springer-Verlag Singapore Pte. Ltd.
117
Developmental Systems Christensen, W, Collier, J. and Hooker, C.A. (1997). "Adaptation, Autonomy and Anticipation: Toward Autonomy-theoretic Foundations for Life and Intelligence in Complex Adaptive Self-organising Systems," Preprint. Collier, J. (1986). "Entropy in Evolution," Philosophy and Biology 1:5-24. Collier, J. (1988). "Supervenience and Reduction in Biological Hierarchies," in Biology and Philosophy (M. Matthen and B. Linsky, eds.) Canadian Journal of Philosophy Supplementary, Vol. 14. Collier, J. (1990a). "Intrinsic Information," in Information, Language and Cognition: Vancouver Studies in Cognitive Science, Vol. 1. (P. Hanson, ed.). Oxford: University of Oxford Press. Collier, J. (1990b). "Two Faces of Maxwell's Demon Reveal the Nature of Irreversibility," Stud. Hist. Phil. Sci. 21:257-268 Collier, J. (1996). "Information Originates in Symmetry Breaking," Symmetry: Culture and Science 7: 247-256 Collier, J. (1999). "Causation is the Transfer of Information," in Causation, Natural Laws and Explanation, (H. Sankey, ed.) Dordrecht: Kluwer. Collier, J. (2000). "Dealing With the Unexpected," Preprint for ISSS 2000 Annual Meeting. Collier, J, and Hooker, C.A. (1999). "Complexly Organized Dynamical Systems," Open Systems. 6:241-302. Datta, R.M., Dana, S.K. and Banerjee, S.N. (1960). "Investigations on the Interspecific Hybridization Between the Auto-Tetraploids of the Cultivated Jute Species (Corchorus olitorius Linn. C. capsularis Linn. ) and on the Failure of Viable Seed Formation in Them," Genet. Iber. 12:1-27. Fisher, R.A. (1936). "The Use of Multiple Measurements in Taxonomic Problems," Ann. Eugenics. 7:179-188. Frautschi, S. (1988). "Entropy in an Expanding Universe," in Entropy, Information, and Evolution. New Perspectives on Physical and Biological Evolution, (B.H. Weber, D.J. Depew and J.D. Smith, eds.) Cambridge: The MIT Press. Gibson, J.J. (1986). The Ecological Approach to Visual Perception. Hillsdale: Laurence Erlbaum Associates, Publishers. 118
Developmental Systems Hilborn, R and Mangel, M. (1997). The Ecological Detective. Confronting Models with Data. Princeton: Princeton University Press. Honda, H., Nagao, S., Hatori, K. and Matsuno, K. 1995. "Slow and Macroscopic Modulation of Thermal Fluctuations in Myofibrils," Biophys. Chem. 54:61-66 Kam, Y.K. and Maze, J. (1974). "Studies on the Relationships and Evolution of Supraspecific Taxa Utilizing Developmental Data. II. Relationships and Evolution of Oryzopsis hymenoides, O. virescens, O. kingii, O. micrantha, and O. asperfolia," Botanical Gazette 135:227-247. Kac, M. and Ulam, S.M. (1969). Mathematics and Logic. Retrospect and Prospects. New York: The New American Library. Kauffman, S.A. (1993). The Origins of Order. Self Organization and Selection in Evolution. New York: Oxford University Press. Kestin, J. (1968). A Course in Thermodynamics. Waltham: Blaisdell. Kosko, B. (1992). Neural Networks and Fuzzy Systems. A Dynamical Systems Approach to Machine Intelligence. Englewood Cliffs: Prentice Hall. Kosko, B. (1997). Fuzzy Engineering. Upper Saddle River: Prentice Hall. Langton, C.G. (1991). "Life at the Edge of Chaos," in Artifiical Life II ( C.G. Langton, C. Taylor, J.D. Farmer and S. Rasmussen, eds.) Redwood: Addison-Wesley Publishing Company. Lemke, J. (1991). "Discourse, Dynamics and Social Change," in Language as Cultural Dynamic (M.A.K. Halliday, ed.). Special issue of Cultural Dynamic. Lemke, J. (1999). "Material Sign Processes and Emergent Ecosocial Organization," in Downward Causation. (P.B. Anderson, ed.) In Press. Matsuno, K. (1989). Protobiology: Physical Basis of Biology. Boca Raton: CRC Press. Matsuno, (1992). "Cohesive Interactions for the Emergence of Eukaryotic Cells," in The Origin and Evolution of the Cell (H. Hartman and K. Matsuno, ec. ) Singapore: World Scientific. Matsuno, K. (1997). "Biodynamics for the Emergence of Energy Consumers," Biosystems. 42: 119-127. Matsuno, K. (1998). "Dynamics of Time and Information in Dynamic Time," Biosystems 46:57-71. 119
Developmental Systems Matsuno, K. and Honda, H. (1991). "Information Aspects of Actomyosin Complex," CRC Crit. Rev. Biomed. Eng. 18:311-321. Mattuck, R.D. (1976). A Guide to Feynman Diagrams in the Many-Body Problem.2nd Edition. New York: Dover. May, R.M. (1976). "Simple Mathematical Models With Very Complicated Dynamics," Nature, 261:459-467. Maze, J. (1983). "A Comparison of Cone and Needle Characters in Abies: a Test of a New Theory of Evolution," Can. J. Bot. 611926-1930: Maze, J. (1998). "Studies Into Abstract Properties of Individuals. II. Analysis for Emergence in Populations, Species, and a Species-Pair," Int. J. Plant Sci. 15:687-694. Maze, J., Robson, K.A. and Banerjee, S. (2000). "Studies Into Abstract Properties of Individuals. V. An Empirical Study of Embryo Development in Achnatherum nelsonii and A. lettermanii," SEED 1:(in press). Maze, J., Bohm, L.R. and Beil, C.E. 1972. "Studies on the Relationships and Evolution of Supraspecific Taxa Utilizing Developmental Data. 1. Stipa lemmonii (Gramineae)," Can. J. Bot. 50:2327-2352. Norris, C. (1982). Deconstruction. Theory and Practice. London: Methuen Oyama, S. (2000). The Ontogeny of Information. 2nd Edition. Durham: Duke University Press. Peirce, C.S. (1998). The Essential Writings.(E.C. Moore, ed.). Amherst: Prometheus Books. Penrose. R. (1989). The Emperors New Mind. Concerning Computers, Minds, and the Laws of Physics. New York: Oxford University Press. Penrose, R. Shimony, A., Cartwright, N and Hawking, S. 1997. The Large, the Small and the Human Mind". Cambridge: Cambridge University Press. Provine, W.B. (1986). Sewall Wright and Evolutionary Biology. Chicago: The University of Chicago Press. Robson, K.A. Maze, J. Scagel, R.K. and Banerjee, S. (1993). "Ontogeny, Phylogeny and Intraspecific Variation in North American Abies Mill. (Pinaceae): an Empirical Approach to Organization and Evolution," Taxon 42:17-34 Renyi, A. (1984). A Diary on Information Theory. Chichester: John Wiley and Sons. 120
Developmental Systems Rosen (1985). Anticipatory Systems: Philosophical, Mathematical and Methodological Foundations. New York: Pergamon Press. Rosen, R.(1991). Life Itself. New York: Columbia University Press. Rosen, R. (1985). "Information and Complexity," in Ecosystem Theory for Biological Oceanography (R.E. Ulanowicz and T. Plat, eds.) Ottawa: Department of Fisheries and Oceans. Salthe. S. (1993). Development and Evolution. Complexity and Change in Biology. Cambridge: MIT Press. Salthe, S. (2000). "Ecology and Infodynamics: A Review Essay of R.E. Ulanowicz, 1997, Ecology the Ascendent Perspective," Journ. of Social and Evolutionary Systems 21:In press. Schrodinger, I. (1944). What is Life. Cambridge: Cambridge University Press Shannon, C.E. and Weaver, W. (1949). The Mathematical Theory of Communication. Urbana: University of Illinois Press. Sibbald, P.R., Banerjee, S. and Maze, J. (1989). "Calculating Higher Order DNA Sequence Information Measures," J. Theor. Biol. 136:475-483. Smith, J.D. H. (1988). "A Class of Mathematical Models for Evolution and Hierarchical Information Theory," Preprint. University of Minnesota: Institute for Mathematics and its Applications Preprint" #396. Smith, J.D.H. (1996). "Competition and the Canonical Ensemble," J. Math. Biosci. 133:69-83. Smith, J.D.H. (1998a). "Demography and the Canonical Ensemble," J. Math. Biosci 153:151-161. Smith, J.D.H. (1998b). "Canonical Ensembles, Evolution of Competing Species and the Arrow of Time," in Evolutionary Systems: Biological and Epistemological Perspectives on Selection and Self-Organization, (G. van de Vijver, S.N. Salthe and M. Delpos, eds.) Dordrecht: Kluwer. Smith, J.D.H. (1998c). "Barycentric Algebras, Canonical Distributions and Legendre Transforms," Iowa State University: ISU Mathematics Report Number M98-01 Smith, J.D.H. (1999). "A Macroscopic Approach to Demography," Iowa State University: ISU Mathematics Report Number AM99-05. 121
Developmental Systems Ulanowicz. R.E. (1997a). Ecology, the Ascendent Perspective. New York: Columbia University Press. Ulanowicz, R.E. (1997b). "Limitations on the Connectivity of Ecosystem Flow Networks," in Biological Models, (A. Rinaldo and A. Marni, eds.). Venice: Istituto Veneto de Scienze, Lettere ed Arti. Ulanowicz, R.E. (1999). "Life After Newton: An Ecological Metaphysic," Biosystems 50:127-142. Ulanowicz, R.E. and Abarca-Arenas, L.G. (1997). "An Informational Synthesis of Ecosystem Structure and Function," Ecological Modeling 95:1-10. Ulanowicz, R.E. and Norden, J.S. (1990). "Symmetrical Overhead in Flow Networks," Int. J. Systems Sci. 21:429-437. Van Ness, H.C. (1983). Understanding Thermodynamics. New York: Dover Publications, Inc. Wicken, J.S. (1987). Evolution, Thermodynamics and Information: Extending the Darwinian Paradigm. New York: Oxford University Press. Wolfram, S. (1994). Cellular Automata and Complexity. Collected Papers. Reading: Addison-Wesley Publishing Company.
122
Developmental Systems
Appendix 1 :Example of the Calculations for Trace 3 Generating Equation: Y=2X2+10X+5 Trace 3 Major Loop 1 X
Y
Int X
Major Loop 2 R11:Int Y
R11IntY/IntX R12:IntY/IntX
b1*
C1*=Y-Y*
R21:IntY/IntX
b2*
C2*
2100
8841005
2200
9702005
100
861000
2300
10603005
100
901000
2400
11544005
100
941000
9410
4
2
24005
2500
12525005
100
981000
9810
4
2
25005
10
10
5
2600
13546005
100
1021000
10210
4
2
26005
10
10
5
2700
14607005
100
1061000
10610
4
2
27005
10
10
5
2800
15708005
100
1101000
11010
4
2
28005
10
10
5
2900
16849005
100
1141000
11410
4
2
29005
10
10
5
3000
18030005
100
1181000
11810
4
2
30005
10
10
5
3100
19251005
100
1221000
12210
4
2
31005
10
10
5
3200
20512005
100
1261000
12610
4
2
32005
10
10
5
3300
21813005
100
1301000
13010
4
2
33005
10
10
5
3400
23154005
100
1341000
13410
4
2
34005
10
10
5
3500
24535005
100
1381000
13810
4
2
35005
10
10
5
3600
25956005
100
1421000
14210
4
2
36005
10
10
5
3700
27417005
100
1461000
14610
4
2
37005
10
10
5
3800
28918005
100
1501000
15010
4
2
38005
10
10
5
Appendix Table 1:Trace Calculations
123
Developmental Systems
124
