Introduction to exploration and exploitation search ...

Introduction to exploration and exploitation search with examples An addendum to “Inventive processes in nature” by Aleš Kralj 23.1.2017 Trial and error cognitive thinking was first proposed in 1855 by Alexander Bain (Bain, A. (1855) The Senses and the Intellect. London, Parker, Son, p. 575). Interestingly his mention of evolutionary concept precedes that of Darwin from 1860. This was opening into the journey of knowledge that led us to the modern cybernetic understandings of all forms of evolution. We all know what exploration is, right? Well, maybe we have some idea of it, but what does exploration mean in cybernetics it is likely not so clear. Exploration and exploitation are two distinctive search mechanisms in evolutionary optimisation algorithms. Now, don't let the word "algorithm" sway you into thinking that these processes are somehow isolated to something done with computers. Quite the opposite. These mechanisms were first identified by evolutionary biologists in 1950’, soon after genetic mechanisms of evolution were discovered. Even before these mechanisms morphed into digital computers in 1990’, through the 1950’ and 1970’s economists realised that technological development resembles that of biologic evolution and terms "exploration and exploitation" were first proposed in the economics of innovation in technology. Let us return to the 1950'. Geneticists realised that information contained in genes was now and then modified by mutations which caused the appearance of new modifications of life. This new variation of life had vanishingly small chance of survival and chance that this new entity has a fitness improvement over the previously know inventions of nature was very small indeed. Mutations have a few properties. They are somewhat random-like (not entirely random though as mutations depend on the noisy environment that directs such mutations into random like changes to the information). They generate changes that can reach any combination from the combinatorial space. This makes probability that a mutation search point would end up near an existing solution remote. This is why mutation exploratory search is also referred to as far-search. This far-search does not need any previous knowledge. Exploration thus boldly goes into the unknown. If the new solution has at least sufficient fitness, its information will remain preserved. If not it will be deleted. It took until Hilario and Gogarten's work in 1993 that horizontal gene transfer becomes understood. Genetic information crossover in sexual reproduction was of course known beforehand, but was poorly understood. Its function in protecting the genetic material was quickly recognised; its function in execution of exploitation evolutionary search has become clearer after Hilario and Gogarten's work combined with experiences of in-silico computational experimentation with evolutionary search algorithms that become widespread in 1990'. As mentioned above noisy (mutational) exploration requires no beforehand knowledge. In chemical evolution, this exploratory search mode exists in nature even without dedicated information storage device such as DNA or RNA. There, molecular configurations themselves store chemical configurational information (=configurational thermodynamic entropy). When such exploratory search identifies some fit solutions, an algorithm can be devised that takes advantage of this existing

knowledge. And indeed stored information on previous experiments is considered knowledge even outside the cybernetics. Now, it is the job of exploratory search to "invent" exploitation mechanism that enables competing entities to profit from stored knowledge. Bio-cybernetic devices (organisms) store knowledge in the informational polymers, most notably in the RNA and DNA. How exactly is this exploitation achieved? Various DNA parts store information on various complete solutions. Transferring, exchanging or adding a complete chunk of DNA to a new experiment would mean that new entity might assume properties that some other being possessed. A notorious example of this horizontal gene exchange is flu virus exchange of genetic material in swine, which makes them transferrable to human hosts (swine flu). Nature invented numerous ways how to exchange these complete segments of prior knowledge. Most known is sexual reproduction where the genetic material of relatives is transferred. Why relatives? Well, nature wanted to make sure the offspring would have a high chance of survival and that changes to the phenotype would be small, incremental and seemingly continuous. This is exploitation near-search. Exploitation is capable of limited far search as horizontal gene transfer can be achieved in nonsexual ways which would overcome safeguards of sexuality, but these events are rare. Nevertheless, even this exploitation far-search is limited to knowledge range previously identified by exploration. It cannot extend beyond it. Why is exploitation search so much more advantageous than exploratory-far search? Mutation search might yield an improved solution in one in a billion for example. Exploitation search might yield an improvement in just one in ten trials. And nearly all subjects survive. Advantage increase might not be as radical as would be achieved in far-search but to beat competitors it is sufficient if you are 5% more fit. No need to be 300% more fit to win local struggle for the evolutionary fitness. And indeed genetic mutation rates first decreased when evolution moved from RNA to DNA and later again when sexual crossover was introduced (Drake et al. 1998). This inevitably means that exploratory far-search was pushed aside. Changes in mutations rates were significant. Typically three orders of magnitude per evolutionary search mode change. This is all about biological cybernetics. What about cognition, technology and AI? All of them have been identified to employ exploration and exploitation principles when dealing with information and optimisation/adaptation. In the context of cybernetics, adaptation is achieved through optimisation. Cognition in particular in humans obeys principles described above. When a child is born, it possesses no prior knowledge. A newborn child starts a noisy search. He makes random like experimental movements which it evaluates and slowly builds knowledge in its neural network. Later in life we also store highly abstract knowledge which again originates in experimentation. Prior knowledge is again used thorough exploitation. More on cybernetics of cognitive thought experimentation can be found in (Maël Donoso et al. 2014). As you expected technological evolution relies on far and near search. Until 1900 search was predominantly far-search as discovery level inventions were accumulated. A few examples of discovery level inventions: electric power, wire and wireless messaging, fossil fuel power, pharmacy and many others. Until about the year 1900 a sufficient number of these far search points were accumulated that it became clear that their refinements through systematic research could be more advantageous. Surely it was evident that improvements to the automobile make more sense than searching for an entirely alternate mode of transportation.

We, humans, evaluate inventions through our own evolutionary fitness. An invention that by utilisation cannot enhance user's fitness will be rejected and deleted from use. This is how biologic natural selection finds its way into technology through back door. Improvements to the existing ideas seem to be the way to go. This is how exploitation slowly replaced exploration in technology (more on this: D. Strumsky, J. Lobo (2015), Youn et al. 2015). Economists, completely oblivious of cybernetic background mechanics behind it, even invented a new theory of economic growth: Endogenous growth theory. “Endogenous growth” is an idea that through investment into targeted R&D best growth results can be achieved. The downside of this paradigm is that it called for near complete abandonment of farsearch exploration, which actually happened as evidenced in Youn's patent analysis (Youn et al. 2015). Far-search was deemed too risky and uneconomical. No one seems to have noticed that such overuse of exploitation necessarily ends up in ever smaller improvements in consecutive inventions. There are efficiency limits imposed by physical laws that can be approached but not overcome. There are now numerous examples of stagnant fields of technology which have now contributed to the global growth stagnation (see Gordon (2012) and Buchanan (2015) and my working paper on Inventive processes). I saved examples in the AI for last. Such use is usually referred to as evolutionary computation or even better evolutionary algorithms (EA) (see Črepinšek et al. 2013). New inventions in technology can be identified in this way (Koza, J. R., et. al. (1999)). In general, EA is used in various applications where a previously unknown solution to a problem has to be located within a predefined exploration space. First known practical use of EA was adaptive digital acoustic filters on the Los Angeles submarines in late 1980’. These were intended to filter out possible unforeseeable noisy countermeasures that might be employed by the Soviet submarines. Nowadays, use of EA moves to various network optimisation jobs and most notably to the intelligence behind the self-driving cars. There they have to mimic driver's improvisation capacity. With exception of sensing the road in extreme weather, this has proven the most elusive goal of autonomous driving.

Does nature make jumps or does it not ? - ResearchGate. Available from: https://www.researchgate.net/post/Does_nature_make_jumps_or_does_it_not/2 [accessed Dec 23,2016].