Cooperative behavior of artificial neural agents ... - Semantic Scholar

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Cooperative behavior of artificial neural agents based on evolutionary architectures Alessandro Londei, Piero Savastano, Marta Olivetti Belardinelli

Interuniversity Center for Research on Cognitive Processing In Natural and Artificial Systems - ECONA, “Sapienza” University of Rome Email: [email protected]


Empiricism vs. Innatism Empiricism Constructivism: Knowledge cannot be instructed by a teacher, it can only be constructed by a learner (learning by environment) Instructivism: Knowledge is instructed by a teacher (learning by a supervisor)

Innatism Nativism: Knowledge (skills or abilities) is hard-wired into the brain at birth. Certain cognitive modules are built-in at birth that allow to learn and acquire certain skills as language (N. Chomsky, J. Fodor) and other cognitive functions The human mind of a newborn is not a tabula rasa


Stimulus

Empiricism in Neural Networks

Response

Supervisor

Supervised learning: parameters (synaptic weights) are modified until the response is correct At the end of the learning phase, knowledge representation is given by weights distribution… …and this is only true for the chosen neural architecture! Architecture plays a crucial role in the ability of a network to learn a specific behavior


Nativism in Neural Networks?

Stimulus

Evolutionary robotics: agents have a neural brain (fixed feed-forward architecture) whose weights are described by an evolutionary process (genetic algorithm) [Nolfi 1994, Belew 1992]

Response Fitness to the environment

Genetic coding

• architecture is not considered as an evolutionary parameter • phenotype is only given by synaptic weights Low plausibility No internal dynamical evolution Absent learning phase (almost…)


Nativism in Neural Networks! Topology and Weight Evolving Artificial Neural Networks (TWEANNs): optimization of neural systems through augmenting topologies [Stanley 2002]

Drawback One gene for each connection Minimal network means weak robustness to failure


An improved nativist approach

Genetic population

fitness

Plasticity selection Fixed feed-forward architecture

crossing-over

Food

New population

mutations

Architecture selection

Dynamical recurrent architecture


Genetic Coding for a Neural Architecture

Phenotype: - 8 sensorial inputs for: directional food detection (3), directional obstacle detection (2), food proximity (2), hunger proprioceptor (1) - 2 motor outputs Neural Network: - 40 max excitatory/inhibitory neurons with sigmoidal activation function - Hebb (through time) learning rule - Neural transmission delay

Genotype: - 7-length slot per neuron - labeling for connection structure - redundancy (forbidden codes)


Neuron and Network Model Neuron: - Sigmoidal activation function - Beta and offset described in genetic code - Excitatory or inhibitory (from label)

(

)

(

)

(

)

y1 t ! " 1 yk t ! " k yN t ! " N

()

(

xn t = % w kn ! y k t " # k k $S

)

( )

! !

y n = ! xn =

Network: - Connections given by labels similarity (Hamming distance) - Distance defines time-delay - PSP given by synaptic weight and delayed inputs - Hebbian learning rule !wnk = e "# t $ y n t $ y k t " % k

() (

)

1

( (

1" exp # $ xn " T

))


Boolean Gates - AND

11 01

00 10

11 01

00 01

10 01

10 11

00 11

11


Boolean Gates - XOR

10

11 00

10 00

00 01

10 11

10 11

10 01

11


Artificial Neural Agents Environment: a square arena containing 4 agents and 10 incremental food (max 80). Arena sides are sticky and dangerous for the agents. Food increases lifetime by 200 steps and sides wounds the agents (lifetime decreases by 4 at each step) Fitness: is given by the sum of all agents lifetime. fitness = ! LTk k

lifetime

Best agent fitness

Average fitness

Worst agent fitness

generations


Agents Behavior Algorithm demonstration Gen #1: casual behavior Gen #200: sensitivity to food and obstacle Gen #400: emergence of a strategy for area exploration Gen #1000: global optimization of social behavior


Features - Small World Watts Nature, 1998

Characteristic Path Length L(p) typical separation between 2 vertices in the graph Clustering Coefficient C(p) cliquishness of a typical neighborhood example C.Elegans: Lnorm=0.85, Cnorm=0.20

Averaged over the first 10 best agents of each generation


Features - Chaos 3D evolution of 3 inner neurons autonomous case Rabbit Olfactory Bulb - Freeman IEEE Trans. on Circ. and Syst., 1988


Features - Chaos ! $ T 1 log ! Lyapunov Exponent ! = Tlim "# T $ 0

( ) ()


Features - Experimentum Crucis


Features - Experimentum Crucis


Features - Cooperation Fitness takes into account the “global ability to survive” fitness = ! LTk " k

1 ! LTk " LTj 2 k,j


Features - Cooperation


Features - Cooperation Agents behavior increases global lifetime by means of a greater modulation of the speed: agents tend to stop after feeding and begin to search for food again after a “refractory” transient

Difference between speed modulations is significant: F1,99=13.43, p