WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Cooperative behavior of artificial neural agents based on evolutionary architectures Alessandro Londei, Piero Savastano, Marta Olivetti Belardinelli
Interuniversity Center for Research on Cognitive Processing In Natural and Artificial Systems - ECONA, “Sapienza” University of Rome Email:
[email protected]
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Empiricism vs. Innatism Empiricism Constructivism: Knowledge cannot be instructed by a teacher, it can only be constructed by a learner (learning by environment) Instructivism: Knowledge is instructed by a teacher (learning by a supervisor)
Innatism Nativism: Knowledge (skills or abilities) is hard-wired into the brain at birth. Certain cognitive modules are built-in at birth that allow to learn and acquire certain skills as language (N. Chomsky, J. Fodor) and other cognitive functions The human mind of a newborn is not a tabula rasa
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Stimulus
Empiricism in Neural Networks
Response
Supervisor
Supervised learning: parameters (synaptic weights) are modified until the response is correct At the end of the learning phase, knowledge representation is given by weights distribution… …and this is only true for the chosen neural architecture! Architecture plays a crucial role in the ability of a network to learn a specific behavior
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Nativism in Neural Networks?
Stimulus
Evolutionary robotics: agents have a neural brain (fixed feed-forward architecture) whose weights are described by an evolutionary process (genetic algorithm) [Nolfi 1994, Belew 1992]
Response Fitness to the environment
Genetic coding
• architecture is not considered as an evolutionary parameter • phenotype is only given by synaptic weights Low plausibility No internal dynamical evolution Absent learning phase (almost…)
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Nativism in Neural Networks! Topology and Weight Evolving Artificial Neural Networks (TWEANNs): optimization of neural systems through augmenting topologies [Stanley 2002]
Drawback One gene for each connection Minimal network means weak robustness to failure
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
An improved nativist approach
Genetic population
fitness
Plasticity selection Fixed feed-forward architecture
crossing-over
Food
New population
mutations
Architecture selection
Dynamical recurrent architecture
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Genetic Coding for a Neural Architecture
Phenotype: - 8 sensorial inputs for: directional food detection (3), directional obstacle detection (2), food proximity (2), hunger proprioceptor (1) - 2 motor outputs Neural Network: - 40 max excitatory/inhibitory neurons with sigmoidal activation function - Hebb (through time) learning rule - Neural transmission delay
Genotype: - 7-length slot per neuron - labeling for connection structure - redundancy (forbidden codes)
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Neuron and Network Model Neuron: - Sigmoidal activation function - Beta and offset described in genetic code - Excitatory or inhibitory (from label)
(
)
(
)
(
)
y1 t ! " 1 yk t ! " k yN t ! " N
()
(
xn t = % w kn ! y k t " # k k $S
)
( )
! !
y n = ! xn =
Network: - Connections given by labels similarity (Hamming distance) - Distance defines time-delay - PSP given by synaptic weight and delayed inputs - Hebbian learning rule !wnk = e "# t $ y n t $ y k t " % k
() (
)
1
( (
1" exp # $ xn " T
))
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Boolean Gates - AND
11 01
00 10
11 01
00 01
10 01
10 11
00 11
11
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Boolean Gates - XOR
10
11 00
10 00
00 01
10 11
10 11
10 01
11
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Artificial Neural Agents Environment: a square arena containing 4 agents and 10 incremental food (max 80). Arena sides are sticky and dangerous for the agents. Food increases lifetime by 200 steps and sides wounds the agents (lifetime decreases by 4 at each step) Fitness: is given by the sum of all agents lifetime. fitness = ! LTk k
lifetime
Best agent fitness
Average fitness
Worst agent fitness
generations
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Agents Behavior Algorithm demonstration Gen #1: casual behavior Gen #200: sensitivity to food and obstacle Gen #400: emergence of a strategy for area exploration Gen #1000: global optimization of social behavior
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Features - Small World Watts Nature, 1998
Characteristic Path Length L(p) typical separation between 2 vertices in the graph Clustering Coefficient C(p) cliquishness of a typical neighborhood example C.Elegans: Lnorm=0.85, Cnorm=0.20
Averaged over the first 10 best agents of each generation
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Features - Chaos 3D evolution of 3 inner neurons autonomous case Rabbit Olfactory Bulb - Freeman IEEE Trans. on Circ. and Syst., 1988
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Features - Chaos ! $ T 1 log ! Lyapunov Exponent ! = Tlim "# T $ 0
( ) ()
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Features - Experimentum Crucis
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Features - Experimentum Crucis
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Features - Cooperation Fitness takes into account the “global ability to survive” fitness = ! LTk " k
1 ! LTk " LTj 2 k,j
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Features - Cooperation
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)
Features - Cooperation Agents behavior increases global lifetime by means of a greater modulation of the speed: agents tend to stop after feeding and begin to search for food again after a “refractory” transient
Difference between speed modulations is significant: F1,99=13.43, p