Cooperative behavior of artificial neural agents ... - Semantic Scholar

3 downloads 0 Views 5MB Size Report
An improved nativist approach. Genetic population fitness crossing-over mutations. New ... Fitness: is given by the sum of all agents lifetime. fitness = LT.
WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Cooperative behavior of artificial neural agents based on evolutionary architectures Alessandro Londei, Piero Savastano, Marta Olivetti Belardinelli

Interuniversity Center for Research on Cognitive Processing In Natural and Artificial Systems - ECONA, “Sapienza” University of Rome Email: [email protected]

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Empiricism vs. Innatism Empiricism Constructivism: Knowledge cannot be instructed by a teacher, it can only be constructed by a learner (learning by environment) Instructivism: Knowledge is instructed by a teacher (learning by a supervisor)

Innatism Nativism: Knowledge (skills or abilities) is hard-wired into the brain at birth. Certain cognitive modules are built-in at birth that allow to learn and acquire certain skills as language (N. Chomsky, J. Fodor) and other cognitive functions The human mind of a newborn is not a tabula rasa

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Stimulus

Empiricism in Neural Networks

Response

Supervisor

Supervised learning: parameters (synaptic weights) are modified until the response is correct At the end of the learning phase, knowledge representation is given by weights distribution… …and this is only true for the chosen neural architecture! Architecture plays a crucial role in the ability of a network to learn a specific behavior

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Nativism in Neural Networks?

Stimulus

Evolutionary robotics: agents have a neural brain (fixed feed-forward architecture) whose weights are described by an evolutionary process (genetic algorithm) [Nolfi 1994, Belew 1992]

Response Fitness to the environment

Genetic coding

• architecture is not considered as an evolutionary parameter • phenotype is only given by synaptic weights Low plausibility No internal dynamical evolution Absent learning phase (almost…)

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Nativism in Neural Networks! Topology and Weight Evolving Artificial Neural Networks (TWEANNs): optimization of neural systems through augmenting topologies [Stanley 2002]

Drawback One gene for each connection Minimal network means weak robustness to failure

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

An improved nativist approach

Genetic population

fitness

Plasticity selection Fixed feed-forward architecture

crossing-over

Food

New population

mutations

Architecture selection

Dynamical recurrent architecture

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Genetic Coding for a Neural Architecture

Phenotype: - 8 sensorial inputs for: directional food detection (3), directional obstacle detection (2), food proximity (2), hunger proprioceptor (1) - 2 motor outputs Neural Network: - 40 max excitatory/inhibitory neurons with sigmoidal activation function - Hebb (through time) learning rule - Neural transmission delay

Genotype: - 7-length slot per neuron - labeling for connection structure - redundancy (forbidden codes)

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Neuron and Network Model Neuron: - Sigmoidal activation function - Beta and offset described in genetic code - Excitatory or inhibitory (from label)

(

)

(

)

(

)

y1 t ! " 1 yk t ! " k yN t ! " N

()

(

xn t = % w kn ! y k t " # k k $S

)

( )

! !

y n = ! xn =

Network: - Connections given by labels similarity (Hamming distance) - Distance defines time-delay - PSP given by synaptic weight and delayed inputs - Hebbian learning rule !wnk = e "# t $ y n t $ y k t " % k

() (

)

1

( (

1" exp # $ xn " T

))

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Boolean Gates - AND

11 01

00 10

11 01

00 01

10 01

10 11

00 11

11

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Boolean Gates - XOR

10

11 00

10 00

00 01

10 11

10 11

10 01

11

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Artificial Neural Agents Environment: a square arena containing 4 agents and 10 incremental food (max 80). Arena sides are sticky and dangerous for the agents. Food increases lifetime by 200 steps and sides wounds the agents (lifetime decreases by 4 at each step) Fitness: is given by the sum of all agents lifetime. fitness = ! LTk k

lifetime

Best agent fitness

Average fitness

Worst agent fitness

generations

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Agents Behavior Algorithm demonstration Gen #1: casual behavior Gen #200: sensitivity to food and obstacle Gen #400: emergence of a strategy for area exploration Gen #1000: global optimization of social behavior

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Features - Small World Watts Nature, 1998

Characteristic Path Length L(p) typical separation between 2 vertices in the graph Clustering Coefficient C(p) cliquishness of a typical neighborhood example C.Elegans: Lnorm=0.85, Cnorm=0.20

Averaged over the first 10 best agents of each generation

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Features - Chaos 3D evolution of 3 inner neurons autonomous case Rabbit Olfactory Bulb - Freeman IEEE Trans. on Circ. and Syst., 1988

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Features - Chaos ! $ T 1 log ! Lyapunov Exponent ! = Tlim "# T $ 0

( ) ()

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Features - Experimentum Crucis

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Features - Experimentum Crucis

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Features - Cooperation Fitness takes into account the “global ability to survive” fitness = ! LTk " k

1 ! LTk " LTj 2 k,j

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Features - Cooperation

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Features - Cooperation Agents behavior increases global lifetime by means of a greater modulation of the speed: agents tend to stop after feeding and begin to search for food again after a “refractory” transient

Difference between speed modulations is significant: F1,99=13.43, p