Parallel neural Circuit SIMulator (PCSIM) - Tutorial - FIAS

Parallel neural Circuit SIMulator (PCSIM) - Tutorial Dejan Pecevski Institute for Theoretical Computer Science Graz University of Technology

FIAS Theoretical Neuroscience Summer School, Frankfurt, August, 2008

Outline of the Tutorial • • • • • •

General Info: What is PCSIM? Features: What is PCSIM useful for? Network elements: neurons and synapses Scalability: Using clusters for parallel simulation. Python Interface PCSIM basic operations  Creating and connecting neurons  Simulating

 Recording signals  Populations and Projections

• Summary Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008

2/40

Who am I? • Ph.D. Student at the Institute for Theoretical Computer Science, Graz University of Technology • One of the developers of PCSIM. • Research Interests  Plasticity and Learning mechanisms in neural circuits  Spiking Neural Networks  Simulation technologies/algorithms for biological neural networks

Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008

3/40

PCSIM in a nutshell • Simulator for parallel simulation of spiking and analog neural networks with point neuron models • Implemented in C++, with Python and Java interfaces • High-level definition of neural networks

• Library of built-in neuron and synapse models • Part of FACETS standardization efforts: the pyNN interface http://www.neuralensemble.org • Open-source, free, released under the GPL license, web page at:

http://www.igi.tugraz.at/pcsim Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008

4/40

Developers Thomas Natschläger Ph.D. Software Competence Center Hagenberg Hagenberg, Austria

Dejan Pecevski DI Institute for theoretical computer science Graz University of Technology Graz, Austria

Other Contributors: Klaus Schuch DI (IGI), Christian Ernstbrunner DI (SCCH), Walter Hargassner DI (SCCH) Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008

5/40

Feature Overview • Object Oriented Design  The user interface/view is object oriented

• General Communication System  Analog and spiking messages  Network elements with multiple input and output ports  Supported in distributed/multi-threaded mode

• Flexible construction of more complex network architectures  Populations of simulated objects  Projections for connectivity specification


6/40

Feature Overview • Parallel simulation  Mixed multi-thread and distributed  Easy to use, transparent to the user

• Easily extensible  A compilation tool for creating PCSIM extension packages

• Usable as backend component in C++, Python, Java, …  Easier and faster setup of simulations in Python and Java

• Runs on Linux or other Unix-like platforms  can be compiled under Windows (still not tested).


7/40

PCSIM high-level structure Utilities in Python NeuroML Parser

Java Interface Python Interface

C++ Network Construction Layer

Built-in neuron and synapse types

BOOST

Simulation Engine

MPI

GSL


8/40

Type of models PCSIM is suitable for • Large recurrent networks with simple point neurons  Distributed and multi-threaded capabilities  Efficient simulation engine in C++  More than 105 spiking neurons and 108 synapses

• Hybrid models  Abstract modules (filters etc.) combined together with circuits of

analog neurons and spiking neurons  Extensible with new custom elements  Closed loop/feedback models


9/40

Type of models PCSIM is suitable for • Structured models  Networks composed of smaller subnetworks (Populations)  Probabilistic connectivity patterns based on neuron’s attributes

• Diversity of neurons and synapses  Different neuron types in the neuron populations  Specifying random distribution for parameter values  New neuron and synapse types can be implemented


10/40

Examples of PCSIM usage • Spiking neural network model of the V1 area in the visual cortex of mammals (Schuch & Rasch) • Testing the computational performance of laminar cortical models based on experimental data. (Schuch & Haeusler)

• Experiments to examine the learning capabilities of rewardmodulated spike-timingdependent plasticity (Legenstein, Pecevski, Maass) Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008

11/40

A simple model represented in PCSIM Model equations dV (t ) V  V (t ) C 2  0 2  I (t )  J 2 dt R

C

dV1 (t ) V0  V1 (t )   J1 dt R

if Vi  VT then V  V0 , spike

dI (t ) s   I (t ) dt

I (t )  I (t ) w after spike

• The equations can be decoupled into separate simulated elements. • Presynaptic neurons send spikes to the synapses. • Synapses inject currents (or modify conductances) in the postsynaptic neuron. Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008

12/40

Network element: basic building block o u tp u t po r ts n e t w o rk e le m e n t n e tw o r k e le m e n t

in p u t po rts

• Nodes of the network (dynamical systems) • Advanced/integrated each time step of the simulation • Represented by class SimObject 

Neurons, synapses, recorders are derived from this class

• Multiple input and output ports, either analog or spiking. • Connections are formed from output to input ports of the same type. Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008

13/40

Neurons and Synapses as Network Elements • Synapses as network elements are attached to the postsynaptic neuron • Synapses have one input port, no output ports • Neurons have only one output, no inputs • The spiking and analog connections can connect neurons on different nodes

p re sy n ap tic ne ur on


s y n a ps e

p o stsy na pt ic n eu ro n

14/40

Built-in Neuron Models Spiking Models

Input Neurons

• LifNeuron, CbLifNeuron

• PoissonInputNeuron • SpikingInputNeuron

Leaky integrate-and-fire • HHNeuron Hodgkin-Huxley Neuron • IzhiNeuron (Izhikevich, 2004) • aeIFNeuron

Analog Neurons • LinearAnalogNeuron

Adaptive Exponential I&F (Brette and Gerstner, 2005) • LinearPoissonNeuron Leaky integrate with Poisson output


15/40

Built-in Synapse Models • Type  current based  conductance based

• PSR Kernel  exponential, alpha,

double-exponential, Square

• Short-term Plasticity (Markram et al. 1998)

• Other Mechanisms  NMDA

(Gabbiani et al. 1994)  GabaB (Mainen et al. 1994)  Reward-modulated STDP (Izhikevich, 2007)  Homeostatic plasticity (Buonomano et al. 2005) 

• Spike-timing-dependent Plasticity  (Froemke and Dan, 2002)  (Gütig et al, 2003)  each pair and nearest

Ornstein-Uhlenbeck noise synapse (Destexhe et al. 2001)

• Analog Synapse

neighbour Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008

16/40

Simulation Strategy • Clock-driven simulation with a fixed time step. • Hybrid Integration Strategy  Neurons are integrated every time step.  Synapses are event driven, i.e. they are integrated only after

receiving a spike.

• Numeric Integration Algorithms  Exponential Euler Method  ODE solvers from GSL


17/40

Parallel Simulation • •

Each MPI process advances subset of neurons MPI as underlying communication layer 

•

spiking and analog messages

Each process can have multiple-threads

PCSIM subnetwork

PCSIM subnetwork

PCSIM subnetwork

MPI

• Transparent to the user  PCSIM subnetwork

Default round-robin distribution of the neurons over nodes  Custom distribution strategies  Easy retrieval of recordings (as in singlethread case)


18/40

Parallel Simulation • Why?  Speed-up simulation of an existing model by using more

processors  Enables simulation of larger models: no memory limit problem  Scale up the number of neurons while keeping the simulation time the same.

• Comparison between single-threaded and distributed simulation  < 104 neurons and 107 synapses on a single machine  > 105 neurons and 108 synapses on clusters  > 106 neurons and 1010 synapses on supercomputers?


19/40

Scalability of PCSIM Hardware: 3.4 Ghz, 512 kb cache, 4GB RAM Model: • 50000 LifNeurons, 4.0 Hz average firing rate • 50106 synapses, 1.5 ms transmission delay


20/40

Scalability of PCSIM Hardware: 3.4 Ghz, 512 kb cache, 4GB RAM Cuba Model: • 5000 LifNeurons per node, 4.0 Hz average firing rate • 103 synapses per neuron, 1.0 ms transmission delay, broad distribution of time synaptic constants


21/40

Extending PCSIM • What are extensions? 

User implemented classes that are plugged into the PCSIM OO framework  usualy coded in C++ (can be coded also in Python)

• Compilation of additional separate Python extension packages, without the need to recompile the main PCSIM code. • The extension classes are automatically wrapped in Python

• Usual extensions for PCSIM 

new neuron/synapse types  other user defined network elements  custom rules for specific network construction


22/40

Python Interface What is Python? (taken from www.python.org) • Python is a dynamic object-oriented programming language • offers strong support for integration with other languages and tools. • comes with extensive standard libraries (“batteries included”). • can be learned in a few days. • Many Python programmers report substantial productivity gains and feel the language encourages the development of higher quality, more maintainable code. • many scientific packages: scipy, numpy, matplotlib, ipython, Rpy etc. Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008

23/40

Python Interface • PCSIM can be imported as a package within Python from pypcsim import *

• The model is represented as a network object: net = SingleThreadNetwork()

• Individual neurons and synapses are accessible as python objects. • Population and Projection objects for high-level construction of networks composed of populations

• Together with the other Python packages for scientific computing provides a complete working environment for neural simulations.


24/40

Creating and connecting neurons # Creates one LIF neuron with default parameter values nrn_model = LifNeuron() nrn_id = net.create( nrn_model ) # Creates array of 10 LIFneurons nrn_array = net.create( LifNeuron(), 10) # Connects the first and the second neuron in the array syn_id = net.connect( nrn_array[0], nrn_array[1], StaticSpikingSynapse())


25/40

Specifying neuron/synapse parameters nrn = net.create(LifNeuron( Cm=2e-10, Rm=1e8, Vthresh=-50e-3, Vresting=-60e-3, Vreset=-60e-3, Trefract=5e-3, Vinit=-60e-3 )) ;

syn = net.connect( pre_nrn, post_nrn, StaticSpikingSynapse( W=1e-10, tau= 5e-3, delay=1e-3 ))


26/40

Recording # record the output spikes of a neuron

rec_spk = net.record( nrn, SpikeTimeRecorder() )

# record the membrane potential rec_vm = net.record( nrn, “Vm”, AnalogRecorder() ) # record the weight of a synapse rec_syn = net.record( syn, “W”, AnalogRecorder() ) # record any field in an element rec = net.record( myElement, “anyFieldName”, AnalogRecorder())


27/40

Simulate the model # simulate the network for 1 second net.simulate(1.0)

# or advance it for 200 time steps net.advance(200)


28/40

Accessing individual neurons/synapses # creates return an ID of the neuron nrn_id = net.create( LifNeuron() ) # get the actual neuron object from the network nrn_object = net.object(nrn_id) # Manipulate the neuron object nrn_object.Vthresh = -59e-3

# getting the recorded values from a recorder values = list(net.object(rec_id).getRecordedValues())


29/40

Synapse classes nomenclature • StaticCurrExpSynapse (StaticSpikingSynapse) Current/conductance based?

• DynamicCondDblExpSynapse

Short-term plasticity

Shape of PSR kernel

• StaticStdpCondAlphaSynapse • DynamicCurrAlphaSynapse • DynamicNMDAAlphaSynapse


30/40

Hello, world! from pypcsim import * net = SingleThreadNetwork() pre_nrn = net.create( LifNeuron(Iinject = 5e-8) ) post_nrn = net.create( LifNeuron(Iinject = 5e-8) ) net.connect( pre_nrn, post_nrn, StaticSpikingSynapse()) rec = net.record( post_nrn, SpikeTimeRecorder() ) net.simulate(0.2) print “spike times:”, list(net.object(rec).getRecordedValues()) Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008

31/40

Specifying inputs # Create input neuron which spikes at specific times net.create(SpikingInputNeuron( [0.1, 0.2, 0.4]))

# Create input neuron with Poisson process spike times net.create( PoissonInputNeuron(rate = 10, duration = 10) )

# Create analog input neuron with an # output analog signal specified as array net.create( AnalogInputNeuron( arange(0,10,1e-4) % 1 ) )


32/40

Creating and connecting many neurons • •

Population of 1000 neurons randomly interconnected with 0.1 probability Population of recorders: one for each neuron in the population

# create a population of 1000 LIF neurons popul = SimObjectPopulation(net, LifNeuron(), 1000) # create the projection proj = ConnectionsProjection( popul, popul, StaticSpikingSynapse(), RandomConnections( 0.1 ) ) # create a recorder population rec_popul = popul.record( SpikeTimeRecorder() )


33/40

Random distributions for parameter values • Factory – an object that generates PCSIM elements • Neuron and synapse objects (used as models for creation) are simple clone factories • SimObjectVariationFactory – factory that attaches random distributions to parameter names nrn_factory = SimObjectVariationFactory( LifNeuron() ) nrn_factory.set( “Vinit”, NormalDistribution( -55e-3, 0.1 ) ) popul = SimObjectPopulation(net, nrn_factory, 1000)


34/40

Spatial heterogenous populations • •

Each neuron has coordinates in 3D space The population is composed of different families of neurons (that can be of different type)

exc_nrn = LifNeuron( Cm = 2e-10, Inoise = 1e-10) inh_nrn = LifNeuron( Cm = 3e-10 ) popul = SpatialFamilyPopulation(net, [ exc_nrn, inh_nrn], RatioBasedFamilies((4,1) ), CuboidIntegerGrid3D(20,10,10)) exc_popul, inh_popul = popul.splitFamilies()


35/40

Distance dependent random connections Creates random connections with probability

p ( D)  C  e



D2

2

where D is the euclidean distance between the two neurons proj = ConnectionsProjection( popul, popul, StaticSpikingSynapse(), EuclideanDistanceRandomConnections( C, lambda ) )


36/40

Saving recorded data from pypcsimplus import * # Create the recording class r = Recordings() # Add the recorder populations as attributes to r r.spikes = popul1.record( SpikeTimeRecorder() ) r.vm_traces = popul2.record( “Vm”, AnalogRecorder() ) # Some additional variables to be saved r.v = 5 # Save the Recordings class to a HDF5 file r.saveInOneH5File(“results.h5”)


37/40

Running a parallel simulation • Change the network class from SingleThreadNetwork to either: MultiThreadNetwork( nThreads = 4 )  DistributedSingleThreadNetwork  DistributedMultiThreadNetwork 

• then run the script with mpirun $ mpirun –n 10 python my_experiment.py


38/40

Summary • PCSIM is a tool for simulation structured random neural networks composed of spiking and analog point neuron models

• Has various built-in neuron and synapse models • Supports parallel simulation: distributed and multi-threaded

• Flexible high-level construction of networks based on probabilistic rules • The Object-oriented framework is designed to support easy extensions • Can be used as a package within the Python interpreting programming language, together with other useful scientific packages Dejan Pecevski, FIAS Theoretical Neuroscience Summer School, August, 2008

39/40

PCSIM Resources • Web page http://www.lsm.tugraz.at/pcsim

• On Sourceforge http://www.sourceforge.net/projects/pcsim • Mailing list http://sourceforge.net/mailarchive/forum.php?forum_name=pcsim-users

• User manual http://www.lsm.tugraz.at/pcsim/usermanual/html/index.html • C++ class reference http://www.lsm.tugraz.at/pcsim/cppclassreference/html/hierarchy.html


40/40

Thank you for your attention!


41/40