Reservoir Computing in materio with LEDs…? Matthew Dale1*, Julian Miller, Susan Stepney1 & Martin Trefzer2 University of York, Departments of Computer Science1 and Electronics2
Email:
[email protected]
Introduction
LED Results
In [1] we discuss Reservoir Computing (RC) with evolvable substrates. In [2,3] we show that the combination not only offers broader applications but also offers competitive performance to other training techniques and in silico reservoirs.
Early results show very promising performances, with an NRMSE for the NARMA-10 task around 0.53 (compared to earlier work in [2]). For comparison, a 50-node Optoelectronic reservoir reaches an NRMSE ≈ 0.41 [4] and for a 20-node simulated network an NRMSE ≈ 0.56 [5] is reported.
In this novel experiment we attempt to exploit nonlinearity in simple Light Emitting Diode (LED) circuits for Reservoir Computing.
Hardware: Light Emitting Diodes (LED) Two test substrates were used: a) a 64-LED printed circuit board (PCB), and b) four separate 16-LED PCB’s. Each board contains one ground that joins all LEDs and the resistor value for each LED is identical.
LED Array
0.025/0.107 0.003/0.003
And for the Santa Fe Laser timeseries, we outperform other published work, including some in silico Echo State Networks [6].
0.038/0.119
0.023/0.045
Reservoir Type
For the Wave Generator task, the LED array greatly outperformed previous work [2].
NRMSE
Nodes
Echo State Network (evolved) [3]
0.098
50
Multi-LED Reservoir
0.12
30
Echo State Network [6]
0.134
50
Optoelectronic (numerical) [7]
0.141
200
64-LED Reservoir
0.15
30
Mackey-Glass (Electronic) [8]
0.151
50
Reservoir Computing in materio [3]
0.195
7
Echo State Network (evolved & sampled) [3]
0.205
50 (7)
Echo State Network (evolved) [3]
0.235
7
Optoelectronic (experimental) [8]
0.35
400
Sounds too good to be true?… It is! (almost) Why?
Isolating the Substrate from the System
For evolvable systems defining what is contributing to the evolved solution can be tricky. Evolutions ability to exploit everything and anything can be both its strength and annoyance – just ask anyone who’s worked with it! Therefore defining the system boundaries, or what is to be considered part of the solution is critical. In this work, evolution found and largely exploited a capacitive effect within the analogue-to-digital conversion (ADC). The capacitance, represented by signal “ghosting”, was unintentionally providing memory and additional complexity to the readout states. This then led to the very competitive performances seen above.
“Ghosting” The ghosting effect occurred because of high and infinite-impedances from the substrate. This resulted in a capacitive loading on the multiplexed ADCs which caused signal reflection and delays across ADC channels. Source images: National Instruments Support: “How Do I Eliminate Ghosting from My Measurements?“, [online]:http://digital.ni.com/public.nsf/allkb/73CB0FB296814E2286256FFD00028DDF
High-Impedance: Signal Infinite-Impedance (open): reflection across channels . Reflection + delay.
[1] M. Dale, J. F. Miller, S. Stepney. ”Reservoir Computing as a Model for In Materio Computing.” in, Advances in Unconventional Computing, vol 1, pp.533-571, Springer, 2017 [2] M. Dale, J. F. Miller, S. Stepney, and M. Trefzer, “Evolving carbon nanotube reservoir computers,” in International Conference on Unconventional Computation and Natural Computation. Springer, 2016, pp. 49–61. [3] M. Dale, S. Stepney, J. F. Miller, and M. Trefzer, “Reservoir computing in materio: An evaluation of configuration through evolution,” In Proceedings of IEEE International Conference on Evolvable Systems, 2016. [4] Y. Paquot, F. Duport, A. Smerieri, et al. “Optoelectronic Reservoir Computing.”, Scientific Reports;2:287. doi:10.1038/srep00287. 2012
[5] J. Herbert. "Adaptive Nonlinear System Identification with Echo State Networks." In Advances in Neural Information Processing Systems, pp. 609-616. 2003. [6] A. Rodan and P. Tino, “Minimum Complexity Echo State Network,” IEEE transactions on Neural Networks, vol. 22, no. 1, pp. 131-144, 2011. [7] R. Nguimdo, et al. "Reducing the phase sensitivity of laser-based optical reservoir computing systems. " Optics express 24.2: 1238-1252. 2016. [8] Appeltant, Lennert. "Reservoir computing based on delay-dynamical systems." Thesis. Vrije Universiteit Brussel/ Universitat de les Illes Balears. 2012.
To overcome ghosting, buffers were placed between the substrate and the ADCs. To compensate for further noise/crosstalk, the training process was modified to form an averaged state over multiple recordings from the substrate, with a set tolerance. The results of the isolated system show a decrease in performance compared to those above – as expected. However, the isolated system shows improved performance over previous published work. This work highlighted a significant issue that verifying substrate isolation should be paramount to any experiment.
