Neuromodulated Synaptic Plasticity on the SpiNNaker Neuromorphic SystemCitation formats

Standard

Neuromodulated Synaptic Plasticity on the SpiNNaker Neuromorphic System. / Mikaitis, Mantas; Pineda García, Garibaldi; Knight, James C.; Furber, Stephen.

In: Frontiers in Neuroscience, Vol. 12, 27.02.2018.

Research output: Contribution to journalArticlepeer-review

Harvard

APA

Vancouver

Author

Mikaitis, Mantas ; Pineda García, Garibaldi ; Knight, James C. ; Furber, Stephen. / Neuromodulated Synaptic Plasticity on the SpiNNaker Neuromorphic System. In: Frontiers in Neuroscience. 2018 ; Vol. 12.

Bibtex

@article{94685b80294747bd8224f28a95da3bf2,
title = "Neuromodulated Synaptic Plasticity on the SpiNNaker Neuromorphic System",
abstract = "SpiNNaker is a digital neuromorphic architecture, designed specifically for the low power simulation of large-scale spiking neural networks at speeds close to biological real-time. Unlike other neuromorphic systems, SpiNNaker allows users to develop their own neuron and synapse models as well as specify arbitrary connectivity. As a result SpiNNaker has proved to be a powerful tool for studying different neuron models as well as synaptic plasticity -- believed to be one of the main mechanisms behind learning and memory in the brain. A number of Spike-Timing-Dependent-Plasticity(STDP) rules have already been implemented on SpiNNaker and have been shown to be capable of solving various learning tasks in real-time. However, while STDP is an important biological theory of learning, it is a form of Hebbian or unsupervised learning and therefore does not explain behaviours that depend on feedback from the environement. Instead, learning rules based on neuromodulated STDP (three-factor learning rules) have been shown to be capable of solving reinforcement learning tasks in a biologically plausible manner. In this paper we demonstrate for the first time how a model of three-factor STDP, with the third-factor representing spikes from dopaminergic neurons, can be implemented on the SpiNNaker neuromorphic system. Using this learning rule we first show how reward and punishment signals can be delivered to a single synapse before going on to demonstrate it in a larger network which solves the credit assignment problem in a Pavlovian conditioning experiment. Because of its extra complexity, we find that our three-factor learning rule requires approximately 2x as much processing time as the existing SpiNNaker STDP learning rules. However, we show that it is still possible to run our Pavlovian conditioning model with up to 10000 neurons in real-time, opening up new research opportunities for modelling behavioural learning on SpiNNaker.",
keywords = "Neuromodulation, STDP, SpiNNaker, three-factor learning rules, reinforcement learning, behavioural learning",
author = "Mantas Mikaitis and {Pineda Garc{\'i}a}, Garibaldi and Knight, {James C.} and Stephen Furber",
year = "2018",
month = feb,
day = "27",
doi = "10.3389/fnins.2018.00105",
language = "English",
volume = "12",
journal = "Frontiers in Neuroscience",
issn = "1662-4548",
publisher = "Frontiers Media S. A.",

}

RIS

TY - JOUR

T1 - Neuromodulated Synaptic Plasticity on the SpiNNaker Neuromorphic System

AU - Mikaitis, Mantas

AU - Pineda García, Garibaldi

AU - Knight, James C.

AU - Furber, Stephen

PY - 2018/2/27

Y1 - 2018/2/27

N2 - SpiNNaker is a digital neuromorphic architecture, designed specifically for the low power simulation of large-scale spiking neural networks at speeds close to biological real-time. Unlike other neuromorphic systems, SpiNNaker allows users to develop their own neuron and synapse models as well as specify arbitrary connectivity. As a result SpiNNaker has proved to be a powerful tool for studying different neuron models as well as synaptic plasticity -- believed to be one of the main mechanisms behind learning and memory in the brain. A number of Spike-Timing-Dependent-Plasticity(STDP) rules have already been implemented on SpiNNaker and have been shown to be capable of solving various learning tasks in real-time. However, while STDP is an important biological theory of learning, it is a form of Hebbian or unsupervised learning and therefore does not explain behaviours that depend on feedback from the environement. Instead, learning rules based on neuromodulated STDP (three-factor learning rules) have been shown to be capable of solving reinforcement learning tasks in a biologically plausible manner. In this paper we demonstrate for the first time how a model of three-factor STDP, with the third-factor representing spikes from dopaminergic neurons, can be implemented on the SpiNNaker neuromorphic system. Using this learning rule we first show how reward and punishment signals can be delivered to a single synapse before going on to demonstrate it in a larger network which solves the credit assignment problem in a Pavlovian conditioning experiment. Because of its extra complexity, we find that our three-factor learning rule requires approximately 2x as much processing time as the existing SpiNNaker STDP learning rules. However, we show that it is still possible to run our Pavlovian conditioning model with up to 10000 neurons in real-time, opening up new research opportunities for modelling behavioural learning on SpiNNaker.

AB - SpiNNaker is a digital neuromorphic architecture, designed specifically for the low power simulation of large-scale spiking neural networks at speeds close to biological real-time. Unlike other neuromorphic systems, SpiNNaker allows users to develop their own neuron and synapse models as well as specify arbitrary connectivity. As a result SpiNNaker has proved to be a powerful tool for studying different neuron models as well as synaptic plasticity -- believed to be one of the main mechanisms behind learning and memory in the brain. A number of Spike-Timing-Dependent-Plasticity(STDP) rules have already been implemented on SpiNNaker and have been shown to be capable of solving various learning tasks in real-time. However, while STDP is an important biological theory of learning, it is a form of Hebbian or unsupervised learning and therefore does not explain behaviours that depend on feedback from the environement. Instead, learning rules based on neuromodulated STDP (three-factor learning rules) have been shown to be capable of solving reinforcement learning tasks in a biologically plausible manner. In this paper we demonstrate for the first time how a model of three-factor STDP, with the third-factor representing spikes from dopaminergic neurons, can be implemented on the SpiNNaker neuromorphic system. Using this learning rule we first show how reward and punishment signals can be delivered to a single synapse before going on to demonstrate it in a larger network which solves the credit assignment problem in a Pavlovian conditioning experiment. Because of its extra complexity, we find that our three-factor learning rule requires approximately 2x as much processing time as the existing SpiNNaker STDP learning rules. However, we show that it is still possible to run our Pavlovian conditioning model with up to 10000 neurons in real-time, opening up new research opportunities for modelling behavioural learning on SpiNNaker.

KW - Neuromodulation

KW - STDP

KW - SpiNNaker

KW - three-factor learning rules

KW - reinforcement learning

KW - behavioural learning

U2 - 10.3389/fnins.2018.00105

DO - 10.3389/fnins.2018.00105

M3 - Article

VL - 12

JO - Frontiers in Neuroscience

JF - Frontiers in Neuroscience

SN - 1662-4548

ER -